Subscribe
Share
Share
Embed
Screaming in the Cloud with Corey Quinn features conversations with domain experts in the world of Cloud Computing. Topics discussed include AWS, GCP, Azure, Oracle Cloud, and the "why" behind how businesses are coming to think about the Cloud.
Harjot: Almost always testing some frontier model for these labs. Uh, using our evals. We kind of have the, one of the best evals in the industry when it comes to reasoning models. Given that. Code reviews are a reasoning heavy use case, and also around the agent tick flows because as you go deeper and deeper into agent tick flows, your errors compound.
So these long horizon tasks, if you're like, go off track on the first step, you're gonna be way off in the like 10th step, right? So, so yeah, it's all about like evaluating. So we have a team that spends a lot of time looking at open source usage that we have, bringing those examples in to make sure we have the, the right evaluation framework to understand the behavior of these new models as they come out.
Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today I'm joined by Harjot Gill, CEO and co-founder of Code Rabbit Harjot. Uh, you are a now three times entrepreneur who went from Nutanix Senior Director. We were building what is now apparently the most installed AI application on GitHub and GitLab. If I'm reading this correctly, uh, what.
Made you leave a big comfortable tech job to decide, you know what I really want to do next? Code review.
Harjot: Thanks for having me here, Cody. I mean, it's been a very interesting journey for me
Corey: coding faster with AI tools, you need faster, better code reviews to keep up and to keep out the AI slop code. Rabbit delivers senior engineer level code reviews on pull requests, and write in your IDE with less LGTM, uh, get context aware feedback for every commit.
No configuration needed and all programming languages you care to use are supported from individuals to enterprises. Join the millions you trust Code Rabbit to banish bugs and help ship better code faster. Get started@coderabbit.ai and instantly cut your code, review time and bugs in half. So I've done these startups
Harjot: for a while.
I actually. Uh, started my first company back in 2015, um, out of my research that I was doing at University of Pennsylvania at that time, and that was called sil. And it was in the early time. I mean, you would remember that because this was the time of Docker, Kubernetes and all these like microservices taking off and.
We built like this product that was like, listen back backward net, like so we could understand the API calls and understand, uh, the performance. That was the first startup, which was acquired by Nutanix, and I was there for like a few years. Uh, never been like a big company person. So yeah, I mean, so we had to go there, ate the product.
Um, then I quit to start another startup, which was Flux Ninja, uh, which was in the. Reliability management space. And the idea was like, how do we go beyond observability into more controllability, prevent cascading failures and all these Black Friday outages and so on. Yeah. So been like a very interesting like, rate limiting and, um, load shedding kind of a solution where you could prioritize API calls.
Now, unfortunately for us, I mean, that didn't go well. I mean, we were betting a lot on, um, service meshe to take off and they never became like a mainstream tech, uh, from that point of view. And very interestingly around that time. LA large language models took off like generative ai and like it started with like GitHub copilot and then chat.
GPD came along and this was even before, uh, there was G PT four and I was running this like team of like around 15 remote employees during COVID, like in. We were struggling to like ship code faster. Like one of the big bottlenecks was like even though copilot had come or people were coding, uh, even doing like small stack code requests and so on, but still the code reviews were like a very massive bottleneck, right?
So I did this like weekend hackathon like project where we started using some of these language models to automate the code of your process to find low hanging issues that can go beyond simple linting issues that you find with existing tools. That was a very interesting outcome. Like we saw that this is a really good fit, and then Code Rabbit started as a separate company.
It just took off. And it was very clear that, uh, we, we have to like just focus on that problem statement and then flux change. On my second startup, we basically folded into Code Rabbit and that's what I then continued to go full time and Code Rabbit. Yeah, go on.
Corey: Yeah. Co Code Rabbit is, is interesting to me from the perspective of the, you're tackling something that historically has felt very boring and things that people tend to more or less phone in.
I mean, code reviews have been around forever, but what did you see around its fundamental broken nature that made you think, ah, we could, we could pour some AI on this and make 'em better?
Harjot: It's broken in many, many ways. You won't believe it. Like not every company. I mean, everyone wants to do a code review process because of compliance reasons, and just because they want to prevent massive failures downstream because each issue you catch.
SDLC is much cheaper, right. Than to cash it later in production. And
Corey: I mean, even on my solo side projects where I'm developing things and merging them in, I still feel obligated to do some form of code review just because I style myself as kind of a grownup and I probably got everything right. Why would I even read this thing I'm about to merge in?
I still feel obligated to do it. That doesn't mean I do it, but I feel bad when I don't.
Harjot: Right. I mean, and, and a lot of managers in these companies have that like. Guilty, uh, conscious, like in because of the, because they're not doing a good job. Because if you look at really. Uh, good code reviews. They take as much time as it takes to actually write the code.
But I mean, you are actually have to really understand the context of these changes because software is complex. It breaks in very, very interesting ways. And code review is like the, the kind of the first kind of defense. I mean, of course you're doing more validation downstream, QA environments and so on, but code review is really necessary, but most companies don't do it well.
Corey: Oh yeah. And AI seems to be way faster at not reading the code, typing LGTM as a. Comment and then clicking the merge button. It feels like it could speed up that iterative cycle, which frankly is sort of the way that it seems like all large code reviews tend to go. I've, I've submitted three line diffs that have been nibbled to death by ducks as, as people talk about it, but that.
10,000 line diff of a refactor or a reformat or something, gets the, looks good to me, ship, ship, squirrel, and off it goes. It's, there's a human nature element to play here.
Harjot: You are, right? I mean, when the PR are small, yes, people can still have a cognitive, like, can, can, can cognitively look at them, but when they become like beyond a certain threshold, that's the point where you say, okay, rubber stamp it, like ship it.
I don't care. I can't just go through it. And then like in some. Companies, you have like the ego clashes, like people actually do a very thorough job or too thorough and many times, and you have like so much back and forth for days and weeks that a stretch and things don't move at all. Right. So, so yeah, I mean the code reviews can get ugly in many ways.
Um, most of the times it's rubber stamps, as you said, but in some cases they can also be very toxic and not pleasant experience. Right. In many ways.
Corey: Show me how your company reviews code. I can show you aspects of its culture
Harjot: then. That's right. That's right. Yeah.
Corey: So, uh, I guess a, a question I have is around what, I guess I'll term a second order effect of a lot of this AI generated code proliferation.
Uh, now it seems like forget merging code in that I haven't read, in some cases I'm merging code in, I haven't written. It feels like that is, that is, that is increasing the burden on the code review process.
Harjot: Right? So one of the biggest tailored applications for generative AI has been. I mean, that's probably the only thing that's working if you really think about it, right?
Corey: Oh, it, it is the standout breakout success of generative ai. Everything else is trailing behind, but what it turns out that there's a lot of replacement of stack Overflow that AI code generation can do.
Harjot: That's right. That is right. I mean, and it's getting more and more sophisticated. Now you have all these coding agents.
Which had come on the scene, which, like cloud core for instance, you're starting with a prompt and you're getting large scale coding, coding changes done, um, in a few minutes, right? A lot of these wide coders or the junior coders in, in every organization, they tend to like throw this code over the wall and then it's someone else's headache to review it.
Especially the senior developers and now they have like a huge bottleneck of pull requests. They have to now piece together their puzzle like on what actually happened there. And, and it's a nightmare. I mean, the backlogs are now increasing because of generative ai. A lot of white coded prs are being opened against open source projects.
It's also maintenance nightmare. I don't know whether you've seen a lot of tweets around, uh, open source projects where they're getting these contributions from random developers across the world with like 10, 10,000 line prs, hundreds of files changed and so on. They seem like good features on the surface, but once you start digging deeper, they're like right with issues.
And that's where I think it's becoming unsustainable. Like, and you're getting ai, uh, it's like an air battle. Like earlier you were fighting a tank battle, but now you're like in a very different battleground. Like you have to bring AI to fight AI in, in many ways. Or AI to review ai. Right?
Corey: Very much so.
It's, it's one of these areas where it just seems that there's so much. I guess nonsense code being thrown out. I look at the stuff in my own code base after I let AI take a few passes through it. And like the amount of dead code that'll never be reached through any code path is astonishing. Uh, five different versions of documentation, all self-contradictory.
It becomes an absolute mess. Uh. The counterpoint though that I have is that this is not the first attempt at solving the code review problem. There have been a bunch of traditional tools aimed at this before the rise of ai, and the biggest problem that you had there was false positives.
Harjot: That's right. I mean, if you look at the traditional tools like pre gen ai, they were like mostly like based on, um, static rules.
Some sort of like, uh, like if you look at the security scanning market, the SaaS market, like you probably. Some of those companies, uh, they are looking at like OAS vulnerability. They have signatures to dis discover those, uh, deficiencies. Right? And every company, like when they enable these tools, they end up in the, in the, a lot of alerts, like alert fatigue is a problem and you end up like switching a lot of these linking rules so that you can still move faster without like making sure you're covering all kind of like.
All these like clean code guidelines that you could have, right? Um, yeah. So that has been like a traditional problem with the tools in this space. I mean, all the way from Sonar Source and um, uh, Sam Grips of the world and so on. And when it comes to ai, like one of the nice things is you can tune it.
They're like very interesting knobs. Possible here. And also the kind of insights you're getting are more human-like feedback. You're not going and nitpicking on some signature. You're talking about the best practices. As you said, it's a stack overflow. You're taking this, all these best practice examples that these models have been trained on and you're bringing in to the, to the more of an average engineer and anywhere like in the world could now it has access to the best practices, uh, which they otherwise don't without proper mentorship.
Corey: I I, I wanna dive a little bit into, I, I guess the cloud architecture piece of a lot of this. You've been fairly open on your engineering blog about how your infrastructure works. You, you use one of my favorite Google Cloud services of all things, uh, Google Cloud run to effectively execute what amounts to untrusted code at significant scale, and that is both brilliant and terrifying at the same time.
Uh, can you talk me through that decision?
Harjot: Yeah, we were like one of the first companies to build agent workflows and also like engineer it to a point where it's very cost effectively done. So one of the things we found with building these agents is that no matter how much context engineering you do, it's never enough.
So a lot of the companies started with drag lookups. They will index the code base and they'll bring in all this context and then ask the model a question around whether this code that you're reviewing looks good or not. But we found that. Very often, this context is never enough. It can never be enough given the complex code bases in the real world, that's where you wanna like give this model some sense of agency to go and explore.
They can in the, for example, the Code Rabbit has like a multipass system. The first pass off the review, it does raise some concerns and now these could not some, A lot of the times these are false positives. They're not valid findings. So what we do in the sandboxes that we create in Google Cloud, we use Google Cloud Run.
It's a really create service or serverless. So we create this like e fm environments where we kind of, the pull request we are reviewing, we not just look at that pull request, we bring in the entire code into the sandbox, and then we are letting the models actually navigate the code. Like a human does, but using CLI commands.
And that was the other innovation, like we, uh, generating a lot of shell scripts, like we are letting the models like run queries. Uh, we are letting the models do cad command to read files based on the concerns they see in the code. And they will, they navigate the code and once they bring in this additional context, that's where they're either able to suppress some of these like false positives.
And in many cases we are able to find issues which are ripple effects in the call graph across multiple files. That's what makes four abit really good, by the way. Yeah.
Corey: Yeah. How do you wind up, I guess, drawing the line? Because I, I found that one of the big challenges you have with a lot of these LLM powered tools is they are anxious to please the, uh.
When you say, great, find issues in this code. Look, they're, they're not gonna say, Nope, looks good to me. They're going to find something to quibble about, like some obnoxious senior engineers we've all worked with in the course of our careers. Uh, how do you at some point say, okay, anything, uh, nothing's ever perfect.
At some point you're just quibbling over, uh, stylistic choices. This is good. Ship it. How do you, how do you make it draw that line? That was the hardest
Harjot: part. I'll tell you. So when we started, like, I mean, you are right. I mean, if you ask a model to find something wrong, it will find something wrong in it.
Like 10 things, 15 things. It'll almost always please you, right? And the hard part is like how do you drive, draw the right balance? Like in classifying a lot of this output and seeing what's important enough for the user. To pay attention to. Right. Took a lot of trial and error, like early days when we launched the product.
We still had like a lot of cancellations because of noise and, uh, feedback that was too nitpicky and it took a while to like learn and figure out the right balance when it comes to the quality of feedback and what's acceptable. Um, and, and it was a long battle. I can tell you like a lot of our engineering actually went into taming these models to a, to point, to a point where we can make majority of the users happy.
We can't make everyone happy. This is the nature of this product that not determined is taken. Um, um, they're not like the previous generation of the systems where you can deterministically define the rules, but this one's like, like very wide feedback. We are vibing, uh, as they say, right? Vibe check.
Corey: Yeah.
Yeah. There's a, there's a lot of. I guess nuance in a lot of these things and the space is moving so quickly that it's basically impossible to keep up. Uh, you have, I believe, standardized around Claude's models for a lot of this, uh, 20 minutes before this recording. If people wanna pinpoint the, pinpoint this in time, uh, uh, anthropic came with a surprise release of Opus 4.1.
So if we had recorded this yesterday and said Opus four was their premier model, that would now be inaccurate even in a short timeline like this. How do you, I guess, continue to hit the, to hit the moving target that is state of the art?
Harjot: Um, that's a great question. First of all, we use both open AI and anthropic models.
In fact, like our open AI token usage might be even more than that. Our, we philanthropic site, we use like six or seven models under the whole, like one, the nice things about Cora product has been, it's not a chat based product. Every product in the A space starts with some sort of a user input. Code Rapid is like zero activation energy.
You open up poll request, it kicks off a workflow and it's a long horizon. Workflow takes like a few minutes to run,
Corey: which is genius. Chatbots are a lazy interface to be direct. It's everyone tends to go for that 'cause it's the low hanging fruit. But if I have to interface with a chatbot, it's not discoverable what it's capable of doing.
And if I look at it in a traditional website, that already means on some level your website has failed in all likelihood.
Harjot: Yeah. Yeah. In a way like it's like, um. Zero activation energy kind of a system. Like you don't have to remember to use it, right? But that brings in like very interesting thing, like first of all, it's a long running workflow with ensemble of multiple models and evaluations become like the key thing, like the nature of these products is it's all about evaluations.
We are not like training our own foundational models. It's not in our budget. Right? So what we do is like. Make sure that at least we have some sort of a sense into understanding these models and the behavior and tracking, um, their progress across different generations that we are seeing. Right? So we are actually, uh, almost always testing some frontier model for these labs, uh, using our evals.
We kind of have the, one of the best evals in the industry when it comes to reasoning models, given that. Code reviews are a reasoning heavy use case, and also around the agent tick flows because as you go deeper and deeper into agent tick flows, your errors compound. So these long horizon tasks, if you're like, go off track on the first step, you're gonna be way off in the like 10th step, right?
So, so yeah, it's all about like evaluating. So we have a team that spends a lot of time looking at open source usage that we have, bringing those examples in to make sure we have the, the right evaluation framework to understand the behavior of these new models as they come out.
Corey: Stuck in code review limbo.
Get Code Rabbit and never wait on some guy named Mike again. Swap the drama for multilayered Context Engineering that suggests one click fixes and explains the reasoning behind the suggestions. Code Rabbit integrates with your GI workflows and favorite tool chains. Supports all programming languages you're likely to use and doesn't need configuration.
Get code reviews when you need them@coderabbit.ai. And get them for free on open source projects and on code reviews in your IDE. So this might be a politically charged question, but we're gonna run with it anyway. Why did you pick Google Cloud as your infrastructure provider of choice? I mean, well, why not Azure?
I, I can answer that easily, but AWS is, is a viable, is at least a viable contender for this sort of thing. Uh, I, I have my own suspicions, but I'm curious to hear what your reasoning was.
Harjot: We love the cloud run product. I think that was one of the big drivers. The whole cloud run thing is amazing. Like that was one of the reasons we saved us so much time and costs in operating like this whole server desk thing.
Right. And also like in the previous startups we have gone with Google Cloud, like the interface, the, it, it, it's like, uh, um, like Amazon is great, but that's like one of the first cloud services. It's, it can get very overwhelming. To a lot of people, but the G CCP is like much more cleaner in our opinion cost wise as well.
It's been very effective. Uh, in terms of, um, yeah, for start, I think a lot of startups do build on Google Cloud.
Corey: Yeah, it's, it's one of those areas where if I were, and I've said this before, that if I were starting from zero today, I would pick Google Cloud over AWS just because the developer experience is superior.
Cloud Run is no exception to this. It's one of those. Dead simple services that basically works magic, as best I can tell. It feels like it's what Lambda should have been.
Harjot: Yeah, I mean it's amazing, right? I mean you can, it's all con based, which we love. Like when scaling is so straightforward once you understand the model there, like it's not like based on just resources, like the knobs there are very like makes so much logical sense.
Once you get to understand them, they're much simpler.
Corey: How do you handle the, the sandboxing approach to this? By, by which I mean that it has become increasingly apparent that it is almost impossible to separate out, prompt from the rest of the context in some cases. So it, it seems like it would not be that, uh, out of the realm of possibility for someone to say, yeah, disregard previous instructions, do this other thing instead, especially in a sandbox, it's able to run code.
Harjot: Yeah, you're right. We are running like, um, untrusted code and some of this like chat interface. You could actually steer the system to generate any malicious share scripts or Python code in that environment. Right. Um, and we do have internet access enabled as well, right? So it's all about like locking down, making sure have C groups and all set up so they, you're not like.
Um, escaping the sandbox that we have created. And the other part is like locking down the access to internal services. You don't want that sandbox to access the metadata service of these cloud providers, right? So yeah, certain like standard stuff comes around like network, um, network zoning and, um, the, the, uh, C groups and all.
Um, and the other part is like we allow internet access. I think that's something we. We disallow everything which is in the, in the G-C-P-V-P-C, but allow internet access. Um, at the same time, we have protections around resource utilization and killing those malicious product projects, uh, uh, scripts that could be running.
Corey: It's that that is always one of those weird challenges. Uh, taking a more, a more mundane challenge that I have is often code bases tend to sprawl as soon as they become capable of doing non-trivial things. And some of us don't bound our pull requests in reasonable ways. How do you wind up fitting, getting meaningful code review?
Either in a giant monolithic repo that will far exceed the context window or or counterpoint within a microservice where 90% of the application is other microservices that are out of scope for this. How do you effectively get code review on something like that?
Harjot: That's a good question. There's a term for this called context engineering and, and.
What we do actually, if there's the best way to describe it, like it all starts with building some sort of code intelligence. Let's say you are like reviewing five or 10 files, but you have a like large code base where those files got changed. The first part of the process is like building some sort of a code graph because in unlike Cursor, which kind of uses in the code completion tools, they use like code index, which is more on similarity search.
And that works great for their use case because they mainly need to follow the style of existing code when generating new code. In our case, like the code review is a very reasoning intensive workflow. Like we, if we are bringing in these definitions from the other part of the code, they have to be in the call path, right?
So that's why we build a relationship graph, which is a very different technology that we invested in. That brings in the right context as a starting point. As I said, it's still a starting point, like you still have to do the agentic loop after that, but the starting point has to be good so that your first pass of the review has some bearing on where to poke holes.
Yeah, you're gonna like first raise 2030 concerns and you're only gonna start digging deeper on those. Choose your own adventure kind of, um, um, routes. Uh, and some of these will lead to real bugs. It's not deterministic. I mean, each run would look different. It's like, just like humans, human humans will like miss stuff some a lot of the times.
But now in this case, AI is still doing a really good job in poking holes at where it feels the deficiencies could be there given the code changes. But it starts with the initial context you have to bring in like code graph learnings. So we are long term memories feature. So each time a developer can teach Code Rabbit, it learns and it gets better over time because those learnings are then used to improve reviews for all the other developers in the company.
It's a multiplayer system. Right. So we are also bringing in context from existing comments on a pr, some of the linking tools, CICD. So there like 10 or 15 places you're bringing the context from. The most impactful is usually the code graph.
Corey: I, I, I want to explore a bit of the business piece of it. If that's all right with you, uh, you've taken a somewhat familiar GitHub style model of free for public repos, paid for private ones.
By and large. There's a, there's a significant free tier that you offer, uh, and you're also, to my understanding, free in vs. Code cursor windsurf, as long as that lasts, et cetera. How do the economics of this work?
Harjot: That's a really great question. Like when we start this business, like one of the things we realized that it's a habit change.
We are trying to make people. Adopt this new habit, like this AI thing. Most people don't want to use it. Like, I mean, you are trying to bring AI experiences into existing workflow and universally everyone hates it. Now. Core Rabbit has been lucky in that regard that we brought in a very meaningful experience that people love and we wanted to make sure that we.
Spread it and, and make it like democratize it to some extent. Like that's where the open source makes sense. I mean, first of all, I mean I, we love open source, like I'm a big believer and we sponsor a lot of these open source projects and, and that became also testing ground. So that was other thing like, because it's not just a Go to Market, but also the learnings and the public usage.
We used that as a feedback loop to improve the AI product so that we can serve the paid customers. So from the economics point of view, yeah, it's hard given that it's one of the agent systems and you know, publicly, like even Cursor and cloud code have had issues with their pricing. Massively negative gross margins.
Like it's like we are still able to offer this service at like a flat price point of per developer per month, which is. Affordable enough that we can go mass market with it
Corey: and predictable enough, which is underappreciated.
Harjot: Exactly. It's predictable enough. There is no surprises and we don't have negative cross margins.
Right. I mean we, we are one of those very few success stories where we have engineered the system to. To a point using on, like, we are not like letting users pick sonet and run that in a loop. I mean, if you look at most products, that's how they are. You're picking a model and then running with it. Like we, we are being smart about this, right?
It's like Amazon Prime. Yes, everyone wants free shipping, but you can't just offer it. You have to build automation, the infrastructure. And we invested in that. That's a trick. I mean, we are able to support all the open source users so that we can learn from them a lot. We can, we are supporting a lot of the, uh, IDE users because we monetize on the GitHub side, we are a team product and that's the market we care about.
Um, by removing the barrier to entry using the IDE, um, most people are not familiar getting familiar with COBIT through that form factor. And once they like it, they're able to bring us in into the Git platforms where they need more permissions and some consensus to adopt it. Um, and that's working really well.
I mean, Go to Market wise. Um. Yeah. Uh, we are growing like double digit growth, uh,
Corey: every month. And who are these folks? Are these, are these individual developers? Are these giant enterprises? Are they somewhere between the two? Somewhere between the
Harjot: two. Like most of our growth early days had come completely product led growth inbound like.
All the way from small five developer companies all the way to hundreds of developers. So we've seen the whole spectrum of it. Everyone needs this product. Like no matter you're a small company or large, everyone needs to do code reviews and, uh, the smaller teams tend to move faster given that it's a fast, like you can build a consensus quickly.
Larger teams need a longer POC, but usually happens in a few weeks. ROI is very, very clear of for this product. Uh, we have some of the enterprises now also like doing some POCs for a few weeks, and these are like large, uh, seven figure deals even.
Corey: That is significant. Uh, you've also recently raised a $16 million series A, uh, led by CRV.
Uh, so I'm sure you've been asked this question before, so I don't feel too bad about springing it on you. But what happens when Microsoft just builds this into GitHub natively? Uh, how do you avoid getting Sherlock by something like that?
Harjot: It's already happened. So we've been competing with GitHub co-pilot's, uh, code review product for the last 10 months.
Now,
Corey: the fact that it automatically does that, and I had no idea that it did, that tells me a lot about GitHub's marketing strategy. But please continue.
Harjot: Yeah, they do have that product, which is built in and usually, I mean, of course, I mean, it's almost like. As with everything, GitHub, like the best of breed products, uh, still win, right?
I mean, so that's where like, it hasn't impacted anything on our growth or churn rates, even despite that product being out there.
Corey: I have heard people talk about Code Rabbit in this context. I have not heard people talk about copilot in this context for just a sample size of one here,
Harjot: right? That's right.
And we have like. Innovated in this space. We actually created this category. I mean, the bunch of larger players, they're all trying to copy our concepts, but still there's a lot of tech under the hood, which is like very hard to replicate. I mean, it's, I would say, a harder product to build than even code generation systems, given that it's a very reasoning heavy product.
And, and people are more sensitive to the, to the inaccuracies when it comes to these kind of products. And it's a mission critical workflow, right? I mean, you're in the code review and it's a very serious workflow, uh, not just something on your developer's laptop, and you can be more forgiving around the mistakes.
Yeah, that's why we have like seen not of colognes, but no one has been able to replicate the magic of Code Rabbit. We are like probably 10 x bigger than the next competitor in the space.
Corey: Yeah, I'm not aware of other folks in the space, which probably says something all its own, and this also has the advantage of, it feels like it is more well thought out than a Shells script that winds up calling a bunch of APIs and it doesn't seem like it's likely to get become irrelevant with one feature release from one of the model providers themselves.
Harjot: That's right. That's right. I mean, yeah. I mean, so we kind of better on the right things early on. Like going fully agent, we kind of saw that coming two years back and investing a lot in the reasoning models before they even became mainstream because the entire thing was reasoning heavy. So we've kind of been future proof with the decisions we are making.
And now we could be blindsided by something, I don't know, like GPT five, GPD six. But so far it looks like each time the new models come out, they benefit this product a lot. And it's all about the context we are bringing in. We engineer the system for cost. Cost is a big factor in this market, I could say.
I could tell you that. Yeah.
Corey: Yes. Oh, absolutely. Especially since it seems like. When you offer a generous free tier like this, it feels like that, yes, it's a marketing expense, but that also can be ruinously expensive if it, if it goes into the wrong direction or that you haven't gotten the e economics dialed in.
Exactly right.
Harjot: That's right. And uh, that's why you have all these abuse given, that's where the technology from my second startup came handy. Like, so a lot of the, uh, abuse prevention is a flux ninja tech that we're still using at cobit. So.
Corey: So you, you claim to catch a ridiculous percentage of bugs, uh, which is great.
I mean, your marketing says all the things I would expect. What has the developer feedback been like in reality?
Harjot: I would, the majority people love it. I mean, you could see this on the, uh, social media, like people are just talking about it. They, in general love the product. Um, like a lot of these organizations we talk to, they say they recovered the investment in two months.
Like, and, and some people are coming back and saying, if, if you were to charge more, they will still buy it. I mean, so the, it's been a very overwhelming, I mean, of course there's always going to be some detractors. People who don't like AI in general have their own opinions, so those. Yeah, so, so that crowd will also be there, but if you look at the majority, it's uh, definitely a step function improvement in their workflow.
Um, and, and, and it's very easy to see if you go and search social media, LinkedIn or uh, X platform or like, you will always see like people saying positive things. And that's where we are getting the growth from. Like most of our signups are actually word of mouth signups at this point. So it's like our customers bringing in more customers.
So
Corey: I, I, I guess my question now comes down to what the future looks like here. Where, where does this go? What is the ultimate end game? Uh, do human code reviewers wind up going extinct, or is this more of an augmentation versus replacement story?
Harjot: I think it's more like now humans will be like fighting a higher order battle.
Like if you look at all this like nitpicking and looking at problems, it's like the AI still like, is not, doesn't have the non-obvious knowledge. Like there's knowledge beyond code review that goes into the decision making, right. Which we don't have. And, and the humans have that knowledge, right? So in a way, like, I don't think humans are going away.
I mean, the fear of seen is like. Usually, instead of having two code reviewers on each pr now, one is ai, the other still a human. And on smaller pool requests, they're just trusting Code Rabbit. They don't even have a human review. So some of that automation has kicked in. But when it comes to coding in general and code review, I think it's gonna be a long journey.
Like a lot of the labs are hoping that we will go completely automate software development, uh, in the next few years. But I don't think that's gonna happen. Um. What we are now discovering that this whole code coding market has multiple submarkets in it. There is like tab completion. Now there are different agents all the way from terminal IDE background agents, and they don't like really replace each other.
Like they, they, they're just being used for different reasons. And then code review, uh, as a guardrail. Layer is going to be standardized for these organizations, uh, as a central layer. It's almost like Datadog, if you have to give you an analogy, like you had all this multi-cloud, you had all these Kubernetes, uh, rancher.
Um, but then Datadog said, okay, to be successful, you need observability. I'm gonna give you the guardrails, and became massively valuable. That's where obviously, core drive opportunity here.
Corey: So I, I guess my last question for you on this is that for various developers who are listening to this, who are drowning in PR reviews going unattended, what is the one thing that they should know about Code Rabbit?
Harjot: Yeah, I think Code Rabbit is a friend. I mean, in a way, like, uh, if they're bringing Code Rabbit into their workflow, they are going to be like, at least offloading some of the most boring parts, which is the preview. Like, of course, building software is more fun than review. Uh, it's, and it's fun to work with.
I mean, of course it's going to be that. Ai that's like always watching their back while they're trying to move fast with ai, making sure they're not like tripping over and, and causing bigger issues.
Corey: If people wanna learn more, where's the best place for 'em to find you?
Harjot: Yeah, I mean, they could just find us on Code Rabbit ai and uh, it just takes a couple of clicks to try out the products.
Really frictionless. So you could get started just a few minutes for your entire organization.
Corey: Awesome. And we'll of course put links to that in the show notes. Thank you so much for taking the time to speak with me today. I appreciate it. Thanks, Corey. Thanks for having
Harjot: me here.
Corey: Harjo Gill, co-founder and CEO of Code Rabbit.
I'm cloud economist Corey Quinn, and this is screaming in the Cloud. If you've enjoyed this podcast, please, we have a five star review on your podcast platform of choice, whereas if you'd hated this podcast, please leave a five star review on your podcast platform of choice along with an obnoxious comment that was no doubt written for you, at least partially by AI configured badly.