Ankit Jain (00:00) Hello everyone and welcome to Hangar DX. This is a podcast about developer experience and how engineering teams solve productivity challenges at scale. I'm your host, Ankit Jain co-founder of Aviator. At Aviator, we are building a verification platform for your AI-generated code that creates guardrails to reduce slop. Today, my guest is Vanita joining in from ThoughtWorks, and we will be talking about everything related to code reviews. So Vanita, thanks for making time to jump on this call. Can you please introduce yourself for the audience? Vanitha (00:33) Of course. Thanks for the opportunity, Ankit. Vanitha Kumar I'm based out of St. Louis, Missouri. I'm a technology consultant at ThoughtWorks. I've been with ThoughtWorks for a little over four years. I've been a full stack developer, infrastructure developer, enterprise architect, platform architect, and with ThoughtWorks I've done it all ⁓ and breadth and I'm happy to have this conversation with Ankid because ⁓ I'm very close to code and ⁓ will always be a developer at heart so Let's have it at it. What are we gonna talk about? Ankit Jain (01:04) Amazing. So yeah, I mean, I believe everyone sort of like knows ThoughtWorks today, kind of like you know, they've been in the industry for a long time, providing a lot of I would say, guidance for how engineering should change and evolve. And so we are obviously in very interesting times where everything is changing, everything is, you know, turning upside down. So ⁓ we want to talk about code reviews. ⁓ more specifically. Vanitha (01:28) Yes. Ankit Jain (01:30) I think a good place to start would be what is a code review? Like what are ⁓ what what does a code review as a practice even mean? And like if you have any thoughts on what are like if you were to kind of like break it down, what is are different parts of code reviews? Vanitha (01:39) Yeah. of course, of course. I've had a transition of what a code review is through throughout my career, and it's like more than twenty-five years long. with ThoughtWorks, my idea of what a code review has completely changed because again, this is a culture where we follow trunk-based development, test driven development, and pair programming as a religion, which means code review is not really a thing, it's not a ceremony per se. So code review for us has always been very light, very low effort. And all of the benefits you would get from code review has always been in the practice of while pair programming. It is something that happens organically. When you're trunk-based developing, so to speak, you do not hold on to a branch for weeks on end. And when you're writing tests sufficiently, like if your pipeline passes, that's good, right? Yeah. So the idea of code review has changed a lot for me. Earlier on in my career, it has always been, I'm a junior developer, someone who's had much more context, knowledge, experience has been the one to look at it. ⁓ tell me better ways of doing it, and use it also as an opportunity to educate, which I also appreciated. So there's there's different interpretation of what code review is. All of that is still relevant, still valuable. ⁓ where we all are thrown a monkey wrench at this point is the volume of code we could generate. Ankit Jain (03:03) Yeah. Vanitha (03:05) And how on earth are we gonna keep this practice of code review still relevant ⁓ and meaningful in this time and age without holding on to our older beliefs of this is how it should be and we should continue doing it. ⁓ so it's an interesting time. What have you been seeing on Ankit Jain (03:20) Yeah, absolutely. if you think a little bit about like, you know, even the background and history of code reviews, it's like it's not very old practice, right? I think code reviews really started maybe even 20 years ago. I remember like in 2006 was when Google started doing their code reviews. They kind of like introduced their internal practice of doing that. If you think about it, like a lot of like Windows original software was shipped without code reviews, right? Right. And and it's like today people feel that it is a very necessary, key factor to kind of like doing and shipping software. Some of it is compliance. Generally, I talk about like code reviews also a theater these days, you know. how have you seen sort of like you know it evolve? Do you see code reviews happening more today? And maybe because of AI kind of like we are also moving in a different direction, maybe we need to rethink why. like do we even need to do code reviews? So kind of like, have you seen any patterns evolve or change? Vanitha (04:13) Yeah. like you mentioned the trajectory has been we started with code review meant for something and at this point by default everyone thinks code review as something that happens pre-integration, right? That's the only place we do code review, nowhere else. Not before, not after that one point. It's it's almost like a muscle memory. That's the stage we do and that's the only stage we should be doing. Ankit Jain (04:30) Yeah so. Vanitha (04:36) There's there's been thought leadership around like, no, like that should not be the only place. ⁓ and there was no forcing function to put that thought into practice. Now that we have these powerful agentic AI and agentic IDEs helping us to produce more code, now the traction behind should pre-integration be the only time where we do code review, everyone's starting to think about it. Because these agents are so good at writing code. that the correctness, the integrity of the code, so to speak, and is it doing what it's supposed to do? Like, yeah, for for for for most part it does. Where the gap could rightly originate ⁓ is at the time of the developer expressing the intent to the IDE. That's where you want to catch, right? Again, the I'm I'm loosely saying as expressing intent, we know how that turns out. It's like like tens and twenty four For markdown files per feature at least, right? How do you review this? And most times we don't even version control those files. What's worth version controlling? What's not worth version controlling is another debate, right? Do you version control every markdown, every project? No. Right? So ⁓ the idea of what is worth code reviewing has shifted, right? Either people want to review only the specs or specs and code. Ankit Jain (05:44) Okay. Vanitha (06:03) Or they don't care. We just will test the running system because the volume is so much more. So that is starting to shift. What do we what do we review? Again, how do we review is also changing? Again, purely because of the volume, because most of us who are used to seeing this in whatever tooling of your choice know if it spans more than five pages, it's it's too much for a human to connect the dots between. what was the intended change and changes across five different files. It's a lot. Imagine that multiplied ten times at least. Then the way you visualize change also has to evolve, right? You want to change the original intent. You may want to change see the path your agent took, what's it called agent traces and the code. And did you practice, ⁓ follow all the ADRs? And by the way, where are the ADRs in my in my workspace? So you kind of start to have this broad dashboard of SARS to be able to do code review. It's not just diff anymore, right? It's not just diff and the ticket. You need a lot more inputs for the human to do code review if they have to do code review to the same extent that they have been doing in the past. So how you do it and what you review has drastically changed and ⁓ all of us are trying to grapple with how do how do we handle it. some people have have applied lever on the front end, meaning like I'll just review the spec, rest all, like trust in God. ⁓ some still take the pain and want to review every single artifact, acknowledging fully well that that is starting to become bottleneck in itself. So yes, the what and how is fundamentally shifting under our foot. Ankit Jain (07:46) Interesting. So ⁓ I have like this thing which we were talking about before, right? So I feel code reviews are not just about code reviews, right? Like there is an aspect of collaboration, an aspect of knowledge sharing, which is why in some ways code reviews have been very helpful, right? Like you were talking about like as a senior engineer, you're providing feedback to sort of like people who are joining the company or like our juniors. And you know, maybe if you're a junior, you kind of like get that feedback and collaborate and learn more about better practices. so from that perspective, it does feel like we should use some degree of code reviews to in make sure we continue having a collaboration. Right? I think what we're talking about here is potentially changing the shape. Right? So so it's more about how do we actually Manage the volume of code which is getting created with agents, build enough confidence and trust that like this code is going to do the thing it's supposed to do, while making sure that we can actually still have a space for learning, collaboration, and arguing and debating on the architectures. In some ways, like I feel like humans have biggest value that humans provide in the entire software development lifecycle is the judgment and the decisions. And that is a part that we still need to somehow keep. Vanitha (09:11) Yep. Yeah, yeah, absolutely. Yeah. Ankit Jain (09:16) Right? So so so I guess like maybe sort of like if you have any thoughts on this direction, you already covered some shape is potentially different, but like have you have you been talking to some of your clients and like you know, you obviously like have huge customer base as well. What has been sort of their opinion on this and sort of like there is a bit of a huge cultural shift or sort of like, you know, ⁓ it's a people problem, a process problem, less so than a tooling problem. Vanitha (09:49) Course. there is there is also a philosophical debate about the question you're asking. ⁓ and it's it's everybody in everybody's mind. Should the change should the change sets be this large? Like over decades we have always thought like smaller chunks, faster feedback, but now we have kind of thrown the principle out of the window. There is so there is that philosophical debate of should we encourage this culture and habit of Ankit Jain (09:59) Ha ha ha. Vanitha (10:18) bigger chain sets. And the other is the tribal knowledge, right? The reviewer having that, hey, there's there's there's a library for handling dates in our platform and it's been around for for years. That's what you should be using, but you haven't had the knowledge of it the three weeks you were building a feature, right? it's I don't know if it's a good thing or bad thing, but The idea is that you kind of have this enterprise context. So you kind of take that gap or take that in opportunity for collaboration away from the humans and you kind of build the enterprise context for it. It's good, it'll scale, it'll work well, but it also takes the opportunity for collaboration. those things are what ⁓ I I would challenge and I would also reflect about your questions. But what we are seeing with our clients is the acknowledgement and also the reckoning that some of their old practices they'll have to step back and see if they're worth keeping and worth pursuing in the extent that they have practiced all these years. So we've had clients where we have piloted spec driven development, which means there is huge amounts of markdown files. this particular client has always had the habit of reviewing documentation consulting companies like us produce we are not full-time employees so anyone who's not a full-time employee the document they produce should also be reviewed and we present them hey here are the markdown files here's where our documentation is they're like this is huge number of documentation and then we produce code the code volume is also large because we are able to produce larger change sets ⁓ They've come to a realization that doing the code reviews or reviews of any sort, the same old-fashioned way in this time and age, will end up them being bottlenecks. So they they are also starting to see in practice, be in the trenches and notice like what this means in practice and recalibrate their ways. We've also had clients who kind of give us full reign on everything and only want to do code reviews, right? some of the projects we do are outcome-based, which is exactly what they want. Like you all figure out, that's why we hire company like like ThoughtWorks. We we're just interested in the outcome, but we want to do code review. In such places we were able to kind of glean their best practices, their organizational way of doing certain things, and kind of build agent around it and say, hey client, I know we have done five code reviews with you. You seem to be catching similar patterns. We have corrected, but will avoid this work for you going forward. And here's an agent you have written here. Look at the markdown. This is exactly what the LLM is doing. And they were cool, like, yes, like less work for us. We are we are we are perfectly fine. there is this aspect of change management. Again, change you can't just coach someone into change. You can't just talk someone into change. You kind of have to create that awareness and show it in practice. Ankit Jain (13:14) Very interesting. where have you seen sort of like these teams accepting that, okay, ⁓ we don't need to do these reviews in such detail? Like, what are they choosing as an alternative? Is it like sort of like are they relying more on these agent tech reviews? Where are the blind spots? Like, so for instance, We are also with our verification platform, we have like this concept of AI slop register, which is like there are things which in systems cannot always catch, but like, you know, you're repeatedly kind of like possibly reporting the same issues. Can you actually like start building a register of these issues and that gets catched automatically? So kind of like, you know, this is just like one thought process, but like I would love to hear where are have you seen sort of like patterns changing? Obviously, like everyone is feeling the. Vanitha (13:43) Mm-hmm. Ankit Jain (14:07) pressure to do reviews. but some of it is also I would say emotional to some degree, right? Like o because in some ways people own different parts of the code bases, they don't want that people be submitting changes there without their review. Vanitha (14:23) I would say if it's it's a time of reckoning for them also, right? Again, so some of these practices that people have been blogging about, talking about for years is coming to fruition now. One thing is collective collective code ownership. Like we've always said like one person in the team should not have all the knowledge and the work should not come to a standstill should that person win lottery. Right. But no one works towards that. We are we are very happy with that one person being the one person who has all the context and enjoys enjoys the attention. We do not want that. So this idea of collective ownership is also coming into fruition in that like Everything everyone knew in their head is kind of going into a markdown or going into a knowledge graph and being fed as context to whatever LLM that's that's doing the work for us. So the idea of collective ownership is kind of ⁓ coming back in fashion, so to speak, such that ⁓ we can get more mileage out of the LMs. ⁓ the other ⁓ thing that I've started noticing is there are only so many markdown files and instructions and script. And hooks that we can write in the coding stage, right? There is still a lot of learning. This intelligent system that's sitting in our IDE can learn beyond that also. Again, it it all comes to the concept of what Birgitta Boeckeler from ThoughtWorks is coining as harness engineering. You harness your system so well. before writing code and after your code is deployed so that the amount of review one individual human being has to do is drastically reduced, right? That's one angle of doing it. The other was the visualization. Give all the information a human's gonna need to do the review and let them let them do the review like they always did. What if you flip the coin and say you don't have to, there are so many checks and balances in in place. You're kind of being human in the loop, approving what has happened. is okay. I so the idea of harness engineering is one you you harness at in the ID stage when you write code and you all also feed back the signals that you see in your pipelines right when when you run the linters, when you run the security scanners or when you catch ⁓ abnormal behavior in your ISD or dash systems. All of that can be fed back because again it's even though it's call IDE, it can still learn those things are a lot more intelligent than they were. So the idea of harnessing before writing code, which is feed fast and feedback to make your system faster in order to reduce toil is also an idea that's that's ⁓ exciting to people who actually review because now it's becoming an overhead to review. It's no longer joyful ⁓ for some folks it's no longer the ego Ankit Jain (16:58) Yeah. Vanitha (17:24) So that they're used to getting, it's just just a lot of work at this point. Ankit Jain (17:28) Very interesting. tell us more about like harness engineering. So how how does that fit into ⁓ the existing ecosystem? do teams need to build some sort of like a feedback mechanism? Is it like getting auto ingested as sort of like more reviews are happening or more code is written? Or is it sort of like, you know, teams sit together occasionally and sort of like write a bunch of like documents? Vanitha (17:52) I hope that's not it. Yeah. ⁓ so the way I simply explain to my teams is it's just platform engineering for the AI days, right? We we all have platforms. We know how our pipeline should look, what are the security gates that must be there, what are the checks and the number of stages a pipeline should have, depending on the kind of industry, the kind of software you build. Just apply a AI schema on top of it is how I simply explain to my teams, meaning Ankit Jain (18:01) Mm-hmm. Vanitha (18:20) for example, you can have CLaude code in your GitHub as a GitHub action in your pipeline, right? So perfect. That is a way to say like, hey, CLaude code, every piece of code that you keep emitting has this peculiar security scan when I security vulnerability, when I do sonar queue, feed it back to the agent that's writing code, then you kind of stop having those repeated Ankit Jain (18:25) Mm-hmm. Vanitha (18:47) Errors or repeated patterns showing up. So these are all things that happen in the platform layer, which we never did before. That's not the level of hardening we did in the past. It was always like ⁓ feedback forward, right? You if if your test failed, you stop. That's it. You you you never had to give it back to the thing that wrote your code because it was not a thing that wrote your code, it was a human who wrote the code. So we just looked at logs or Ankit Jain (18:56) Okay. Vanitha (19:14) alerts from our pipeline and we manually correct it. Just think of it as an agent receiving those and being able to correct and not making those mistakes again and again. So the idea of harnessing your pipeline and idea of harnessing your IDE together forms this practice of harness engineering. Again, it is a practice that's evolving, but it's only a natural progression of, hey, we have done platform engineering for years. How is platform engineering going to evolve for AI days? Because we're going to be producing so much more code. ⁓ That's gonna flow through the pipeline and who's gonna catch it? How are we gonna catch it? And all of that good stuff has led to the practice of harness engineering. Again, it's it's quite new. I would say the first time I heard it was January-ish, and the first time it got published on the internet this sometime in f in Feb. ⁓ Birgitta has the definition of ⁓ harness engineering at this point. I haven't seen ⁓ others come up with other definition because I think it's perfect at this point. So Ankit Jain (20:12) Ha ha ha. Vanitha (20:13) platforms where the answer is, that's where we are going to put a lot of our harness. Platforms what carried a lot of our engineering discipline for a long time. So this is platform engineering for the AIL. That's that's how I'm thinking harness engineering is Ankit Jain (20:28) Nice. So ⁓ kind of like you know, one of the things that I have like we've been talking also in our community is this whole aspect of up until now we were building software by hand. And now we are actually building software with machines. So this is somewhat similar to, in some ways, the industrial revolution. And I think the teams who are actually doing well are the ones who are investing into Vanitha (20:47) Yes. Ankit Jain (20:57) building better machines. And I think that's kind of like one way of thinking about the harness engineering you're saying is like, you know, instead of thinking about building software, think about building machines that build software. Vanitha (20:58) Yes, yes. Yep. That's it. Yes. Yeah. The anthropics and OpenAIs have given us a way to do it, but it's still on the engineering teams to assemble it, guardrail it, harness it, and kind of not let it become wild wild west because we know once it happens, there is it's it's hard to turn that chip on. Ankit Jain (21:16) Right. Right. In a way, kind of like I also feel like, almost what platform engineering started like even as a movement like 10 years ago, today, or like at least in the future, almost every company needs a very big platform team. In fact, most of their engineering would probably be platform team, right? And then there is like a product team that actually leverages it. But because the most of the effort would be required in the platform team to maintain the infrastructure. Vanitha (21:55) Yes. Ankit Jain (21:55) product teams get like ten x the value from each like individual. Right. So in some ways like I feel like the teams themselves will be more skewed towards platform in large teams. Vanitha (22:06) ⁓ one hundred percent. A colleague of mine and I were building a team topology diagram for a client engagement. So there's always this platform layer, right? so far has been delivery infrastructure, that it's a C I C D team or the CLaude infrastructure team. That's that's been the extent of that team's responsibility. You started drawing an entirely different box, still called it platform, and it's like this is the agent dig developer platform. Right. This this is this is where the straitjacketing of developers ⁓ to use the right framework, to use the right models and and to have your your cost controls. All of that's in place. Right. You do an entirely different box and we were thinking like, no, it it won't be two different boxes for too long. It's gonna compress into one platform, right, that does like all of these things, CI C D, ⁓ your runtime infrastructure. And all the all the guardrails for agentic development, except it's gonna go further left into our software development lifecycle because previously it always started with the first commit. Comet and then on is the platform. But now the definition of where the platform can reach is gonna go further left and it's it's exciting to see how platforms gonna evolve because I would say Ankit Jain (23:06) Yeah. Vanitha (23:19) Earlier last year or even before that, I've always reached out to my platform friends and say, Hey, what are you you doing in AI space? What are you doing in AI space? That space has always been like it's it's it's pretty automated. There's a not there's not a lot of AI application there, right? There's not a lot of manual hand coding to be done. So that space f until now has been less impacted by all the AI influence. Now that AI code is generated at large, now everyone's like, ⁓ my gosh, platform engineering, that's where our answer is. That's that's where all the controls on the straitjacketing is gonna happen and that's that's that's the platform engineering, the next next evolution of platform engineering is going to be, I think. I it's very exciting to think how platform's gonna evolve now. Now is the time, right? One year ago no one would have thought like platform engineering has to evolve. Now we know. Ankit Jain (24:03) Mm. Vanitha (24:13) Now we know because of the amount of code we are producing and all the like we've it's a lived experience. Now we know platform engineering is going to matter. Ankit Jain (24:13) Yeah. Right, right. And I think in some ways, ⁓ kind of just taking the analogy of like, you know, factories and machines itself, like almost every company will have to build their sort of like customization of their own factories. Right. It's not like every company will use the same factory. Like, you know, no matter which ⁓ model company produces it, like you still need to customize it and own it yourself. Vanitha (24:43) much as people want software to be commodity, there's always going to be the small little thing that a project needs and it I I don't think it will ever be a commodity commodity. Ankit Jain (24:52) Right. So ⁓ I do want to touch on one more topic, which is around AI code reviews. So I have like this opinion that AI code reviews should not really exist in a UI. So for instance, like today in a UI, ⁓ an interface. So for instance, like today, most of the AI code review systems like sit in top of GitHub or like GitLab and they kind of like provide you feedback, and then you know, maybe an agent looks at it and kind of like, Vanitha (25:07) You know what? Okay. Okay. yeah. Ankit Jain (25:21) argues and kind of like you know either make changes or kind of like pushes back. I feel like that type of theater doesn't need interface. In some ways, some of like these things should further shift left. Kind of like, you know, as agent is building these things, like maybe there's another advisorial agent which is kind of like providing some feedback and guidance. By the time you do a review, like it should really, you know, most of these things should have already been settled. Like how have Vanitha (25:34) Mm-hmm. Yes. Ankit Jain (25:49) seen sort of like AI review as a category. Vanitha (25:53) remember the one story I told about the client wanting to do review and we wrote an agent for said review. So it all started with using Claude code. I created the first version. ⁓ my desire was mostly to educate the team ⁓ before they get the code in its uncooked, ⁓ completely natural form to the client. I have vested interest in my my team looking their best in front of the client. So I wrote that agent and it was not really code review, it was purely catch anti-patterns. And explain it to the developer like you would explain the developer's archetype, right? Like I I know my team's archetype. So review this code, review it against these practices that's in conference document. Explain where my team is doing differently or where they're deviating. Explain the deviation and why the deviation is going to be detrimental. So I kind of built it as a teaching agent more than a More than a code review agent, it kinda evolved into a code review agent. Then what you said occurred to me. It's not something we should be catching way later when the comment has happened. Code review, whether it's human or an agent doing, has to happen much earlier and it doesn't have to be reviewed. It could be a ⁓ rubber ducking or a teaching moment. It also gives the developer space to ask those stupid, dumb questions to an LLM, which Ankit Jain (27:00) Ha ha ha. Vanitha (27:24) So far hasn't started judging us yet. So yes, pull them all as left as possible so that the amount of work you have to do further down is lot less. And yes, absolutely yes to the UI, especially for someone who can't stand dips for too long. Yes, the UI mechanism is it only goes so far. Ankit Jain (27:44) I mean and also like not to forget that it actually puts pressure on your CI systems as well. Right. Because like by the time you send some a thing for review, it's already expected that you're running your CIs and then kind of like, you know, the more iterations you Vanitha (27:44) Yeah. Yeah. Mm-hmm. Ankit Jain (27:56) loads that are increased. Vanitha (27:58) I've also I've also noticed something else, which is all things front end. I I have a fair bit of teams that do the front end work and the mobile development work. ⁓ what has kind of fundamentally shifted is their ability to visualize changes with tools of these times like like what you get in Figma these days, what you get in Repli. You're able to build concepts, build prototypes faster and Ankit Jain (27:58) No, no, no, good, good. Mm-hmm. Vanitha (28:25) the assets from the protot prototyping stage could very well be developer ready, right? So the idea of code reviewing for UX and UI changes has also started shifting because you're able to visualize them using other tools. Ankit Jain (28:40) actually this is something which we are also using a lot in our own concepts of verification, you know, code may look right, but if we are able to do end to end verification by spinning up the servers, you know, like sending the traffic, or maybe sort of like, you know, doing some human clicks, like lot of these things today agents are actually capable of. Right. So our thesis is also can we actually provide enough evidence that like when the reviewer is looking at it, they're only looking at is the evidence make sense? Vanitha (29:08) Yeah. Ankit Jain (29:08) is the intent makes sense. And those are the things how we can actually align better instead of like having to review each change line by line. Vanitha (29:16) what what you said reminded me like if don't don't don't you don't you think code review is starting to kind of blend into or bleed into all things testing, which is regression testing. Some of what you said is like, yeah, like that's what regression test is for. And the enforcement layer in the platform and like the responsibility of code review keeps shrinking and shrinking, right? Like before we were able to do Ankit Jain (29:30) Absolutely. Vanitha (29:40) All the SAST and DAST and IAST and sonar cubes in the pipeline. Those were some of the things I remember you doing as a lead developer, right? Like I would I would memorize the OASP ten and look for it in code. I didn't have to do it anymore. So the responsibility is kind of bled in starting to blend into other activities we do, and it's kind of getting like thinner and thinner is what I think. Ankit Jain (29:51) Ha ha ha. Right, right. I think that's probably like a good way to also like, you know, conclude this, which is in some ways, I do feel like AI is giving us an opportunity to reduce the toil from the review part of it and actually keep the collaboration and keep the judgment part of it still alive for the reviews. I think there's definitely a lot to be built, a lot to be thought through. But the good thing is today at least everyone is open to new ideas. Vanitha (30:13) Yep. Yes. Yes. We we we all are forced to shed our old beliefs and just be open to new ideas. One hundred percent. Ankit Jain (30:38) Right. Okay, cool. Awesome. Vanita, this was a pleasure talking to you. Thank you so much for sharing some of your time, ⁓ jamming on this topic. And thank you, everyone, who was listening. If you're interested in developer experience or work in DevOps, join us in Hangar DX is DX dot community. We do sessions like these, also virtually in group sessions. So again, thank you, Vanita, so much for your time. Vanitha (30:48) Yeah. Yes, th th thanks Anki, thanks for the opportunity.