Dan Gerlanc: I'm Dan Gerlanc and welcome to Agents and Engineers, the podcast about agentic AI and software development. Today our guest is Paul Everitt. He's a developer advocate at JetBrains. Previously, he was the president and executive director of the Plone Foundation. and he is a co-founder of Zope. Paul, great to have you on today.
Paul Everitt: Hey Dan, thanks for having me. This is gonna be a fun conversation, baby. Good way to end the week.
Dan Gerlanc: I'm amped up as well. So you've got a lot going on lately. You were working on a course with Andrew Ng. You've been diving into agentic engineering and spec-driven development. Where where should we start?
Paul Everitt: Boy, are we in a weird moment in our profession. How about that? I wonder how you feel
Dan Gerlanc: Yes.
Paul Everitt: about the moment that we're in. And I don't mean like this decade or this year. It's like this hour.
Dan Gerlanc: Yes, I will probably ask you this later, but I often ask folks, what are you doing differently six months ago than today? And often it could be, what are you doing differently six hours ago versus today?
Paul Everitt: Yeah, in fact that is a good way to put it because six hours ago my MacBook was churning away on doing some things overnight while I was sleeping. I'm definitely investigating a lot of the topics that you're covering about agentic engineering, what's next. And my remit is as a developer advocate at JetBrains is professional developers. We are the best damn software development tool company in the world. and our target base is, you know, people who have a job doing this stuff and just want to be a little more productive. But my other remit is in the world of Python, what does this all mean for us? Where are we going to be when all the smoke settles and AI becomes kind of a more normal thing? Can we in Python, make the future that we want to be in.
Dan Gerlanc: And I know some people have said things like, they're moving to more statically typed languages or what how do you think that affects Python today and going forward?
Paul Everitt: You're right. I saw an article on Medium yesterday about Rust will be the replacement for everything now that you don't have to write anything, right? I've got some theories about it. I think the question comes back to do you believe in the human in the loop? If you're one of the people, the AI fabulists who think that humans will never look at code again. Then you might make some decisions about the language choice that are t steered more towards performance. Even then, you know, Go can compile pretty fast. I don't know if Rust can compile pretty fast. I don't know if I would want an agent loop repeating a bunch of compilations over and over until I got it right. But if you were more like me, an AI realist, and I think that the pendulum has swung a little bit in the last couple of weeks. We're going to have humans in the loop, and humans are going to need to read code and sign off on it before shipping it. And readability will still matter, and Python remains the hands-on winner for that. We just need to embrace some of the benefits that Rust gives you for agentic engineering. Static analysis type pinning. And that has been very much. the focus of my six hours ago, six days ago, the past few weeks.
Dan Gerlanc: Tell me a bit more about that. How has type hinting static analysis been on your mind or what you've been focused on in the Python world?
Paul Everitt: Sure. my interest is in local AI. And that's the next course that is that we're doing or wrapping up right now for deep learning. When you start talking about local AI, you know, not everybody's got a box of NVIDIA chips in a corner. Not everybody has an M5 with 128 gigs of RAM. The global south kind of Th lots of people in the world who matter, who are brilliant, but don't have access to the highest end equipment. If you're talking about that, you're talking about a small language model. If you're talking about a small language model, it has a doesn't have a lot of headroom for reasoning and things like that. How can we help it succeed? How could we help a 27B or even a 12B, 12 billion parameter local model succeed? What I'm finding is take a lot of work out of it and bring it back over to static analysis. This has been a field about hooking up LSPs to agents. Claude tried it in December, struggled, kind of landed it in January, maybe in February, but still the models don't want to use LSP. They want to use find, grep, ls, sed, awk, blah, blah. If you can crack the the code on that, you'll reduce the number of turns dramatically. And if we can come up with something that helps a small language model get to the finish line for a Python application quickly and correctly, then we can include a lot more people in this revolution.
Dan Gerlanc: What have you seen in terms of what is the state of the art for doing that today? Does does it exist or is it something that's still you think very much in development?
Paul Everitt: It's ready to exist. How about that? I apologize, Dan, if that sounds like waffling.
Dan Gerlanc: No, I that's I think ready to
Paul Everitt: but it it
Dan Gerlanc: exist is almost more interesting than existent.
Paul Everitt: Yeah. It arxiv.org has so many revolutions just sitting in there in research papers waiting to be implemented and deployed. In a few weeks, I'm gonna go to EuroPython. And I'm giving the closing keynote. It's gonna be spicy. And at EuroPython, I'll be surrounded by PhDs. A lot I'll be surrounded by a lot of people like you, Dan, that have been doing ML for since the prehistoric age. There are there are so many ideas
Dan Gerlanc: Since R the R days.
Paul Everitt: out there. If you step back and make an agent that is local first, my contention is an agent that's local first will be different than the current harnesses.
Dan Gerlanc: Did you have your keynote written yet?
Paul Everitt: I get on the treadmill and I write and rehearse
Dan Gerlanc: Ha ha ha.
Paul Everitt: in my head. I did clear with the conference people so that they know what's coming.
Dan Gerlanc: Have you worked I mean I know a popular area well popular might be an overstatement, but one example of a different architecture is recursive language models.
Paul Everitt: Yeah. Mm-hmm.
Dan Gerlanc: Do you have experience testing those out with local models, or do you think that could be one direction that would make them stronger for local development?
Paul Everitt: It's an interesting question. I haven't, and it will be interesting to see if some of these smaller models can come out of the labs already with that. I think Google did either a variant of Gemma or maybe Gemma itself was built with quantization, I believe, already. In there, like expected to be done like that. And so if some of these things can be baked into the model like recursive, then you might get this big gain. But what I'm talking about is like a a deeper rethink, which is move some of the work out of the model, move some of the loop out of the model. In the world of Python, we've got so much ML sitting around, you know, old school, old magic. And it can do a lot of the work on its own. It could have the equivalent of an agentic loop written in Pydantic AI or something like that. They can take a step, think, take another step, like a repair step or something like that. A scout that goes and looks and asks the Pyrefly type checker who knows all the symbols, not the strings, the symbols, that's so much better. And you can eliminate all these turns wandering around in your project trying to find the right thing. It knows the right thing. Can we make it use it? And if you're doing a refactoring, an edit, can we do that over on the what I call the left-hand side in the Python loop? Well, it turns out you can. Tons of libraries for doing little patch fixes in Python. Grab the AST, all this other stuff. And so I'm just interested in ways to reduce the amount of work a small language model has to do in agentic coding. If you can focus it on a smaller problem, it's not the galaxy brain, Dan. It doesn't have to cook your dinner. It's going to be much smaller. It's not a solar system brain. It's not a planet brain. It's a country brain. It's the Python brain. And we're going to find ways to make it really good at the way we work in Python. And I think this is achievable.
Dan Gerlanc: Are there any if you were I mean I know we're at the stage where the next libraries need to exist to
Paul Everitt: Right.
Dan Gerlanc: make this better, but if today I was going to run a local model, I have enough RAM,
Paul Everitt: Mm-hmm.
Dan Gerlanc: what is there a go-to you would use, like a local harness, like OpenCode or Pi? or if
Paul Everitt: Mm-hmm.
Dan Gerlanc: specific static analysis tools that exist now that you are helping to make this better.
Paul Everitt: I think the first point is you'll be reintroduced to your CPU fan. And the lap part of laptop will no longer be true because it will be too hot to put in your lap. And so you'll be if you did it right now, if you like took Pi or OpenCode and set up a a local and I'll I'll I'll use terms in the Macintosh. The Mac ecosystem. If you used LM Studio or used oMLX or one of these other inference servers, and you got a model and you hooked OpenCode up to it, or hooked Pi up to it, you're gonna have a certain experience if you don't have a cluster of GPUs. and that is kind of the door opening. It's possible, it's not probable. So what would you do? Well If you wanted the most capable model, I'm going to say a term most of your listeners probably haven't heard of, but they should. DwarfStar. The creator of Redis. DeepSeek Flash, which should require an acre of computers, and did a bunch of things to get it running on a very high-end Mac, like 128 gigs of RAM. But also on the NVIDIA stack and also on the AMD stack. And it has a bunch of interesting tricks, which I think are the d the next door opening showing there will be a generation after where we're we're at right now that will blow people's minds by doing things differently. And if you'd like, I could go into what those things are.
Dan Gerlanc: Yeah, I think that would be great.
Paul Everitt: Sure. And then we'll get into the middle tier and then we'll get into the lower tier. What is DwarfStar doing? The first thing is removing Python from the inference server. And he is a god at C and C optimization and Redis data structures and things. And he worked with, and he freely admits it, he worked with GPT 5.5 to write a new stack. In low-level, highly optimized C that takes advantage of the hardware that it's sitting on. And so that's the first big breakthrough. second big breakthrough regarding getting it all into memory. Once he had control of the inference server and he could control the weights and attentions and all those kinds of things. He got it down to a Q2 quantization. Quantization is the Q in a lot of model IDs that you might see. And it is, you could think of it like compression in JPEG. How to retain the important information and throw out the less important information. And in this case, it's more about. His decisions about focusing on the right things in the model, the right knowledge in the model. And Flash is from DeepSeek. It's open weights and he it's a mixture of experts. And so not everything has to be quote unquote active at once. A lot of the stuff that you don't care about is sitting there in the model in memory. So the second big breakthrough is. SSD streaming. Take the parts of the model you're probably not going to need and serve them from high speed flash. Let's talk about that one a little bit. Is that okay if we talk about that a little bit? The idea here is Apple's and maybe others, but Apple's NVMe story for SSD got really good. M1 was fast. Apparently, M2 was a step back. M3 was a step back to M1. M4 got a little bit better, but M5 is four times faster than the M1 chip. And so serving from disk, Apple wrote a paper called Flash-MOE or something about this technique. And it's a radical way to get a huge model squeezed onto a small device. It's also central to what Apple released last week, announced last week at WWDC.
Dan Gerlanc: Yeah, so then the Apple announcement from last week seems like there could be some foundational changes to how we can use local LLMs there. What do what do you think is coming from that?
Paul Everitt: Extremely bullish on this. this is similar to what I said about DwarfStar, that things are coming that will in the stack down to the hardware that will redefine your expectations for local AI. What Apple has done is a BFD. It's a big deal. It's a they're on device for Mac at least, I guess for iPhone as well. There will be a 20 billion parameter model. A lot smaller than what I just said about Flash, but bigger than QWEN 9B. And it will have some really aggressive Apple optimizations. They're moving from something called Core ML to Core AI. And the way the data structures in the models are implemented in hardware is changing. they're gonna take advantage of their neural engine more than they have in the past. The neural engine and the M5 got fused to the GPU so those two things can talk directly to each other. And now they're gonna take advantage of that. And they're gonna drop in the SSD streaming thing so that your phone can have a big model on it. It's nuts. But just the idea that there will already be a model. And an inference server sitting on your device, managed by them, is just a big deal. And everyone said it's the Gemini model. It's not the Gemini model, it's their model that they use Gemini to help on distillation or something like that. This is Apple really taking all the layers in the stack and lining them all up behind this local AI story. And I'm hoping I don't hear my CPU fans.
Dan Gerlanc: Yes, and it seems like I mean I have a Mac and an M1 Mac Mini, but
Paul Everitt: Mm-hmm.
Dan Gerlanc: seems like in this case their having control of both the hardware and the software layers gives them a significant advantage for local models, unless you had a NVIDIA GPU of some kind.
Paul Everitt: Right. I think you're exactly right. Now, in this whole local AI experience revolution, the first big thing to understand is manage your expectations. There's going to be cases where you need a solution with multiple levers. Maybe your big brain is still sitting in the cloud and your little brain is on your device. You know what? 80% of the tokens don't need the big brain. They're just grep. And so that's already a big win from better, faster, cheaper. And the big brain doesn't have to be in OpenAI or Anthropic. It could be because of OpenRouter, you could choose your own inference provider. You could self-host your inference provider at AWS or something like that. So I don't want people to ha to think it'll never be able to do everything. that I need. Cool. But it can do a large chunk of what you need. And that's going to be a big deal.
Dan Gerlanc: Yeah, that has been something I've w wondered about. I mean, right now, de facto people just go straight to the big models in part because
Paul Everitt: Yeah. Let's talk about that. Why do they do that?
Dan Gerlanc: I mean my main take is that you have a fixed most individual developers have a fixed price subscription.
Paul Everitt: Mm-hmm.
Dan Gerlanc: And it's just easier to do.
Paul Everitt: Yeah. Those days are coming to an end.
Dan Gerlanc: Yes.
Paul Everitt: I let me follow up with you a little bit more on this. People do that. I think. It's because developers just don't want to think. They want to type in a box any damn thing and have magic happen. They don't want to put in the engineering work anymore. What do you think?
Dan Gerlanc: For sure, whatever the I mean whatever the laziest thing that
Paul Everitt: Yeah.
Dan Gerlanc: can be done. And as Larry Wall would say, it is it is one of the virtues of a developer. though
Paul Everitt: Ha ha.
Dan Gerlanc: though maybe agentic development has allowed us to take that to an extreme somewhat.
Paul Everitt: Indeed. do you feel that there's a little bit of a backlash happening now that we've seen the consequences of this?
Dan Gerlanc: I think we still so yes, because I think most people who are working with agentic code are seeing that the code that is created is not as good as the code that a competent human developer would write. But it can still be delivered more quickly.
Paul Everitt: Yes. Yes, I I think
Dan Gerlanc: And there therein lies the contention, I think.
Paul Everitt: I think you summarized it exactly the way I feel about it, is that they embrace these tools, but price and quality are now becoming evident. The consequences are becoming evident. Certainly, price is very easy to look at and see. But from a quality perspective, I I don't know if this has been your experience, but I've done some really long-running, pretty advanced. experiments like building agent loops and things like that. And when I go on this run where I stop looking and just like, go keep going, whatever, invariably it falls apart. It gets to a point where the compromise on top of the compromise on top of the compromise on top of the compromise is no longer tenable. And I have to stop and Like redo the whole thing slowly and steer at every step. And that is my pitch for agentic engineering. I think professional development is going to need engineering skills. I think things like spec-driven development, where we work in small steps and make sure we know where we're going and where we went, are the best way to get good results from. All of this.
Dan Gerlanc: On spec-driven development, you have been teaching a course or you recorded a course for DeepLearning.ai on spec-driven development,
Paul Everitt: Mm-hmm.
Dan Gerlanc: correct?
Paul Everitt: Right. Yeah.
Dan Gerlanc: What should developers other than taking your course, what
Paul Everitt: Yeah.
Dan Gerlanc: what should they take out? as the top things to think about in terms of how they apply spec-driven development to agentic engineering.
Paul Everitt: That it's great to be an engineer. Be an engineer. You still have an important job to do. That's the number one take. But the kinds of things that are in agentic engineering versus software engineering are a little bit different. And we have to invent this field instead of YOLO. We at some point we need to hit bottom and realize the YOLO's not going to get us there. These one-shot prompts and don't show me the code is still sucking up all the oxygen in the discussion. The people that work at these money furnace companies want us to burn all the tokens possible. So take what they say with a grain of salt.
Dan Gerlanc: Yeah, my experience definitely has been for a very small code change, you can use a smaller prompt or try
Paul Everitt: Mm-hmm.
Dan Gerlanc: to let a very good model work through it, but anything even slightly significant
Paul Everitt: Yeah.
Dan Gerlanc: or within a larger code base, you have to create a spec, an implementation plan. And rely on tests, otherwise you're
Paul Everitt: look at it so that you don't get left behind. It's it needs to be your code. And when you work in spec-driven development in small steps, you're actually going to get better results from a context window and stuff like that. you're gonna do a better job of guiding the agent and the model to the destination. And then to tie it back to the first conversation, this is like the biggest hope for local AI in small language models is close direction. Give them the right sized packet of work to do for their capabilities. If it's a 27B or a 12B or a 9B or whatever, give them the right sized packet. And then give them a clear definition of success, like write the like red-green testing. Write the test first and then the failing test first and then go implement it. Spec-driven development, I think, is the biggest leverage point for succeeding at local AI, local agentic coding.
Dan Gerlanc: Would you think someone might be using a larger model to iterate on the spec interactively with a user and then create the plan and then actually have that implementation done by a local model once you have those instructions well laid out?
Paul Everitt: Well said, very well said. with spec-driven development, there's kind of this project kickoff phase. in Spec Kit they call it the constitution. If you did that in a really big brain like DeepSeek Pro or GLM 5.2 hosted somewhere, then they can the I said they listen to me. What is it called? Anthropomorphophores. I'm assigning agency to the agent. then you and the agent can collaborate on digesting the universe of your problem space. Which is way too big for the context window and the power you can put on a local computer. But it's a one-time thing. Go through a long conversation with the agent about what you're trying to do and let it write the result. It will write a pretty good result. that has your tech stack, your mission statement, your roadmap. And the roadmap will be organized into chunks, hopefully sized correctly for your computing power. Then you have the medium sized brain the little brain. You have the big brain, medium brain, little brain. Medium brain is like the orchestrator. It goes and grabs a unit of work from the roadmap and comes up with an execution plan, possibly in multiple steps, to hand to the subagents. And they don't have Any context except what they were given from the orchestration agent. Maybe that's a 27B model. Maybe it's DwarfStar if you've got enough capacity. and then a small model that's doing 80% of the tokens by just you know grepping all that stuff. And so this handoff packet between The orc medium brain and the little brain is kind of the secret to success for small language models and local AI, in my opinion.
Dan Gerlanc: This actually is how, I mean not necessarily with local models, because my hardware is pretty wimpy, but deferring to the smallest remote model possible that I've worked with, have built the plan with Opus or GPT-5,
Paul Everitt: Mm-hmm. Mm-hmm
Dan Gerlanc: and then use Haiku or GPT nano, something like that, to actually implement.
Paul Everitt: And what you'll find, let's stick with that scenario because we actually are going to teach the first three lessons in our course are using Claude and friends. Because we're trying to teach techniques about subagents and cheaper models and things like that. So you're exactly right. the big brain. Can craft a really good plan for the little brain. And it can review the work. And working this way is so interesting. You're like, okay, this time I'll use nano, GPT nano for the little brain. And then the big brain reviews, and you see, wow, it found some some more things than it found with Haiku or a Haiku class. And that's helping you in so many ways. It's reminding you about the constraints. Hey, I just optimize for speed and price at the expense of quality. It's also getting you back to engineering. Cause that contract, the protocol between the big brain and the little brain. There's a lot that can go into that that is up to you. I'm not even getting yet into telemetry and evals to see, speak with facts about better, faster, cheaper. That's more engineering that can be done. But what you just described as your experience is very well said, and it's also the path to local AI. If you don't have enough horsepower, sit in OpenCode and use the big remote brain for the thinking and your little local brain for the doing.
Dan Gerlanc: Is there a specific set of practices or prompts that you like to use for spec-driven development. I mean you mentioned Spec Kit, but I know there's like Kiro code has its own ways of doing it. There's the superpowers set of skills that a lot of folks use. Is there ones that is there a standard that you would go to or have you developed your own system around this? Or some combination of
Paul Everitt: Before I answer, what's your take? What are you hearing out there? What are you seeing, including I hate spec-driven development?
Dan Gerlanc: I mean I definitely see People Push back, spec-driven development verse. doing it as it's more work. So inherently
Paul Everitt: Right.
Dan Gerlanc: if you don't I think if you want to get good results then you're doing it. If you don't then you're probably in the cohort that is still not seeing good results from working with these models. But you do you need to work through that process of defining your success criteria and making that as deterministic as possible for the models and you as a human to verify.
Paul Everitt: I like the point that you just made. I had a slide in my talk at Andrew's conference about how do we know what is good? So having a success criteria that you and the agent agree to would be a pretty nice thing. And that is part of spec-driven development, the validation stuff. I sense the same thing you do. There is a cohort out there that loathes spec-driven development, or at least ridicules it, maybe, is a better way to put it. And fundamentally, for me, it comes down to do you want to write things that do you want to think or do you not want to think? And a lot of people using these don't want to think. They just want to, they're, they want their ideas to be more about the ideas rather than the execution of the ideas. They don't want to drive the car. They want to be a passenger and tell the driver where to go. but there's still a lot of people who do want to put in the work to guide things and get the results and know that they got the results that they want. So for that crowd, that's the audience for spec-driven development. I think it's also going to be important for local AI. I am currently a big fan and a heavy user and extender of superpowers. I started with one system and then I used Spec Kit for a while. Then I wrote my own, and then I adopted superpowers, and then I wrote a bastardization extension of superpowers, which yesterday I abandoned. Because I saw Superpower Six came out with some really interesting things in it. And when you dig under the hood of superpowers, you'll see some things happening. that are deeper ideas that will apply as we get towards agentic engineering.
Dan Gerlanc: What what was in your bastardized version of superpowers that you abandoned?
Paul Everitt: The superpowers crowd hate roadmaps. A lot of these spec systems hate roadmaps. And I just I've got the memory of a fruit fly. And I need to capture things and put them in a trusted place. I'm a getting a GTD. I'm a getting things done guy. And I need an outboard brain, a trusted system that I can put stuff and come back to it later. And so I'll do all these like research tangents. And I don't want to throw it away, and I can capture it into a research document, but I'd love to put it in a backlog. I'd love to group together work on a roadmap. And if things can go in parallel, I'd like them to go in parallel. If they are dependent on another thing, I'd like to record the start to finish dependency. superpowers doesn't have any of that. I added a tiny little thing back last night. Tiny little thing that I think stays within the spirit of superpowers to give me a roadmap. But the interesting things going on with superpowers now are pretty fascinating. And it turns out that superpowers Six landed on the exact thing I was researching for my pro my Python project for how to have a better small agent.
Dan Gerlanc: What is it?
Paul Everitt: The handoff packet, the protocol between the big brain and the little brain. And in superpowers, they have a number of specialized agents. And the main agent invokes them as needed. You know, the planner and the scout and these other kinds of agents. and the main, those subagents really have no context. By definition, you you don't want them to have your whole history of your conversation. They have a job to do. Do it, throw away your contacts. So you give them exactly what you want them to do, and they formalized that into kind of a handoff packet, same structure every time, telling the s sub agent, the smaller agent, how to please them. But they've gone further into things like choice of model. Hey, this handoff packet is pretty complicated. Sub agent, I want you to use DeepSeek Pro. This one is just routine, piddly little stuff. You can use DeepSeek Flash or even a local model. and that's useful orchestration. They also did something to dramatic superpowers had a reputation for blowing up your context. And so they have a dramatic reduction in token usage. But these ideas of welding your planning and your execution together so that your the system you're using can be better, faster, cheaper. For me is a path on the way to local AI.
Dan Gerlanc: Yeah, for sure. I'm a big fan of superpowers. I do find it works well for up to large changes. The part that maybe you have experienced and had worked on on your own. System and I yeah. Yeah.
Paul Everitt: if you'd like to. we could also talk about the downsides of spec-driven development
Dan Gerlanc: it's not really something I've talked too much about, so would be interested to hear your thoughts on that.
Paul Everitt: I'd say the biggest knock is like you've moved from code debt to markdown debt. Very few of these systems expect the artifacts, the specs, to be living. You write them, the code is created, and they are immediately out of date. There are theories out there, TESSL, T-E-S-S-L, I think, is trying to pursue a direction where the spec is the truth. You can throw all your code away, replay the spec, and you'll get back to the kind of place you were at. Not the exact place because it's not deterministic. So I'd say that's the first biggest problem is drift, drift between the spec and the ongoing code. And I mean another one is if you're in the multi-person project and not everyone's using that spec system, they're just committing changes to the code. The spec doesn't know anything about those changes. The second is when you my projects have gotten to bazillions of lines of markdown. And at some point, these systems have like cognitive debt. I have cognitive debt. I can't remember yesterday anymore. And the system can't l keep the entire corpus of specs in its brain at once. And so you want a better spec system that can archive things, that can partition things into modules in the project so it can kind of focus its attention just on the part that you're trying to work on.
Dan Gerlanc: Yeah, that's a part I wonder about and not something that's really focused on in superpowers is what is the declarative view of your system? How do you document that?
Paul Everitt: Yeah, right. And do people even want that? Do people even want that? You do the superpowers brainstorming and it's like, okay, I've before we proceed to the design, I just wrote the spec, review it. And tell and before we proceed. You know what everyone types? Proceed. They don't review the markdown file. They don't review the design file. They don't review the plan file. They just want to get through the code. And that's just a challenge. You'll never have a spec system that will succeed with people who don't like specs.
Dan Gerlanc: mean I don't think people want to read the markdown files. Should you?
Paul Everitt: Mm-hmm.
Dan Gerlanc: Probably yes.
Paul Everitt: And this comes back to my original question, do people even want to see the code? Do they want to own the results? Is it if it's under your name and you never looked at anything? People cool with that?
Dan Gerlanc: I mean I guess it depends on what's at stake and how reversible it is.
Paul Everitt: You just stated the the primal truth that I've kind of arrived at is when I'm just messing around and I'm never gonna show it to anybody, I'm like, go. Right rewriting. The fourth attempt at this thing, intending to publish it on PyPI for the world of Python. And you know what, Dan? I'm going real slow in looking at every line of code and fixing the tests and all that stuff, because it's gonna have my name on it, and I don't wanna be embarrassed in front of the Python community.
Dan Gerlanc: So at that point is it faster to write it yourself or do you still get a boost from working with the models?
Paul Everitt: field that you know really well, and you have a hundred years in the ML space, you probably could write the project about as fast as DeepSeek Flash. Because it's going to be rewriting and making mistakes and correcting and all that stuff. But if it's something you don't know, and that's me, I don't know this stuff, then I would have to go learn it. And we're talking months. So it would not be faster for me to just write it myself because I don't know this field.
Dan Gerlanc: Yeah, I think that side of things you can write code now that wouldn't be viable because that time you're spending on that is also then not time you're spending on the thing you already knew well. And then those skills atrophy or you're not keeping up with the latest.
Paul Everitt: Well said also. And we're not even getting into the you know, breakthrough techniques on arXiv.org that you don't have time to read all these academic papers every week.
Dan Gerlanc: How do you keep up with what's happening if that's even possible?
Paul Everitt: Really, I don't. What I do is I have some kind of suspicion that the idea that I'm having has already been had. And I have a little agent skill that knows how to there's a MCP CLI thing for arXiv.org And so I can tell the agent, hey, I'm thinking about this, that, and the other thing, go do some deep research and see if any of that fits our project. And it comes back with some fascinating opinionated opportunities for spikes and experiments. This has happened repeatedly.
Dan Gerlanc: Is there an example of that that you can share?
Paul Everitt: Yes, it it is the thing that is sitting on my screen right now. You are exactly right. I've got a specific example. In the world of Python, there's our type checkers. Meta has one called Pyrefly, written in Rust, super fast, pretty mature, just hit 1.0. And the thing I'm trying to do is to help. A small language model complete Python tasks faster and better by using a type checker. The most common way to use a type checker is through the LSP. And I had started with go do some research on LSPs and LLMs. Big field, pretty popular. But as it turns out, Pyrefly has some capabilities that aren't exposed through LSP, that you have to talk to it directly. important, very important in fact. One was in Python we have structural types, not just nominal types, also known as duck types. There wasn't really a way to use duck types through the LSP, but the CLI had a way to go locate something that had the right shape. So now my project can do duct typing because I said Hey, you know, I'll bet Pyrefly does have something. I just cloned the Pyrefly repository. Go look around in all that Rust code and see if you can find duct typing.
Dan Gerlanc: Do you have to define things as protocols or does it determine it as long as you're not using super dynamic behavior, it can verify it.
Paul Everitt: Dan, I wish we could talk for five hours because that is a very penetrating and prescient question. what is the work the agent is trying to do? You're telling it to do something and it's got to go look around the code base, find candidates, and operate on them. And if you've got a kind of thing, like an is a or a kind of relationship. It's not a subclassing relationship. It is a shape of the object relationship. And in Python, you can now write protocols, like you said. But there isn't a way to associate an implementation with the protocol. The PEP in Python says you can state it yourself by subclassing from the protocol as a way to kind of declare that I'm an implementer of that thing. Nobody does that for reasons. It messes up the MRO. But if you were able, if you had like this Rust thing that was faster than the CERN super collider, and you could say, go find me every symbol in the code base that has this shape, that would be pretty cool. Then you could do duck-typing in your agent.
Dan Gerlanc: So are these the kinds of tools you think we need to continue pushing local models? I know we we talked about a little of this earlier, but this is the next horizon potentially of where we need to go.
Paul Everitt: Yes. I think that there were there are thousands of people that in the world of Python with masters and PhDs that have been thinking things forever that have a hundred ideas better than me. And we can think about this problem and do something freaking awesome. I'm in a unique position because also at JetBrains, two weeks ago, we released a small language model called Mellum, Mellum 2. and it is a coding model, 12 billion parameters, 2.5 active. It's a mixture of experts. And it is open weights, but it's open source. And in fact, it's ethically trained, and it comes from Europe. It just checks a lot of these. You know, the moment we're in and the anger we feel at our industry. It checks a lot of these boxes. And the Mellum team are pretty interested in, you know, what could be done that's new and interesting for agent coding that's local, like better tool calling for an LSP approach. The field of better tool calling for a model is wide open. That was the realization my experiment got to was the Mellum struggle wasn't really just about location. It was about patching.
Dan Gerlanc: So that is an area I have some experience in on the structured generation side. So in theory, if you know what your tool calls need to look like, you can create these schemas ahead of time and modify the logits such that any tool call you generate is guaranteed to be syntactically correct. May not be semantically correct, it could make no sense. Which is I think a lot of why people have slept a bit on structure generation is larger models tend to get both the syntax and semantics correct.
Paul Everitt: for local AI, turns are the enemy. So what you just said matches the research that I've done. It's like a three or even four layer problem about kind of the string representation, the semantic representation, the structural representation. There's even one beyond that. and they all might have different techniques for how the model needs to tell the harness. To do it. Because if the model, instead of trying to generate it itself, could say, Hey, you're sitting over there with an AST, and I want you to refactor this line, why don't I just give you a description of what I'd like to do and you keep working on it until the type checker passes, the formatter passes. some kind of CST analysis passes or whatever for semantics and then hand it back to me for review.
Dan Gerlanc: Yeah, it's like if you need to do a global rename, right? Should you you why use sed when you could use the LSP implementation or if you're in a JetBrains product, the built-in version, because depending on what language you're using, you might miss stuff with just a pure rename.
Paul Everitt: You're exactly right. Trust me, JetBrains is thinking about this issue space. But it's interesting what you're saying about the logits and there there's a lot sitting in there with attention and layers and weights and and all of this machinery. But it's all pointed for the last few years it's been pointed at this kind of YOLO galaxy mode. And in fact, even tools were an afterthought, right? I mean, the original model was chat.
Dan Gerlanc: Yeah, a hundred percent until a bit more than six months ago 'cause it came to chat first and web web search and things like that,
Paul Everitt: Yeah, for those of you that want to play with loc small local models, you'll find the biggest first problem you're gonna have is the format of the response gets mangled. It tries to put a pile of JSON in the middle of a string and it's bad JSON for the tool calling information. Cause they weren't built to do tool calling and and JSON structured responses. Structured output is a problem.
Dan Gerlanc: And there are solutions to it. and I think, if you're doing more AI engineering on that side of things, people have been using structured outputs, I think, since they were available because the cost of Doing a retry is big in terms of the user might have more delay or things like that. So it's inherently or you have more cost as you're bearing it with another LLM call.
Paul Everitt: Let me ask you that question then. if you were building a model right now for agentic coding, how and we'll maybe we'll say a small model on a a laptop or something. Maybe not. What how would it be different? How would you what if you could just snap your fingers and make something really fit for purpose, how would it be different?
Dan Gerlanc: The most success you will have is the more you can train that model to your specific use case. And so this was something I would always say in my .txt and structured generation days, is there's a few different levers you have for better model, better performance. One is use a bigger model or a better model. Two is fine-tune to your specific problem given a size of the model. And then three is throw in structure to your specific problem. And you can do any or all of those three. So I think for a a coding model, if you know you're gonna be generating a certain kind of code or things like that and have the resources to do it. Any or all of those three that you can apply is going to be give you better results.
Paul Everitt: But if you could have a new model architecture, like if it could be a JSON RPC type of interface where you're getting the tool calling information back as a JSON native data structure, not some Pydantic model that's converting a string into an object, or something that natively understood tool calling. Is there something you would do radically different?
Dan Gerlanc: I don't think so. I mean, I think the main thing that matters is that you define a format and stick to it. There's a lot of JSON on the internet to begin with, which is historically why it's tended to work well. There's already a lot of JSON that the models have seen. But in general, whether you put it into JSON or a mix of JSON and plain text, as long as you define the contract and know what to expect. I don't I don't think it matters too much as long as I guess the the closer it is to the training distribution, of course, the better you're always going to be.
Paul Everitt: On your point about training data, the I've been talking with the people at flower.ai and they are federated learning. Really interesting group headquartered in Germany. And just you know, a hypothetical, imagine if the world of Python contributed training data. And it's federated learning, and we could solve the problem of licensing and feeling like you're not being exploited. imagine a model on Hugging Face for Python 3.15 beta 2 that in addition to the help being kept up to date, the model was being kept up to date. right now I don't think any of the Frontier models are aware of Python 3.14, which came out in October. Yes.
Dan Gerlanc: No, and I still if I generate Python code it's always declaring from future annotations for f features that are already in the version I'm using.
Paul Everitt: Yes. Yes. And my agent rule file says never do from never, never, ever, never, mega, never, ever. I still struggle to get it to to not do that. You're exactly right. That's a a perfect way of describing it. But I think you're right that that is the beg the biggest lever is to get the definition of right into. A community maintained model. not just right but best, opinionated. so that a small language model has a better shot at getting it right the first time and not having to do five turns
Dan Gerlanc: Yeah, I mean I'm hoping we see this as cost becomes more of a concern. Before it was just the progress in the frontier models was too fast to I think really make it worth investing in these unless you had a very specific use case. Like there are companies like like Liquid AI that fo specialize in small models.
Paul Everitt: But as far as I know, they haven't yet pointed it at the direction of coding, right? They're keeping it under eight billion parameters aimed at verticals. But they have a different approach on some of the fundamentals and it's pretty interesting. I hope that when if you invite me back at the end of the year for a recap on how it went, that it's a great positive story, that we don't have to cover fifty percent of the Earth's surfaces with data centers or have data centers in orbit. And why? Because we made some changes so that we can do most of the work locally. there's just so many reasons for this. Better, faster, cheaper, but ecological, whatever. but we've just got to put in the work, bring back the engineering.
Dan Gerlanc: bring back the engineering. It's not it's not over yet. There's there's value in being a classically trained software engineer, as I like to call it.
Paul Everitt: Indeed. Indeed. the God box is enticing, but just like in the Odyssey, the sirens call leads to bad consequences.
Dan Gerlanc: That's right and and what happens when you get tied to the mast?
Paul Everitt: Yes. I like the the direction this has gone. How we can get out of agents what's best for us and what's best for our profession, the people who come after us, et cetera. things are happening so fast, we have a tendency to just be passengers in our own car. It's time for us to get behind the wheel.
Dan Gerlanc: Well, Paul, th thank you for joining us and will be great to have you on again.
Dan Gerlanc: That was Paul Everitt, developer advocate at JetBrains. You can find Paul online and check out his latest courses with DeepLearning.AI. All the links will be in the show notes. This episode was recorded June 18th, 2026. I'm your host, Dan Gerlanc.