You are literally just burning context dollars because your giant AGENTS.md file or CLAUDE.md or whatever is filling the context and it's not even making your agent more useful. Take a look at this. Here we have that Claude Code has actually removed 80% of Claude Code's system prompt for more advanced models and this has just changed everything and this is one of the things that frustrates me to no end with our AI agents and trying to figure out the right workflow is as awesome as it is that our models continue to get better. It's super frustrating that our workflows have to constantly be changing for that as well but that's the world that we live in and we adapt to it. This is Better with Kent where you and I get better together by learning durable skills and while it may not seem like a very durable skill to worry about updating your AGENTS.md file, it actually is a valuable thing for you to do today and so that's why we're jumping into this. Let's continue to look at this blog post from the Claude Code team and so you can go through a lot of this yourself. I want to focus on a couple of specific things and so we've seen some massive improvements to the AI models both out of Anthropic as well as all of the other labs and as those models change we have to change our workflows to adapt to their improvements and typically those adaptations to our workflows are actually simplifying things and it's very possible that you've got documentation for agents that was useful for older models that are no longer as good as the frontier models that you might be using now. So here one of the big problems that happened in Claude and this is even before the models got good was that our system prompt skills and user requests clash with each other. So this is like in fact I have actually not been an advocate for very large AGENTS.md files or skills like I honestly don't use very many skills at all. I've made a couple of recent videos about the handful of skills that I do use but in general I don't have those auto-loaded. I normally bring those in on demand because of this very problem where maybe the system prompt says leave documentation as appropriate but then there's a skill that gets auto-loaded that says don't add comments but then the user prompt says just make it work like the old one and the old one had documentation or something. So the agent now has to reconcile these conflicting things and so in general you want the system prompt and the skills to be more generally applicable and not have such specific instructions especially if they're loaded every single time and now you probably don't have control over the system prompt unless you've built your own harness and so you don't have to worry so much about this but you do probably have skills and an AGENTS.md or CLAUDE.md file that you do have to worry about this in and so that is why you care about learning about these things. So one other thing that I find really interesting in their blog post here is they say there are a number of previous context engineering best practices that had become myths at this point and again this is why I do think that it's valuable for you to understand how harnesses work and how the models work because this is going to help you use them better but one of the best takeaways you can take from this is that you have to constantly be keeping track of the workflows that you have now if you're going to be adopting new models because things do change over time and things that used to be best practices are now no longer best practices and so these are all the headings from the rest of this blog post that used to be completely different like a year ago or even six months ago which just goes to show how much this industry is changing by the advancements of not only AI models the LLMs that power everything but the harnesses that harness their power. Alright like I said you can actually look through a lot of this lots of these are new best practices and we'll see how long these hold but if we look at the CLAUDE.md or this applies to you if you're using an AGENTS.md like I am same sort of things apply to you here so here it says avoid stating the obvious things that your agent should know just by looking at the file system the repo good tip and use progressive disclosure heavily. I've actually been doing this for over at least nine months and this has worked out really well for me and I actually have some numbers that I'm going to show you a little bit later on in the video and actually this reminds me of a blog post that I made a long time ago about React talking about when to useMemo and useCallback and kind of harkening back to the thing I said at the start where you're literally burning context dollars doing all of these optimizations that you find are not actually all that useful and that's exactly what was going on with the useCallback and useMemo as people were using these things and you don't have to understand React to be able to get value out of this but they're using these as optimizations without actually measuring whether they were optimizing anything at all and the tricky thing here too is that the the framework underneath us is not React it's agents and they're constantly changing and so you might have added an optimization at some point and then boom everything changes underneath you and now that optimization is actually a de-optimization which was my whole point in this blog post and so here toward the end I come out with this like statement that I want you to remember forever and that is performance optimizations are not free they always come with a cost but do not always come with a benefit to offset that cost same exact thing is going on right now with all the optimizations you're making around your agents they are coming at a cost and there's like the actual cost of the tokens that are added to every prompt but also the cost of maybe distracting the model and then the fact that the agents could get better and now it's a cost that's not proving any benefit so you do actually need to measure this and see how things are actually helping you or whether they're not helping you at all and what's interesting is there's actually this research paper evaluating AGENTS.md and whether they're actually all that useful and not to spoil anything for you but they are not often and so providing context files like AGENTS.md does not generally improve task success rates while increasing inference cost by over 20% on average now you can dive in deeper into this paper because I think that not all context files are the same and that's what we're going to take a look at here in a second but I think the idea that just adding more context is really proving to be just adding more cost to every task that you ask and so not all context is created equal and honestly like you can't really blame people for this happening it's a very natural thing to do where the agent does something wrong and so you're like well I never want that to happen again so you're going to create a rule or a skill or add it to your AGENTS.md to make sure the agent never does it again and you just keep repeating this loop for months and months until all of a sudden it's just a big ball of mud so let's take a look at a couple AGENTS.md files to see what I mean so here you start out with what is probably a pretty straightforward way to do this hey you always want to write DAG in title case in all the prose like they probably saw agents writing things in funny ways and here are a couple anti-patterns and stuff and then oh we want to make sure that the agent knows the environment we don't want it to try an NPM install this is a Python project after all and you know what it would be kind of useful to include all these different ways to run individual tests and everything so let's add those and maybe this was built at a time where the agents weren't quite as good as deriving all of this from the code base but now they very much are and oh let's make sure that it knows the repository structure so it doesn't have to shell out and find out where all the files are now this is not a problem that we have at all and oh yeah agents need to know about architecture and security and the shared libraries we're using and coding standards and pretty soon like this just gets to be a really really big ball of mess and it's not helpful most of the time according to this research paper and my own personal experience here's another one where we've got a whole bunch of stuff on debugging tips and testing and the way that we want the agent to behave inside of this where all the packages are what NPM or PNPM that we're using all of that is included in this AGENTS.md when the agents are actually pretty capable of managing and discovering a lot of this themselves so like the basic rule is if the agent can find it by reading the repo then it probably shouldn't be in there now there is something to say for token efficiency and things like that but in general if the agent can find the information it needs by like going through the repo then you just keep it out of the AGENTS.md and that gets us back to the progressive disclosure that Anthropic was talking about with Claude Code where it's better for the agent to discover the things that it needs rather than just having everything in context from the very start because it might not need those things when we're talking about the stuff that's included from the start that should be very general generic stuff that the agent can that the agent definitely needs right up front and then the agent can decide what pieces of additional information to load into its own context this works really well it's been how i've been operating for a while now now let's take a look at one of my own AGENTS.md files and i literally just made an update to this one 26 minutes ago but i'm even just looking at it right now i'm like nah i did not remove enough so we're going to do this together and we'll see how this proves out of course we need to measure it long term and everything but uh yeah so first of all we really don't need to have this first line these are instructions for yada yada no it it knows what these things are so we're going to get rid of all of this though i do think that having this kind of guidance could be useful i think we're going to get rid of this entirely and then this is an explanation of what this project is i actually do think that that is useful to have in the context for every single prompt that i put inside of the Kody repo it's useful for the agent to know okay what where am i right so like that is something that we're going to keep in here i do think that that maybe is a little bit too much right here the the general like what this thing is is useful enough everything else i think the agent can discover as it needs here we're saying use node 26 the agent anywhere i'm running this agent should have node 26 already installed and it should be able to just very easily see oh i've got a package.json and a package-lock i know that this is npm and also the npm run validate i'm actually going to leave this one in because recently my agents have not been running the git hooks because there's some sort of bug right now with that and of course that's outside of my control it's the bug with the agents and so i'm going to leave this part about running validate in but this is one of those things that you have to come back and loop on and make sure that it's actually still providing value so i'm going to leave this piece in as well as the validate fix i don't think that i need that one in there anymore so we'll get rid of that as well and in fact what a single line on this is sufficient most of the time in my experience so this file is intentionally brief i am going to leave that in there because i don't want agents to proactively add more and more context to the AGENTS.md file and then detailed instructions live in focused docs and then here's the index and i'm happy with this index of course the agent can find all of these things by itself and based on the numbers i'm going to show you here in a second it actually does do this so we're going to leave that index in here and then also Cursor Cloud specific instructions can be useful i use Cursor Cloud agents a ton and so having some like gotchas in there landmines and that sort of thing is useful and then pointing here so that if i'm running an agent locally it doesn't have to deal with all the context of stuff that's inside of those documents now the contents of each one of these documents that can be a whole video unto itself of like how much do you put into each one of those relative to what the agent can discover on its own but you do want to be like chopping away at this you want your agents md to be a really small file and even the documentation should be relatively small for two reasons one you want to make sure that it doesn't fall out of date though agents do a really good job of keeping documentation up to date but then two it's also a context problem where it fills it with way more context than it necessarily needs on the particulars of like what it needs and this is one reason why we don't just put it all in a single agents md and why we also don't put it all in a single document we have these split up across different documents so the agent can open up the one that it specifically needs and honestly humans are the same way though if a human opens up a file it doesn't instantly load the human's brain with everything inside of that file and maybe there are some harnesses that are more intelligent about reading large files so that it's like okay let's just read the headings and then we'll you know similar to the way a human would do this but in general i think it's better to break things out into individual files so that the agent can just grab the files that it thinks it needs and load those into its context all right so let's go ahead and get this committed we're letting Copilot write our commit messages who writes commit messages anymore but yeah let's go right here this is some analysis that my agent did for me on the last 24 hours of my Cursor Cloud agents as well as local agents so it analyzed 37 agents and each one of them it wanted to decide whether that agent ever reread agents md and of course it did not because it's already pre-loaded with that but there's some proof it never needed to read the agents md again but then 86% of these pulled documentation and skills and if i broke this down i don't have skills that are automatically loaded so some of them proactively either through something that i said like with the slash or decided that it needed to go and grab a skill and all of these did grab the relevant documentation and lots of them actually opened up just like directly to the leaf node so lots of my index in my agents md points to other indexes but for the vast majority of these they actually just went straight to the document they needed they didn't need to do that extra hop from one index to another which i find is pretty interesting so you really should be talking to your agent and asking how effective are the documentation pieces that i have available to you at helping you find what you need and not overloading you with too much context this is something that you really should do so really your agents md should just be a router that's the primary thing that it's useful for is it helps the agent find what it needs to find and so your agents md is like a tiny always on map for the agent it always knows where to go to find the additional progressively disclosed information that it needs and then that can map to other maps and in my experience typically the agent just ignores the other maps and it just finds the rest of the stuff that it needs so maybe you don't need sub indexes uh Kody in particular the one we were looking at Kody is a quarter of a million lines of code and so i do feel like having some index of indexes and things could be useful as your code base grows and if you've got multiple millions of lines of code it probably makes sense to have some sort of tree structure of your indexes but in any case the index just links to all the other documents and then the agent can go and find the relevant pieces to what it needs to do for the specific job you've asked it to do so your agents md is just a router for context so whatever you're using you probably have some mechanism for determining where all of the context is coming from and seeing that context grow here in Cursor we've got this little circle thing that says how many tokens are being used and i literally just said say hi and this is the system prompt is 451 tokens which is not a lot tool definitions 9.1 i think i've got like uh well Cursor has a whole bunch of tool definitions themselves uh and then rules 3.5 i honestly don't even know where those are coming from so that would be a job for me to go and look around at those skills 1.7 thousand i honestly don't have any skills that are automatically loaded so another thing for me to look at and then 5.8 thousand on MCP and dynamic tools which interestingly some agents actually will dynamically load MCP servers as needed to reduce the amount used there what's interesting for me is that i have Kody in here along with a couple other MCP servers i probably don't need and Kody has the capabilities of like doing thousands and thousands of things and so because the way that Kody is structured with code mode it's really really shrunk down if i were to have all of the context necessary for all the things you could possibly do with Kody it would definitely blow past my context window by a large margin and so being smart about the MCP and other tools and skills that you have in there is pretty useful and then we've got sub-agent conversation stuff in here you know that's there's not much from the conversation and hopefully most of what is going into this context usage is actually from the conversation and the tool calls that are being made to go and read files and stuff like that so your job is to shrink as much as you possibly can all of the stuff that shows up in this context before you have even said a word to your agent and so that's the homework for you it should take you like 15 30 minutes talk with your agent about the agents md file and rip out all of this stuff that is discoverable from the repository itself keep some of the non-standard non-default things that seem to trip up your agents today on a regular basis but limit those significantly and if there are a lot of them then make a document and point your agents over to that and only include the ones that happen literally every time no matter what you're doing in that agents md and then yeah make an index of all of the other important and useful documents like if you're going to write tests here are the testing principles if you're going to interact with our security aspects or authentication or whatever and then here's like how that is structured and all those sorts of things put those in documents that the agent can go and look for and then maybe the agent never uses those documents and that's fine it just sits in your repo is dead weight and nobody's going to look at it and it's not a big deal but it's not taking up room in your context window and you can evaluate over time and look back at transcripts and say has anybody ever looked at this documentation then let's just delete it if nobody has so that is your homework is to slim down your system prompt loaded stuff like your agents md i hope that you enjoyed this episode of Better with Kent where you and i get better together by learning the skills that we need to make us the most efficient programmers around thank you so much for spending some time with me to get better and please if you could like comment and subscribe and share this with the people that you know and love that would be super helpful to me as well. Thank you so much for getting better with me.