WEBVTT 00:00:00.000 --> 00:01:27.000 If you're listening to this podcast, you've probably used a large language model with billions or trillions of parameters. In Bayesian modeling, the number of parameters is usually much fewer, because the parameters are chosen to reflect, as well as possible, the mechanics of the real-world problem. In this episode, we explore with our guest how AI agents have made Bayesian methods more accessible to non-experts, and how Agentic Engineering has finally helped deliver on some of the original promises of data science. Welcome to Agents and Engineers. I'm your host, Dan Gerlanc. Today's guest is Thomas Wiecki. Thomas is a co-author of PyMC and the founder of PyMC Labs. He holds a PhD in computational cognitive neuroscience from Brown University and previously served as VP of Data Science and Head of Research at Quantopian Inc. In this episode, we also discuss how Thomas's work in grad school building models of the brain led him to Bayesian statistics. How effectively embedding agents in tools like Slack or Discord can democratize access to company information, and what parts of data science still require talking with humans. Enjoy the show. 00:01:27.000 --> 00:01:30.000 Thomas, great to have you on with us today. 00:01:30.000 --> 00:01:33.000 Yeah, excited to be here. 00:01:33.000 --> 00:01:58.000 So I know you guys have, aside from doing lots of Bayesian PyMC work, have been getting into agentic engineering and applying that to a lot of your workflows. Tell me a bit about that. How did that get started? Where has it taken you? 00:01:58.000 --> 00:05:27.000 Yeah, I mean, for me, it's just a natural extension, I think, of what I have been doing and where my interests lie. So I always have been interested in technology more broadly and really how technology can help us solve applied real world problems. And so when I started studying bioinformatics in Tübingen, I mean, I picked bioinformatics in the first place because it's the application of computer science to biology problems. So I always thought that that was the interesting combination. And then that led me to double down on neuroscience and how we can like build models of the brain to do that. And then of course, there statistics and machine learning comes into play, right? Because that's really the type of questions that you're, that we're after. And that's where, yeah, I really became interested in Bayesian modeling and found that this is like just such an amazing method to like really apply these tools. And back then I was in grad school, so I had a lot of time to really immerse myself deeply in it and understand all the core and then got involved with PyMC, of course, and this big rewrite back then called PyMC3. And that just was an amazing distraction. I think everyone needs that real project to be not their dissertation or their graduating. And that was sort of mine. Then of course with building PyMC Labs where we like help companies use these methods to their fullest extent. I mean, it's amazing because now we can really, I think realize what most of us in our Bayesian corner have always like believed to be true is that this is like such an amazing tool to answer questions in like a causal structured way, right? We're not sort of just. dumping a whole lot of data in and you're hoping that like some machine learning algorithm will figure things out, but you're really thinking about the problem from first principles, building a very crafted custom model for that particular use case, right? And then asking very pointed questions where you get them probabilistic answers and critically like the combination with the causal aspect, right? Where like, if you want to apply that causal understanding, to the world and then to your models, you can essentially think about this as like a scenario simulator, right? We're like, okay, well, and you're asking all these what if questions, which are causal intervention questions, right? So what if I did this? And that's what makes it so actionable, right? So it's not just about like, well, what do I forecast? What do I think is gonna happen? But really, how do I affect the outcome, right? What actions do I take to get to the outcome that I want? And if I'm thinking about like actions and manipulating the world in whatever way, right? And there can be rebalancing marketing portfolio or like a particular intervention in like a health setting. What mechanistic path does it take from that intervention to the outcome, right? And then once I have that understanding, right? I can like optimize and like come up with the best. So really it's integrally connected with decision science. But as I alluded to, right, like, 00:05:27.000 --> 00:05:57.000 Yeah, I think a lot of people don't know what the di like, I guess maybe I do think you were talking about it, for folks who don't know about the difference between a Bayesian approach versus a more traditional approach to machine learning or statistical learning. I know you're no you're getting into it, but like a quick I guess a quick summary of how those approaches are different and what they let you do 00:05:57.000 --> 00:09:57.000 Yeah, absolutely. the I mean, there's all kinds of ways of comparing Bayesian modeling to other things. The most instructive, think, is machine learning, because that is like the big competitor. And actually, it's not really a competition, because I think just like the problem domains or the type of things you want to answer are usually quite different. But I would argue that you actually want to that most of the questions in business, certainly, are actually causal questions at heart, right? Because what I mentioned with the taking actions, right? So in business, really, do you just want to know like, like next quarter's sales are forecasted to go up or down, right? So that would be, that is good to know, right? So having just like a black box model where I don't really care about like how it made that particular forecast, right? How it came up with that. So that's why these models are often called black box. That then is like good to know, but really, I mean, the next question will be, okay, well, how do I stop it from going down? And then like things like seasonality come in, right? And those are things that really, when you include in the model, and that's the core difference between a Bayesian model where rather than just like fitting some relationships, some correlations, you're exploiting some correlations in your data, to predict the outcome, here we're applying already what we know, building that causal model. And then with that mechanistic understanding, we can simulate the world and then we can manipulate basically this simulator to model scenarios where we do take certain actions. And then we know basically what our decisions really lead to and then select the best one. So, yeah, so, but the cost that you pay, right, is that you have to build that causal model in the first place. So that's like the effortful part. And that's why companies come to work with us because they're like, okay, well, we understand that there is this model that is really powerful once we have it, but the question is, how do you get it? And so we come in and we do that and it's quite difficult, right? So the getting like a model that is really trusted and verified and has good convergence and all the nice properties of what you want, right? A that you can trust ultimately, right? That's what it's about. And that scales. that is requires a lot of work. You need to work with stakeholders to like really understand it. And that is extremely valuable work, right? Because in like solving the problem, you also understand the problem and learning about the problem. So I think that's a fair cost to pay in a way. But obviously if you can avoid it, then you'd want to make it faster and more user friendly, right? And that has always been the thing that really bothered me with PyMC and Bayesian modeling in general is that it's just like really, hard to use, right? So the software is like from experts for experts. And now with AI, these methods can be made accessible to a lot more people, a lot more intuitively. And that I get incredibly excited about is that like this premise back then of like, well, let's just really revolutionize decision-making on a global scale by having like an open source package that provides that, right? And you can apply in all kinds of different domains to really have that impact. If you have expertise, well, if we lower the expertise, we can multiply the impact that we're having and just build like really, really smart, better decision-making systems. And that, think, the agents will play a fundamental role in. 00:09:57.000 --> 00:10:22.000 So on the Bayesian side, yes, to summarize, you are really working bottoms up in defining how the business process or whatever you're modeling works and iterating with that the companies or organizations you're working with. 00:10:22.000 --> 00:12:11.000 Yeah, like bottom up, think is a good way. Well, like, yeah, you start with like, okay, well, what are like the things that affect my outcome, right? So in marketing, that will be like the weather, right? So a lot of models would have weather included or just seasonality factors, like in general, in summer sales might go up, right? And then, but then because sales are going up in summer, That's where also maybe people are increasing the marketing spend already because they're now like, okay, well now like increasing marketing spend will have a big effect. And then the classic mistake is you like run your statistical correlation or model and or like a machine learning model and we'll say like, okay, like this seems to really strongly predict the outcome but it's actually not the predictiveness of the summer. It's actually just like that people have increased spent already. because they know it's going to pick up. So there's all these different ways of how you like get confused by these confounders if you don't sort of distangle them and then having those effects in your data set that you just like train and classify on that will give answers that accurate, but they don't tell you like, okay, well, what are like the underlying actions? it because I anticipatorily increased my marketing budget or is it actually that like there is like more demand or just like increasing marketing effectiveness during that time period. So yeah, it's about like identifying the causal mechanisms behind things and then you can actually like, yeah, start to make real decisions with real world interests. 00:12:11.000 --> 00:12:41.000 And you're building essentially a custom model for each of these versus a lot of I mean, in like deep learning models, like that's I guess the model is gonna be a little more custom, but with a lot of other machine learning techniques, you're kind of taking the same technique, here's a bunch of data, figure out what's going on. It's not necessarily customized specific to each problem. 00:12:41.000 --> 00:15:41.000 Yeah, and yeah, so I mean, doing that, right, there's like all kinds of problems with it. I mean, one thing is that used to be like a big challenge for us, right, as we build these really sophisticated models that are like amazing. And then we used to like, if we're like already working with like quite technical data scientists, we maybe sent them like a Jupyter notebook with like all the things and documented and or maybe like a whole package basically that can fit the model on the data. But in that we find it's just really difficult to really use, right? Because then you need to become the user of that package or just become intimately familiar with that notebook and requires lot of expertise once we leave, right? So now we just basically give them an agent. that like knows everything about the model and can like work with the model and has like really nice interface where we have a dashboard and then right next to it you have a just a chat window, right? And then you basically have the you can talk to a model really. So before you're like, oh, okay, well let me go into the Jupyter Notebook and like figure things out. Well, now you're just like really, so like, okay, well. how should next quarter's marketing budget be distributed, right? Or I need to like reduce this ad spend on Google ads, but keep TV at 10%, right? So all these different things, now you just like sort of tell them and then the model knows how to translate the human query, which could have come from a non-technical person, right? Which usually are the end users now can be translated by the model into like the right Python code that will like get the posterior and understand the model structure and how to basically run the simulator, right? To like compare different scenarios and then provide like with the right plot, the correct answer. So that, yeah, it's like just a lot more powerful. And then of course, there's like a lot of questions around like, yeah, the interface and how do you even like design software now in this agentic world, right? Like, do we build? just dashboards, do we, yeah, and there's like a lot that is changing in the world of software engineering. And I think, yeah, a lot of it is getting better, but yeah, I'm definitely like very curious about like what direction it's gonna take. And it seems to just change every month where like sort of new paradigms are coming out and new development approaches. Yeah, so I don't know, how do you feel about like how the... The nature of software is changing that we're seeing and that's coming out and all this shaking out. 00:15:41.000 --> 00:17:25.000 Yeah, it's very strange to see where historically software was something that required a large team or complex software to build and create. And now the cost of execution for that software is I mean not zero, but essentially zero. The verification, of course, is Not zero, but once you've specified it and have the way of verifying it, that cost is very low. So it starts to go to what what where are we where are we focused then and what are we doing now that we don't have to invest all this energy into the actual execution and creation of software. I think that's that's the part that we're still all kind of figuring out. I don't know if we know the answer yet or if it depends on I think some to some degree are you in a smaller or larger company too where if you have software systems that already work, do you need to go build a lot of new stuff or are you focused on maintaining the existing things? I think that also weighs on how we're applying this technology versus in a newer company, you're you gotta go and build stuff, you don't have anything. So I think that gives a lot of leverage to newer companies as well. 00:17:25.000 --> 00:19:16.000 Yeah, I agree. And I mean, we do see that at PyMC Labs we're just like very eager to adopt like all these new tools for our own company as well. And in doing so, yeah, I mean, I definitely have the feeling that the whole approach to like software building, but I mean, then also, of course, like the type of software, like really the interface of the software is going and needs to change dramatically and this has been an evolution for me. And I think just an evolution in general in software engineering, where way back when we started with command line interfaces, because we didn't have any graphical interfaces. And that was cool. And then, of course, the desktop came along, and we had a mouse and wanted to click on things. And then, of course, web apps and those things were. And then I think these agentic interfaces, right? I like just a fundamentally new class, right? Before you had to like click around and someone had to like have already like thought about like all the different things you could click, right? So the workflows are like really static and like only allow what the developer has allowed you to do really. To, and that is like the software that I mean, I, we have been building to a large extent. And then I, started to build a tool called Alchemize. And that basically is a compiler that takes, for example, a stat model, and then compiles that to PyMC. Or it takes a deep net that you have in JAX and rewrites it in PyTorch. And it verifies the outputs. so compilers are nothing new. 00:19:16.000 --> 00:19:18.000 And this is using agents. 00:19:18.000 --> 00:28:46.000 Exactly. So it's not like sort of the classic like, oh, okay, well, I mean, this code, I know how to like sort of translate it. And then you like write a compiler, right? That's what people have been doing for decades. But I just basically said like, oh, okay, well, actually like the software itself, there's like no translation logic. It's just the agent in a box basically inside the software that's like taking the code and translating it. And it has skill files that tell it like, oh, okay, well actually, Like these are the caveats when you translate from like a Stan to a PyMC model, right? Like this sort of pattern might translate in this particular way because obviously they will sort of have problems with certain things that will otherwise take them like a long time to figure out. And then the thing about the software alchemize is that it then compares directly the numerical outputs from the pre-translated to the post-translated thing, right? So you know what you're getting is like exact translation. So there's like this verification step, which is cool. But what is interesting to me is like, oh, okay, well actually, a lot of like Python logic, like whatever you write your compiler in is now like just the Claude Agent SDK, right? So it's just like the agent living inside of the software, which is like command line interface. But... Then now the kind of software I wrote, right, actually is not like even like an agent in a box. It's really just skill files. And I think that's really sort of where everything is going is we, don't think like there's much use in like, I mean, we will still want like interfaces, right? But those interfaces either like are spun off from like an agentic workflow. Like for this particular step, I want to be the human, loop, right? And I want to do, something where I like sort of click on things, right? Because sometimes that's the logical thing, but then that happens basically at the right time and the agent, like I just work with the agent. And so actually just like yesterday, I felt like revolutionized a workflow using Claude code, the desktop app, which I think is like brilliant. I mean, the Codex app has the same functionality, but Just the way that they're building this is, I think, really smart. And the workflow actually like, so it's not my workflow, it's from our partner and she does like online marketing. So you like browse like certain trends on TikTok, and then you come up with ideas for your own posts, you create your own posts, and probably there's like an image with like text on top. So that needs to look appealing. So you upload that to Canva, and then you like, edit it there and then you post it on TikTok and you see how well it goes and then the next round. a lot of things in that workflow are like, you want like either like Canva or Miro or right like TikTok scraping, right? Like all of these things that are like actually really hard to automate. But Claude Code has like all these nice connectors and integrations that you can like just connect it with Canva and then it can like remotely control Canva or Miro. Right? So then the workflow is really just like, okay, well, let's just like write that entire workflow out of like, okay, well, first we're gonna scrape and like look around TikTok. Then we're gonna like find the best ideas and then we're gonna like present them in this dashboard and then we're gonna like write some copy. So that entire thing is not like a classic piece of software which you could have had, right? Which is like a web app that you probably would bring up and you have like buttons for everything and it like takes the user through. Now, you are doing the workflow in Claude Code, just following the skill files. And then all of a sudden, it's a totally different paradigm, right? Because now the software is super flexible and it's not that everything I would have had to think about ahead of time to design, right? And it can deviate to new things and unforeseen things. And if I need to just keep iterating on this one thing, can. So that flexibility is amazing, but it only is there really when you have the entire workflow really built on agents where agents are. So it's agent-driven development or agentic software development where really the main user is the agent itself, right? And then it's just orchestrated by the human. That I feel is, yeah, just like has so many benefits. in a way, the, I feel like the agent is becoming like the operating system of the next iteration basically, right? Where like everything that I'm gonna do, I mean, already like I spent 90 % of my time just like in Claude code, responding to emails and doing like other automation stuff or reviewing. contracts and like running, building software, right? Going through workflows that like are very custom to me that I have like with, with skill files, right? So that, I don't know, I just feel strongly about that. Like all the things need to be connected to the agent and, that's the central piece that you're, and then from there, everything else you have orchestrate. So it doesn't mean we need to like, throw away all the old software, we just need to centralize it there. And that will give us best of both worlds, essentially. And of course then there's the question, okay, well now that's just for me, myself, the workflow, the next place where then usually it's like, okay, well now we wanna like... build this, like make this available, right, to other people. So how do I port that workflow from myself to someone else? So I could just have a GitHub repo with like all the skills in, that's usually what we do, that then someone else sets up, but then they also need to set up their own integrations, right? Like in that example I had with Canva, like, oh, okay, well now I, in order to run this workflow with all the skills, I need to have my own integrations. And I mean, you can do that, but. What we found to work really well then is to actually like have the agent and really the Claude Code live inside the team chat. So when you like move from single to organization, you want other people to use those workflows with those same integrations. So just like the other day, we released Daimon, which is our Discord agent that joins your team chat. And then there has like all the integrations that we have. So in our case, HubSpot and LinkedIn and email and GitHub and Notion and toggle for time tracking and basically all the different things. So now rather than just like the operating system for me, this has become the operating system for the entire company, for the entire organization. And it's not just me that like is like prompting and like figuring all these amazing things and workflow hacks out by myself, but it's really me and everyone else in the team that's like, cool. Like this is sort of what Thomas did with Daimon, right? And they see because it's in the public chat, mean, internal chat, but they see like, oh, okay, cool. And then people that like before haven't really had much of a inclination to AI, they haven't really gotten around to it. Well, now it's like in where they already work, right? In Discord or Slack. So it doesn't matter which one. Now they really have access to great AI with all the integrations so they don't have it set up themselves and can interact with others. So collaborative multiplayer AI, I think is then sort of the next frontier after that, that we have now worked on for like a year. And that has been like absolutely transformational for the company in having these agentic. software design principles applied not just to my workflow, but now to like the company workflows and processes and onboarding and shepherding through contracts and like so many business processes, right? Like now just really become so easy to automate and still like be flexible enough, right? That's really the key. Like before automation was always brittle and required a of support. Now it's flexible. And of course there's like more kinks to be worked out, but nonetheless, so that I feel like is, I like just really powerful paradigm that haven't been possible before that now are like sort of emerging and yeah, really, I think changing things in a major way. 00:28:46.000 --> 00:29:49.000 Yeah, I feel like that's an important point that before automating these kinds of things, the degree of complexity involved, or you might need like five different models one for each step of the process or something like that. And it really had to be a very high value problem to make it worth solving and and And one that was tractable to this kind that it was even there was even a possibility of solving it in a way that was worth doing. So I feel like agents and AI have opened that up in a way that just traditionally was reserved for like you're trading millions of dollars or like quant finance, things like that. Now you can apply it to your What am I gonna get at the grocery store even? 00:29:49.000 --> 00:34:25.000 So the personal impact it has had, but then, yeah, the organizational impact it is having as well. And I like your framing around like, okay, well, yeah, like now we like significantly lowered the barrier of entry or the cost of developing software and making it accessible and flexible, right? So it's like a lot more open and we can automate a lot more processes now because the cost has gone down. And I mean, that is like the same effect, right? Applied to data science. So before, right, you needed like a data science organization, like a data science team to like, basically, yeah, do your media mix modeling, which is a lot of what we do, but like work across verticals. But so that is, yeah, its own team and they spent months building models, running models, deploying models, monitoring models, providing slideshows from the outputs of those models to present to stakeholders the results and give them the answers that they're looking for. And that has been just the only way to do it, really. And how that is going to change or whether, I don't know, but Now, like all these type of questions, and I mean, this is what we see with Daimon at PyMC Labs too, is now a lot more people can actually ask these types of like data questions, right? Like before, really the only people that could even answer those questions, right? I mean, everyone can ask them, but the questions who can answer them, they were inside the data science silo where they have like, all the access to all the different things, the accounts, right? And like the knowledge of like, where does that data sit and how do I retrieve it and what do do with it and where do the models sit and what do the outputs mean? So it just requires all that expert knowledge just to operate and provide answers. But with agentic data science, I think also we can make it a lot more accessible for other people to ask these type of questions. and Daimon, we found to be like an excellent tool because like, well, how, where do those like business folks, for example, or like maybe as an engineer in our case, right? Where like they have some idea for how we can increase traction on LinkedIn or improve our SEO ranking for our website, right? And then they don't need to like talk and like become a customer of the data science team. They just like in chat, team chat like ask like, okay, well, I don't know, like pull the data, the LinkedIn sales data or the web data from Google Analytics, run this analysis, suggest improvements, make a pull request, right? So with just like a single prompt, like you really can like move mountains now. And again, the key benefit is like, well, the integration is already there. So Daimon has access to all of these things. We taught it like the semantic layer basically, like where does the data sit? What does it mean? So there is like definitely work that goes into like standing a system like that up and like making sure it has not only the correct connectors, but also like knows how to operate them and answer questions in like a reliable way and have the right business context at the right time. So these are like difficult things to figure out. And then of course, just like daily sort of mundane usability questions, like does it reply only when you're like really at it or when it just like sort of sees something when things like that. But I mean, those can be solved and then you can connect them to like your models, right? So they orchestrate and spin up the right PyMC marketing model at the right time to like answer that question and rerun it once new data becomes available. and then updates the dashboard, right? Which we maybe still want to keep around just like for clicking on. And Daimon, for example, also has a feature and that is open source now, right? So people here can check it out in our PyMC Labs and there's Daimon. Yeah. 00:34:25.000 --> 00:34:28.000 Daimon is open source now. 00:34:28.000 --> 00:37:17.000 Yeah. So we just released it yesterday. So this is like super new, but yeah. So this is Daimon, D-A-I-M-O-N dot decision dot AI. And next week, so probably by the time this is out, you can go to their website and just click on, add this to my Discord, add this to my Slack. And you don't have to even host anything. just will join. Let's say there's $5 of credit on us that you can just play around with and see what it is. Because then really the magic unfolds when you're like, oh, cool. Now it's actually here. And I can see it. Yeah. Do all these cool things. And. Yeah, so that I feel is the power now is making data science much more available and deliver on that promise. Premise we had like, I don't know, 15 years ago, we're like, oh yeah, data science is going to be like this amazing thing that's like going to deliver all this business value. I don't think we have really delivered. I think for a large part we've failed, right? And I mean, not for fault of like data scientists, but really because it is so difficult, right, to figure out like the entire data infrastructure story. So to get to the point where you can answer like interesting questions is like so much work and like so much plumbing. And then, I mean, yeah, you need the entire data engineering thing more or less figured out. And then of course, data cleanness and context and then like the modeling, right? So there's like a lot of the conditions. yeah, like now when you have all that in place and you connect it with an agent that like sits at the center, which is the operating system, to orchestrate, everyone can answer, can ask these questions, get good answers. And of course, there's still like the work in validating the system and making sure that like the models is running correct. So it's not that the data scientist goes away, but the data scientist, think, will have a lot more time to actually do the work that matters and build the models and verify the models, but not sort of the Q &A for this person that has a random question, by the time you answer that, they already forgot about, right? So I think that's one of the motivations, the frustrations for data scientists is just, yeah, much more really providing answers. than doing the actual modeling, which for most of us I think is the fun part and the interesting, more theoretically rounded, satisfying work. 00:37:17.000 --> 00:37:43.000 Yeah, like both of us have worked in quant finance and my experience was often the modeling aspect was relatively small compared to the whole build a system, keep it running, make sure that the data is correct, because that's as important in an operational system as the model. 00:37:43.000 --> 00:42:02.000 A lot of work, yeah. And like I said, it's not that that work just naturally resolves into AI pixie dust, even though I would often hope that that's the case. But we do need this, right? And to figure things out and to have also verification around our agents. They're not magic either. They're getting really good, really fast. And that will continue. I think we can just also bank on that. But at the same time, you want verification. And there's many ways you can do it that give you just like data science and hallucinated answers. But there's ways of building these agentic data science systems that are robust and have skills that you verify and benchmark. so a lot of the work that we've been doing also then is around that is, how do we... I hope I have sold people on the premise of like this agentic data science revolution, but then how do you actually, well, make it work? So there's definitely like new principles that we have to develop that we are developing. And actually we're teaching in our course with Hugo Bowne-Anderson and Luca Fiaschi of the Master Agentic Data Science. So on our courses site on pymclabs.com And that's exactly where we like take you through, okay, like you maybe know some Python and some key libraries, but then how do you build agentic data science systems that provide these type of ambitious requirements and deliver on the purpose, but also like really are trustworthy and monitorable and like enterprise prise grade really, right? If you want to use this in your business, you need verifiable answers, you need telemetry, logging and all those things. there is definitely, yeah, a lot of nuance required, a lot of new science required, right? Like, I don't think anyone has really figured this out. So I mean, I'm changing my workflow every week almost, and like discovering new things as new things come out, right? So that is the other thing which I find interesting to note where, in like, for example, teaching that course, right? Like usually when you teach a course, you're like, okay, well, this is like how it's been done and how like people have figured out how to do this, right? But here's like, okay, well, we really, I mean, don't know. Like here's like what people have so far figured out that works. Here's what I have figured out for myself that works. So it's really just like at the end of the day, like principles and tricks. mean, just now, right, like there's this debate on Twitter, like, well, loops were like the thing, right? We're like, well, we got to stop prompting your AI. You're just going to set up loops and they're just going to optimize themselves to super intelligence and amazing things. And then it's like, well, actually it's graphs, right? Because like when you have these these Jeremiah Lowin-style directed agentic graphs, where now instead of, yeah, just like optimizing a thing, it's like when talking about these AI workflow automations, mean, a workflow will have different type of states, right? You can think of it like a state machine of, okay, well now we wanna verify the user ID, how do I do that? And then you sort of take the model through and still have like a lot of flexibility around this but there's like some structure but I mean what that sort of current fad where that's going to lead like I don't know I mean it's interesting but yeah I mean the honest truth is we're all just figuring this out and and I don't think anyone really knows where it's going but it's definitely going somewhere. 00:42:02.000 --> 00:42:05.000 Yeah, and and doing the show 00:42:05.000 --> 00:42:53.000 pretty fast. 00:42:05.000 --> 00:42:53.000 and talking to different people. I mean folks I know who are getting good results use very different methods as well. Like some people are all about fully remote agents. Don't steer at all, essentially write your spec and then check later kind of the anti-loop of you wanna be there to see kind of things that are happening and be able to steer things back in the right direction and everything somewhat in between. So It I think it's yeah, no one no one really knows, I don't I don't think. 00:42:53.000 --> 00:51:08.000 Yeah, and that's it's so interesting that your observation there matches that as well. And yeah, there's like definitely the spectrum of like, you need to be like just always steering the AI and like just be involved in every step. Then there's like the more YOLO people like myself. So I try to just like, like my philosophy is like at the core, I'm just like really lazy. And if I don't have to like do hardly anything, then like great. So the more automation I can accomplish and the more I can scale the outputs of my work, right? Like the better. I mean, like just the other day, I released a new package, open source, PyMC forecast. And basically what it is, so you have a PyMC time series model, right? And now you wanna test that. So there's like certain primitives of course, with like, well, you wanna test that on like how to do it in the future. or maybe you want to run a backtest, right? So it provides a lot of those primitives to like really validate your time series models, your probabilistic time series models. And that is a port, a fork. Well, it's a port of a NumPyro forecast from Juan Orduz. And he imported himself from like another package, but so that doesn't really matter, but I... I vibe coded this in like, I mean, vibe coded, guess I built this, I ported this in like two days, pretty much. And it's like a fairly complex code base. But I mean, obviously like this is easier than spinning something up from scratch, right? So like, I just had to port and like make sure that like all the, it's much easier to verify. So this is like an easier problem, but nonetheless, I mean, I also added like new features to it, but. I was able to do that in such a short amount of time because, well, one is the verification, right? So whenever you have verification, you can just then put the agent into a much bigger loop because you know at end of the day, okay, well, when it did figure it out, you know it did. And then there's the question of how do you get there? And their AI will certainly make a lot of... dumb choices, And like misunderstand or over engineer. I mean, that's like one of my pet peeves. like, it's just like, yeah, maybe oftentimes like trying to be too smart. And then like, yeah, like there's so many different approaches and everyone has their own, right? We're like, well, you need to write this like, spec-driven development, right? Where like, before you start anything, you come up with this big spec where you've thought through all the details and then it's really just a coding exercise for the agent. And you know, because then it'll build the right thing because you really map that out carefully with the agent ahead of time. But for me, that's already too much work. I don't want to spend hours developing a spec. then, right, because you discover something later when you implement it that's like, I guess I should have changed the spec. So then there's ways of like, revising and going back to the spec part and right so that then you build like, well, yeah, guess, agent engineering workflows. And there's like, there's a little tools and every day 10 new ones come out of like, well, this is really how I like get fast results with coding. And it's very tempting, I think, to like want to like spin up these systems, right? And like, so Wes McKinney was on like, so with Hugo Bowne-Anderson and also we do like a show, Show Us Your Agent Skills, where we bring on guests like, Wes McKinney and like, okay, well, how do you do it? Right? And like, we see the same thing. We're like, well, just like the amount of workflows that people have and how unique they are is like tremendous. Everyone really has their own thing. And that's cool. And like, I have built those systems and then revised those systems because like I said, right? Like my workflow, is in constant flux. Like it doesn't seem like any of us will probably like really just figure this one thing out and then that's just like gonna be my workflow for the next year, right? Like just now I talked about like Claude Code and like how the desktop app for me is just like sort of the sweet spot of like having access to multiple agents like just with one click and being able to jump between them and then like all the integrations and so that's cool. But the, so, and the insight, so the point that I wanna make with that is actually, because things are so in flux, I'm starting to wonder, I'm starting to question whether actually building really complex systems is the way to go because then you need to constantly revise, right? And you need to maintain those systems. So my results with large scale, really, engineering systems, agentic engineering workflow tools, they do work. but there are often times I feel like lot of too much overhead. So for building PyMC forecast, what I did was like so simple, but like very effective is I had Codex and Claude Code just run side by side. And for a new feature, I asked both of them to implement that same feature, right? And there's like Claude Fable, which is like, I think amazing. And then the latest 5.6 Sol. So really smart models, both of them independently implementing a feature, opening a pull request on GitHub. So I have two pull requests that do the same thing. And then I go back to Codex and Claude Code and ask them to like cross-review basically the other one's pull request and just like borrowed, pick the best one and then like port over the things from the other one that like it learned. And that like already I felt like I mean, it's super simple, right? Because I mean, I don't have to set up any system, any orchestration. I just have like the two desktop apps. I just like prompt them. And then I get like actually a lot from that because if I just have a single LLM do it, right? I get just like that one result, right? And then like I try to stay away from the code as much as possible. because I'm lazy, so I don't really want to have to think through all the implications, really review the code. But I do want to get a bit of an intuition for like, well, what are the different ways of even doing this? What are the design choices that the LLM made and the way that this problem breaks apart into individual chunks? So I don't get that from reading the pull request. I don't get that from just a single pull request, but I do get that from two. like independent pull requests because then I can see like OK, well like here it actually used like a different package manager than this one. Like a trivial example, right? Like and this is like the the results that that causes or like implementing the feature through like a different library, right? Like. Produces like code like this and then at the end like I just like so and of course like you could like have five. implementation of the same thing in parallel that you then compare against that would be even more comprehensive and probably lead to even better results. I would expect so. that's probably where that workflow would go. But even just two gives you a pretty good idea of, OK, well, yeah, those are two paths up that same mountain. And those are the implications. And then I have more understanding of what the true causal mechanisms behind this approach are can compare and advise and give feedback. 00:51:08.000 --> 00:51:16.000 This is the Bayesian method of software engineering. You're exploring the distribution of the problem space. 00:51:16.000 --> 00:51:37.000 I hadn't even thought about it like that, but you're exactly right. Yeah, it is like we're like basically drawing samples from that distribution of like possible implementations. So I'm just like drawing two potential implementations from that distribution. Probably want to eventually sample the entire posterior, right? And then just like integrate 00:51:37.000 --> 00:51:38.000 Yeah. 00:51:38.000 --> 00:53:07.000 to get like the ultimate Bayesian expected value implementation. And yeah, so I think there's a lot to be said about simplicity and development speed. then with that, and it really links back to the point I made earlier, Well, like, okay, well, do I really want a static web app that someone built for me that has all the buttons I want in the right place? Or do I really just want like some instructions for my agent to follow through that I can then like really adapt all the time and it's always flexible. I mean, that seems so much more valuable and yeah, so that's. the argument for keeping the implementation really lightweight because then I can change it, I can port it, right? Like I'm not like tied to Claude Code. I mean, I can just like port that to Codex or to run the same workflow because it's just a bunch of skill files in Daimon. So to then teach it the organization, right? So I like just first run this myself and then I port it over. So. So yeah, like those are definitely some of the patterns that I feel have been emerging. 00:53:07.000 --> 00:53:59.000 Yeah, I think that's a good point in terms of do you want to be maintaining this large software service when you can have a agent build the specific thing you need that may be a lot simpler for your specific problem? And I feel like this is a big question with open source software. Like historically big open source projects, you needed them because you needed to get a lot of people involved so you could actually build it. And then you wanted to cover what ninety-five percent of users needed in your frameworks and things like that. But if you can just spin it up specifically for what you need, do you wanna have to maintain this huge framework? 00:53:59.000 --> 00:56:14.000 Yeah, exactly. yeah, so the more lightweight, the easier it is to maintain. So that just makes a lot of sense to me. And yeah, I mean, really, I think what we're talking about is that just, I mean, the customer of the software needs to change. mean, we will still probably need a lot of the tooling, but it's not going to be any humans that like use that, right? certainly not the primary user. And this is true for like, PyMC models, right? So why would I want any human to like, know how to like, write the PyMC code to like, get that particular answer? I mean, that's really, I just need to teach the agent, right? And whatever the most efficient way is to do that, because everyone will just like, access anything, I believe, through the agent. So, and that will be an interesting world. We're like, because then like also, and I mean, of course now I'm like projecting ahead, I mean, given how fast things are developing, the question is how far ahead? maybe that's five years, maybe that's 10 years, maybe that's five years, maybe that's just three years, but we're like almost like we won't be able to like even do much of it ourselves because like it's all like only gonna be accessible through agents. And part of it also, probably due to necessity, We're like, okay, well, I mean, it's just so much more convenient to do things through my agent. But also because it's like so complex, some things, right? Like if we now like are racing towards like the, if we are on the exponential and this is the lift off scenario, right? Like the complexity of the world around us, I think will explode and And with that, will become really difficult for people to use all that stuff. It will only be agents that can actually act in the world. they just do that for us. 00:56:14.000 --> 00:57:08.000 Yeah, I think in if as a solo or small team of developers, if you're building software that previously could not have been built unless you were like AWS scale or something like that, at some point you don't have enough people or experience to really understand that software at the level that historically you would have had understanding in building it. So I just don't think if you're using these things at that scale with a different organizational structure, you're operating at a whole different level of complexity than what a team of that size can really do. You need the agents to operate it and continue working on it just at a physical level, I feel like, at some point. 00:57:08.000 --> 01:02:15.000 Yeah, exactly. And yeah, mean, those will, I think, allow us to just like get a lot more stuff done. So I think it will be net positive, but yeah, it'll definitely be quite a change for, yeah, I mean, I guess our own expected understanding even of like how the world works. And I really liked your example also with like the organizations because that's really what, where I think it's pretty obvious that actually there's like a big cost to the traditional way of doing things, right? Where you had like, well, you know, a bunch of software engineers and they maintained this like growing insane code base of like hundreds of thousands of lines of code. And the people, right, that have worked on that code base for like 10 years, are absolutely invaluable to your organization because they're the only ones that like sort of have that mental map of like, oh, well, why did you do it like this? Well, actually like this has long history and definitely don't touch this piece because then like it all crumbles. We tried this three times. So that type of, yeah, tribal knowledge is really... challenging right to just like have in the heads of people and I mean, obviously then you write docs and try and get it out of the heads of people. But as a business, I mean, yeah, the bus factor risk is real. if that person, the only person that I really understand the code base gets hit by a bus, that's a problem. But if we like always make sure that like the agents understand and like are able to operate and they will be trained by the person with a 15 year history, right? So that like the skills files that it has and the understanding of the code base and its ability to operate and modify it will be solid and like trustworthy and maintained. I mean, yeah, that's just like is a whole lot better and reduces the risk of the business a lot. I mean, with with PyMC Labs at the same thing where people that have worked for long time on client projects, building PyMC models, that same institutional knowledge stays there. And by using agents and continuously in your workflow, you already are outsourcing that into an external place where you know, okay, well, yeah, there's going to be someone able to operate it and And yeah, that's like a big plus point. And those plus points, right? I mean, they will just like make facts. The business pressures around that I think will become like so strong that like these things will just happen, right? Like we can, and we haven't really talked about any, and also I'm like, super interested in, I go, well, what are like the societal implications? Not because I don't think those are not important. I think they're very important. But I just think that there is not that much control over a lot of it, right? Like these things are being pushed by like really strong forces along like that capabilities, technology gradient. and, and yeah, so they will, they will change. our world. so far, I would argue like for the positive, I mean, I'm like so much more productive and, and just like the amount of things that are now able that haven't been able before, I think are like a massive benefit. And my expectation is that that will continue to compound. But yeah, I mean, again, they're close to say, whether we're like driving to the AI Doomsday scenario, or like, an amazing future or maybe just like the same messy reality that we have right now. And it just like will continue to sort of go on and everyone will ask questions and have concerns. But yeah, I mean, I'm optimistic. I'm excited. I'm excited to be part of building this new agentic world and bring the advanced. methods that have been arcane knowledge for a lot of us to the big stage and allow everyone to use them and become like their own data science masters even if they haven't studied that topic in university and just like yeah give more people access and superpowers that it wouldn't have before. 01:02:15.000 --> 01:03:56.000 Yeah, I similarly I've said to people I feel like agents and AI, it's kind of like the automobile or airplanes. Like we can debate that there's certainly negative externalities and positive external or positive features, but Are we gonna go back to using horse and buggy? Probably not. So how do we live with these systems and at the core of it I still don't think The ultimate problem is are we solving things that make people's lives easier or automate things like that hasn't changed. But maybe it puts more focus on that now that the part of building the actual software part is less challenging, we've gotta spend more time thinking what actually should we be doing versus before was like, we've gotta engineer this so we can kind of focus on that. I think that to me seems a big shift and Maybe for the better, it like you were saying, it empowers more people to solve problems they have versus needing an expert or having some barrier between what they want and what gets done. 01:03:56.000 --> 01:10:04.000 Exactly. I really like that framing. And I mean, just like I mentioned, I mean, I was able to like have this package PyMC forecast up in like two days. And or like I did a new research project on prior data fitted networks. And that is like really cool and also like really combines really well with PyMC. So I will. release that soon and I'm really excited about that project. think these PFNs are very powerful, especially when you build domain-specific PFNs. before, that would have been a long research project. And it would have been me reading all the important papers about it and then toying around the results and having different type of experiments. this was sped up, I don't know, 100, 1000X to the point where I wouldn't have done it, right? Because I I'm the founder of a company, so my time in just being able to really do research and build cool projects is super limited. So I either don't do it or I lean as hard on agents as I can. And since then, mean, now that I have that, right? I mean, yeah, I like... have become so much more prolific in putting out open source projects and blog posts and sharing ideas. So yeah, if we're all empowered like that, right? And then, of course, people using those things and building on top of them will just accelerate. And that's where open source, believe, is really powerful because then we just make more progress faster. We share in like those learnings and the capabilities and the ability to share and give superpowers to others. And yeah, I mean, just like, guess have more progress. I mean, so far we talked to really mostly about technology and then a little bit about like decision making and data science and how that can support. But of course, I mean, this is also core to like science, right? Just like. figuring out new drugs, solving really hard physical problems, problems underlying energy generation, Better batteries, self-driving cars, better medical procedures, right? So these things are all, I think... really ripe for disruption or for progress. mean, we don't have to disrupt, we just have to like sort of push down the gas pedal. I mean, this is apparently the point in the interview where I reveal that I'm like an accelerationist. don't know if I would get that far, but I do think like technological progress has been like the most transformative power for humanity ever. Right. And so I was in New Zealand, the other month and so we were like going on these gorgeous hikes and of course they're sort of like it's the three-day hike like the Kepler Trail track or something and you like have your backpack with your food and things like that so it's very stripped down and what was so interesting about that experience is when it's so stripped down I feel it's lot easier to notice sort of the daily affordances of I mean, really what other humans have built for us, because there was some comforts there. We didn't have to carry around a tent and build that up. There were huts, like service huts with gas. And for a couple of hours, there was solar power light. And those things alone, they really didn't boil water. for like your food and your tea after like a long day of hiking or just like having like a bunk, right, where you can like place down your sleeping bag. And like that stuff is all maintained, right, and built by someone and the fact that there's like a track in the first place. So how amazing is that? I mean, just like that for me made it somewhat easier to imagine a world without that. So and then, I mean, these are like very basic things. Now we're talking about like other things like health insurance and like, broke my ankle last year, right? And then I like had surgery with like medical professionals that have been trained for like decades and like done nothing else but this, right? And like have like all the medical knowledge about like how to do this properly and how to honestly do it, which is like immediate like direct benefit to myself. And then of course, like everything we talked about with agents where like, well, now I don't even need to like write code anymore. do many other things. And yeah, so I think it's so easy to take all that for granted now that, but yeah, so it's very instructive, I think, to take that step back and really see like, oh, like everything around me that I'm seeing here, right? This laptop is because someone else, some other human like built that and some other human like provided the conditions for that. So it was really just like, yeah. humanity helping itself to have to give people like me and you much more comfortable lives than without like the invention of fire ultimately, right? So I think that's, yeah, that's why I care deeply about innovation and technological progress. Because I think it like is the driving force and like making people's lives better. 01:10:04.000 --> 01:12:05.000 Yeah, definitely. I mean I think as as both of us people were born in the the nineteen hundreds, as they would say, these days. I think like I was born in nineteen eighty-four, I think forty years before that is like before I was born is nineteen forty three. So that the equivalent amount of time has passed as since I was born to time I mean, I guess early computers in the nineteen forties, but where we are then relative to the forty years before that is a pretty stark difference. And I was watching there was a movie that came out called Pressure about planning the D-Day landing and how they were looking trying to figure out what the weather was going to be and looking at all of these analog charts and things like that and there's a room of like forty people working on this and one thing I was thinking is okay now you don't need I mean, even pre agents, you don't need forty people to to do that job. Like the the data they had is a computer today and one person maybe. But now you can you're working on the next problem. Like what how do we understand weather or model it in a way that we couldn't do before or building the simulation systems that are beyond the complexity that could reasonably be handled. So I think a lot of times it's not things change, but we have to focus on how can we change things in the positive way too. I think that's always the responsibility we all have as technologists or people or in general. 01:12:05.000 --> 01:13:07.000 Yeah, yeah, I agree. I mean, I think we do have a big responsibility to try and affect that. And that's why I think conversations like these are so valuable, right? Just like, yeah, people talking, thinking about, OK, where are we going and how do we affect that for the better, which is like closing the circle to our conversation where we started like, well, if you want to affect outcomes, you really need a causal model. So. Yeah, if we want to make affect the outcomes of AI and move them to the good future, we need to understand what the different levers are for affecting that outcome. And yeah, mean, as technologists and people like building this, right, we have a lot of influence. yeah, I don't take that lightly. I think it's important to get Right, and that's sort of the premise from which I'm operating. 01:13:07.000 --> 01:13:19.000 Well Thomas, thanks again for joining us today. It's been great to have you and great to go into all the different sides of where we are with AI today. 01:13:19.000 --> 01:13:23.000 Yeah, thanks. I had a great time discussing and thought we covered some interesting ground. 01:13:23.000 --> 01:14:00.000 And as always, the different things Thomas mentioned, like Daimon as well as the course, the links for that will be in the show notes. So check those out and Everything we mentioned will be there. thanks again.