Subscribe
Share
Share
Embed
Dan and Niels Bantilan discuss how AI agents are changing Niels's work on two open-source projects, Flyte and Pandera. Flyte began as an MLOps orchestrator and is evolving into an AI runtime for the code, compute, and execution systems around models and agents. Pandera remains a smaller, community-focused data-validation project.
Niels finds agents most useful in mature codebases with strong structure, linters, type checks, and tests. He estimates that his coding velocity has increased at least threefold. Local models handle small fixes, while commercial tools perform better on longer tasks that require broad codebase analysis. Pull requests and code review remain central, with reviewers checking for code smells, security problems, and performance issues.
Agents now participate in Niels's debugging loop inside live Kubernetes clusters. Through Flyte's MCP server, an agent can inspect logs, identify an out-of-memory error, update the Flyte configuration, and retry the workload. In one case, an agent found an off-by-one error in tensor loading within five minutes, fixing a model that had been emitting garbage symbols. The experience also exposed a risk: Niels has started skimming the agent's report instead of reconstructing every bug himself.
At Union, internal agents have narrow responsibilities and return reviewable artifacts. Nody handles customer requests to change node-pool limits and opens pull requests for engineers to review. Doxy monitors SDK changes and proposes documentation updates. Niels applies the same pattern to PRDs, go-to-market writing, and code examples. Agents should have clear access boundaries and produce work that people can inspect.
Niels imagines Flyte letting agents assemble workflows instead of following fixed DAGs. Typed tasks define the available building blocks, while Pydantic Monty safely runs the control-flow code an agent writes. Flyte can move files between pods, route heavy work to suitable compute, and resume a 100-step pipeline at step 98 instead of starting over. Niels sees this as the foundation for an AI runtime that combines agents with training, inference, and reinforcement-learning rollouts.
Agents have also made it easier for Niels to maintain Pandera while raising a young family. He is exploring validation schemas for vectors, images, and tensor containers, with Narwhals and LanceDB as possible paths into multimodal data. The design remains open. Pandera's concise plain-text errors work well for agents, while HTML reports may better serve people. Across both projects, Niels sees a continuing human responsibility: understand enough of the system to decide whether an agent's output is worth keeping.
Chapters
⠀
Links from the show
--------------------
⠀
Guests
-------
Niels Bantilan, Chief Machine Learning Engineer, Union
⠀
Follow the podcast
-------------------
⠀
Follow Dan Gerlanc
-------------------
The podcast about agentic AI, agentic software engineering, and entrepreneurship.
Each episode is a conversation with people building with agentic AI. Join me as I follow the stories, the behind-the-scenes, and the people behind the code.
About your host, Dan Gerlanc:
Dan brings his experience as a 4x founder with 20 years of experience in ML and software to find unique insights on the impact of AI in tech, software engineering, and entrepreneurship.
Dan Gerlanc: I'm Dan Gerlanc and welcome to Agents and Engineers,
Dan Gerlanc: the podcast about agentic AI and software development.
Dan Gerlanc: Today our guest is Niels Bantilan Niels is the Chief Machine Learning
Dan Gerlanc: Engineer at Union, a core maintainer of Flyte an open source
Dan Gerlanc: AI orchestration platform.
Dan Gerlanc: The creator of Pandera, a data validation and testing tool for data frames.
Dan Gerlanc: His mission is to help ML and AI practitioners be more productive.
Dan Gerlanc: He has a master's in public health informatics,
Dan Gerlanc: and prior to that a background in developmental biology and immunology.
Dan Gerlanc: His research interests include reinforcement learning,
Dan Gerlanc: NLP, ML in creative applications,
Dan Gerlanc: and fairness, accountability, and transparency in automated system.
Dan Gerlanc: Niels thanks for joining us today.
Niels Bantilan: Hey Dan, thanks for having me.
Niels Bantilan: Excited to be here.
Dan Gerlanc: Likewise. So tell me a bit about
Dan Gerlanc: how Agentic AI has changed the different open source projects
Dan Gerlanc: you're involved in and maintain.
Dan Gerlanc: I'll let you let you pick where to start.
Niels Bantilan: Yeah.
Niels Bantilan: Yeah. I guess I'll I'll start with Flyte.
Niels Bantilan: I am as you said, the maintainer of Flyte and and Pandera.
Niels Bantilan: Two very different flavors of open source.
Niels Bantilan: Flyte, just for context, is kind of MLOps.
Niels Bantilan: It grew up in the ML ops orchestration space.
Niels Bantilan: We're kind of pivoting, we're kind of changing like how we talk about ourselves
Niels Bantilan: and talking more.
Niels Bantilan: AI runtime and what that means,
Niels Bantilan: but
Niels Bantilan: Effectively, it solves the problem of how do you build systems
Niels Bantilan: in production that have a machine learning component and with agents
Niels Bantilan: and AI increasingly what is the I guess cognitive architecture,
Niels Bantilan: if you want to call it that, around the the code that supports the code
Niels Bantilan: and the other pieces in the stack that supports the typically LLM that helps
Niels Bantilan: you build very flexible, powerful systems.
Niels Bantilan: So Flyte is that type of project similar in the problem spaces like
Niels Bantilan: Airflow or other orchestrators out there with a few,
Niels Bantilan: or I would say a couple of core assumptions that that make
Niels Bantilan: it particularly well suited for machine learning and AI workloads
Niels Bantilan: at its like primitive level.
Niels Bantilan: So that's Flyte. And then Pandera is a data frame validation library that supports
Niels Bantilan: all sorts of data frame formats now.
Niels Bantilan: I'll start with Flyte just because it's there's a very interesting kind
Niels Bantilan: of balance that you may relate to as well around open source and commercial.
Niels Bantilan: So whereas Pandera is very much a pure open source
Niels Bantilan: project that
Niels Bantilan: I have not made any moves to commercialize at this point.
Niels Bantilan: That that may change at some point in the future.
Niels Bantilan: But right now it's it's like a labor of love.
Niels Bantilan: I started it as a side project,
Niels Bantilan: kind of merged it into Union as an open source project after I joined.
Niels Bantilan: and so that has some different properties,
Niels Bantilan: but I think for Flyte
Niels Bantilan: And for both, right? But in particular,
Niels Bantilan: Flyte Agents has really, I think,
Niels Bantilan: helped me and and the Union team,
Niels Bantilan: who is like the main maintainer and shepherd of of that project.
Niels Bantilan: Flyte is also it's owned by the Linux Foundation.
Niels Bantilan: but I mean I can start at the front line,
Niels Bantilan: right? So a lot of our developers are now just using agents like to implement
Niels Bantilan: features and do bug fixes. I I would say if you already have
Niels Bantilan: a pretty well structured and opinionated project that has linters
Niels Bantilan: and type type linting, type checking,
Niels Bantilan: like a s kind of a style guide,
Niels Bantilan: so to speak.
Niels Bantilan: and fairly mature in this the structure of how modular modules are laid out,
Niels Bantilan: how the unit tests are all structured.
Niels Bantilan: Actually agents at this point in June 2026 are are really fantastic.
Niels Bantilan: I use kind of a mix. I generally use Claude or Cursor.
Niels Bantilan: I'm slowly migrating over to Pi.
Niels Bantilan: I've dabbled with OpenCode but yeah,
Niels Bantilan: I'm just trying out the whole local sovereign AI story and it's you know,
Niels Bantilan: it's it's not as good, I would say,
Niels Bantilan: off the bat, as the commercial offerings out there,
Niels Bantilan: but I think I'm slowly getting there.
Niels Bantilan: a lot of it I think just has to do with
Niels Bantilan: the prompting and like sort of the there's a little bit more work you need
Niels Bantilan: to do in terms of like the skills and the harness you have to attach
Niels Bantilan: to these open source agent harnesses.
Dan Gerlanc: Do you have a Mac with a
Dan Gerlanc: lot of RAM?
Niels Bantilan: Yeah, I have a DGX Spark actually.
Dan Gerlanc: Okay.
Niels Bantilan: So I I host I'm currently hosting Qwen 80 Billion Coder Next on it.
Niels Bantilan: And yeah, I think the the class of tasks in
Niels Bantilan: my job that the local AI does really well are mostly bug fixes
Niels Bantilan: and very small scoped changes.
Niels Bantilan: Like, hey, I need to add a flag to the CLI.
Niels Bantilan: And make sure all the changes proc propagate through all the relevant parts
Niels Bantilan: of the code base does that super well.
Niels Bantilan: if it's a longer running task that might take an hour to complete.
Niels Bantilan: I found that I can give
Niels Bantilan: a pretty sizable like part of a PRD pro product requirements
Niels Bantilan: doc over to like Claude and it will kind of
Niels Bantilan: For the most part, hit it out of the park.
Niels Bantilan: And as I yeah, as I mentioned earlier to you before we started recording,
Niels Bantilan: it's like a lot of reading. So it's sort of like you have a you have
Niels Bantilan: a coworker and they can spit out thousands,
Niels Bantilan: hundreds of thousands of lines of code.
Niels Bantilan: So you just you wanna be careful because you're not you don't want to review like
Niels Bantilan: a PR that's massive, just like with irregular,
Niels Bantilan: like pre-AI agent workflows.
Niels Bantilan: So the
Dan Gerlanc: Is that something
Dan Gerlanc: you have fairly strict requirements around that like things still go through
Dan Gerlanc: a PR workflow into very well-defined changes?
Niels Bantilan: For the most part in the project,
Niels Bantilan: yes. our team is has an engineering background.
Niels Bantilan: Most of us at at Union AI are backend engineers.
Niels Bantilan: I am a MLE data scientist by training.
Niels Bantilan: but I have I would say I do have more of an engineering mindset than like
Niels Bantilan: a researcher mindset. I do get a little,
Niels Bantilan: more particular about sort of like style and and formatting and
Niels Bantilan: and things like that. I mean we have linters that for that now to like abstract that
Niels Bantilan: away.
Niels Bantilan: So yeah, I mean we we still do PRs,
Niels Bantilan: we still do code review. most of it I would suspect.
Niels Bantilan: I I haven't like asked this of my team,
Niels Bantilan: but I suspect most of us are have a fairly high level of trust of these systems now,
Niels Bantilan: and we're mainly looking for code smells,
Niels Bantilan: security-related things,
Niels Bantilan: performance-related things.
Niels Bantilan: that just immediately kinda stand out to us.
Niels Bantilan: but our velocity is like I would say much faster.
Dan Gerlanc: What would you say compared to before if you had to estimate increase in velocity?
Niels Bantilan: I mean I I wouldn't say it's 10x I would say it's like
Niels Bantilan: At least three.
Niels Bantilan: I mean as I said, I it's it's been a while for me that I've
Niels Bantilan: got into the weeds and gone into like the into the traditional debug
Niels Bantilan: like REPL loop of running the debugger on like
Niels Bantilan: VS Code or Cursor. And like my debug loop is literally the agent making those
Niels Bantilan: changes. I'll like add I'll add tests.
Niels Bantilan: via prompting. It was like, hey,
Niels Bantilan: you didn't catch this, you know,
Niels Bantilan: like, hey, test this part more and you know,
Niels Bantilan: maybe do I mean we don't use hypothesis like
Niels Bantilan: the the property-based testing thing quite yet.
Niels Bantilan: but you know just add more test cases to cover this like part of the execution path.
Niels Bantilan: and then yeah for the most part if if my prompt
Niels Bantilan: is is small enough in scope it'll like do that very well as well.
Niels Bantilan: when I debug stuff also, so Union and Flyte,
Niels Bantilan: it's like a cluster, it's a Kubernetes cluster on some cloud somewhere.
Niels Bantilan: I literally point we have a Flyte MCP now,
Niels Bantilan: so I just point
Niels Bantilan: My agent to the config file that that will authenticate through my browser
Niels Bantilan: and then open up a session and it'll just like run workloads and
Niels Bantilan: the MCP exposes logs and errors that happen on the Kubernetes cluster.
Niels Bantilan: So my agent basically has access to Kubernetes arbitrary computes within limits
Niels Bantilan: to test test out what I'm building.
Dan Gerlanc: And do you think this is a good use case for MCP?
Dan Gerlanc: I know there's some talk of people say,
Dan Gerlanc: like, skills or MCP, which I mean at the end of the day you can do
Dan Gerlanc: the same things with either or CLIs.
Dan Gerlanc: How do you feel
Niels Bantilan: Yeah.
Dan Gerlanc: about that implement do you feel like in this case?
Dan Gerlanc: There was a strong reason to go with MCP because you're then hooking into
Dan Gerlanc: a Kubernetes cluster.
Niels Bantilan: Yeah, I would say kind of the latter.
Niels Bantilan: So I could have created
Niels Bantilan: a skill that like the answer is auth,
Niels Bantilan: basically. Like I could have created a a s a skill that describes
Niels Bantilan: the kubectl commands to like get the logs from the pods that are erring out.
Niels Bantilan: But Flyte also is like an abstraction over Kubernetes,
Niels Bantilan: so I
Niels Bantilan: Like I don't want to deal with Kubernetes as much as possible.
Niels Bantilan: that's just you know.
Dan Gerlanc: Most people don't.
Niels Bantilan: An
Niels Bantilan: an explicit decision that yes,
Niels Bantilan: that I've made. I'm not a platform person.
Niels Bantilan: I don't ever really want to interact with it as much as possible.
Niels Bantilan: so
Niels Bantilan: The Flyte MCP that we have just I I provide my like Flyte API key,
Niels Bantilan: and that just you know it has access to whatever parts of the Kubernetes
Niels Bantilan: API surface we surface to users through the Flyte API.
Niels Bantilan: so it's a it's a much smaller constrained surface.
Niels Bantilan: It gives me all the logs and errors and stuff.
Niels Bantilan: That's mostly what I need for a proper like agent debugging loop.
Niels Bantilan: and it's nice too because the agent can figure,
Niels Bantilan: hey,
Niels Bantilan: this was an out of memory error,
Niels Bantilan: so let me just like slightly change my Flyte configuration to ask ask
Niels Bantilan: for a little bit more memory. it can even like potentially profile
Niels Bantilan: the data set that I'm working with and analyze it a little bit
Niels Bantilan: and kind of estimate like okay now I need to provision maybe four more gigs.
Niels Bantilan: so yeah it's I I don't know I feel very spoiled that like I have
Niels Bantilan: Access to agents in general for development,
Niels Bantilan: but then as an ML person, ML
Niels Bantilan: is like the the core diff difference between a software engineer and
Niels Bantilan: an ML engineer is like generally compute.
Niels Bantilan: Like a software engineer, you can like unit test stuff,
Niels Bantilan: right? You can like mock out things,
Niels Bantilan: which you could do to a certain extent in machine learning,
Niels Bantilan: but
Niels Bantilan: For anyone out there who's like trained a model,
Niels Bantilan: like the magic only really happens at a certain scale.
Niels Bantilan: You have to wait for a while for like the loss to go down enough,
Niels Bantilan: for things to start kind of snapping into place.
Dan Gerlanc: How does it feel to be seeing the whole world
Dan Gerlanc: of AI engineering as a traditional ML engineer
Niels Bantilan: Ha ha
Dan Gerlanc: developer?
Niels Bantilan: it's it's funny 'cause you can say that I've been chasing titles all of my career,
Niels Bantilan: right? 'Cause
Dan Gerlanc: Mm-hmm.
Niels Bantilan: I've I started off as a data scientist.
Niels Bantilan: Because I I mean I enjoyed the visualization part,
Niels Bantilan: the understanding of the data part,
Niels Bantilan: the modeling of it. at the outset when I sort of like came in from grad school
Niels Bantilan: and and started learning about the space.
Niels Bantilan: soon after though,
Niels Bantilan: when I started productionizing stuff at my first startup job,
Niels Bantilan: That's sort of that's when I learned about like build systems.
Niels Bantilan: I I wasn't I'm not like a CS or classically trained software engineer.
Niels Bantilan: I guess software engineers generally like you learn on the job actually anyway.
Niels Bantilan: So all the stuff around CI testing,
Niels Bantilan: unit testing, integration testing,
Niels Bantilan: all that good stuff. That that part actually was fun for me also.
Niels Bantilan: And so I my sh my
Niels Bantilan: Mindset kind of shifted to okay,
Niels Bantilan: it's cool to have like stuff in notebooks and like analysis that provide insights
Niels Bantilan: and may help you make decisions,
Niels Bantilan: but it's like when you start serving models in production,
Niels Bantilan: there's like a whole set of other concerns that you have to learn about
Niels Bantilan: on top of like the all the other skills that I think it's is good to pick
Niels Bantilan: up as a data scientist. and so now with AI engineering,
Niels Bantilan: my my
Niels Bantilan: kind of reasoning by analogy is inference has taken up the r this
Niels Bantilan: the oxygen in the room. Like ML was all about training.
Niels Bantilan: AI engineering is mostly about inference time things you can do with the model,
Niels Bantilan: which before was like all I do is predict a thing,
Niels Bantilan: right? And it's just like a scalar,
Niels Bantilan: like for the the regression or a classification thing.
Niels Bantilan: Now with LLMs and like the multimodal stuff,
Niels Bantilan: it's whole documents and whole videos and images,
Niels Bantilan: right? So
Niels Bantilan: the the space of applications has just exploded on the inference side.
Niels Bantilan: And you're kind of now seeing a little bit of a return with RL
Niels Bantilan: and like a j it's an overloaded term,
Niels Bantilan: right? You have RL agents and then you have AI agents and there's an overlap,
Niels Bantilan: obviously. and so a little bit more emphasis coming back
Niels Bantilan: to RL and like training.
Niels Bantilan: And it's a little bit it's a hybrid thing because with RL there are rollouts,
Niels Bantilan: so you need like an inference server somewhere generating rollouts
Niels Bantilan: for whatever your latest checkpoint is or your latest policy is.
Niels Bantilan: So we're starting to, you know,
Niels Bantilan: we're we're because we're a platform company,
Niels Bantilan: Union AI is, we
Niels Bantilan: We have to serve our existing customer base and as trends shift and
Niels Bantilan: as people's use cases change, we turn our attention to those things.
Niels Bantilan: And so RL is something I have the the RL textbook here,
Niels Bantilan: you know, so I've kind of like picked it back up
Dan Gerlanc: Yeah, nice.
Niels Bantilan: and just the the Sutton and Barto Barto one.
Niels Bantilan: so just refreshing my basics it's been a while.
Dan Gerlanc: Is that a feature you're looking to support in Flyte to be able
Dan Gerlanc: to do or Union to be able to do reinforcement learning in the platform?
Niels Bantilan: Yeah, we're exploring this. I mean,
Niels Bantilan: it's because Union is general enough,
Niels Bantilan: you can like build everything yourself from scratch.
Niels Bantilan: But you know, the a big part of my job is just like ergonomics
Niels Bantilan: and developer experience. and like even though humans
Niels Bantilan: maybe take less and less of the coding time,
Niels Bantilan: agents still need, I think, good APIs,
Niels Bantilan: you know, for them to understand stuff.
Niels Bantilan: in scare quotes and also humans for for reading the APIs and
Niels Bantilan: and sometimes like fixing the code manually so I mean
Niels Bantilan: at the platform level there's there's many things to do.
Niels Bantilan: the this textbook is more algorithmic so this is more just like what
Niels Bantilan: are the core pieces you need, right?
Niels Bantilan: You need an environment which could be CPU-bound maybe the generate frames
Niels Bantilan: in the whatever world you're in.
Niels Bantilan: could be GPU bound also. Your model is GPU bound,
Niels Bantilan: your your your like the training,
Niels Bantilan: the model that the is updating weights is GPU bound,
Niels Bantilan: the inference servers be GPU bound.
Niels Bantilan: So Flyte allows you to kind of like orchestrate all these pieces somewhat seamless
Niels Bantilan: seamlessly so that you can feel like you're just programming locally,
Niels Bantilan: but you're actually
Niels Bantilan: Like when you do a for loop over your environment,
Niels Bantilan: you're actually hitting an a a separate pod from the one that's
Niels Bantilan: in your training loop. And the data is just being transferred super quickly through
Niels Bantilan: like a Rust powered object store so that you know it's
Niels Bantilan: the data transfer is has a little overhead,
Niels Bantilan: but you know, it's like within some like tolerance that's acceptable.
Niels Bantilan: so they're they're like those infrastructure pieces that we're we're starting
Niels Bantilan: to look at.
Dan Gerlanc: Have you had to do a lot of custom Kubernetes programming to handle some
Dan Gerlanc: of the data movement constraints and things when you start getting into
Dan Gerlanc: GPU programming and whether it stays between nodes and pods and things like that
Dan Gerlanc: on Kubernetes?
Niels Bantilan: Yeah, so I guess the the broader point I'll make here
Niels Bantilan: is that when you're building a platform that serves not just
Niels Bantilan: one company's requirements but like a lot of people's requirements,
Niels Bantilan: it's hard to really optimize for everything,
Niels Bantilan: especially if it's like open source.
Niels Bantilan: If it's a closed source platform.
Niels Bantilan: Everything is hidden, right? So it's like the the thing you can really focus
Niels Bantilan: on DevEx and just making the the the user-facing SDK super nice
Niels Bantilan: and making the experience magical.
Niels Bantilan: And so if if your MLE, ML researcher,
Niels Bantilan: AI and is your main audience, that's like your main focus.
Niels Bantilan: Our challenge is we have to serve that end user,
Niels Bantilan: but also a platform engineer who has to maintain this on a cloud somewhere.
Niels Bantilan: so we have to make very careful decisions on what technology choices we make.
Niels Bantilan: All that's to say on the data transfer and like all that like optimization.
Niels Bantilan: level stuff. We haven't invested a ton on sort of like the GPU stack.
Niels Bantilan: We we we often rely on open source technology for that.
Niels Bantilan: I mean the Rust ecosystem has been amazing
Niels Bantilan: for us in a general sense because it's just way more performant than Python.
Niels Bantilan: our team loves like programming in it
Niels Bantilan: I don't know the latest on if agents are very good at it.
Niels Bantilan: I think my my vibe my vibe sense is that they're like okay,
Niels Bantilan: but not as good as Python or JavaScript.
Niels Bantilan: but in any case, the we've
Dan Gerlanc: Yeah, anecdotally
Dan Gerlanc: I've heard that they don't always use the type system to its full advantage.
Niels Bantilan: Mm, okay, well that I wonder if that mirrors the distribution of like
Niels Bantilan: the average Rust project or something.
Niels Bantilan: I don't know.
Dan Gerlanc: Yeah, I mean I created a small Rust project when I was first I tried with
Dan Gerlanc: a few different languages, Go and Rust,
Dan Gerlanc: to see how it would fare and I mean personally as someone who has spent
Dan Gerlanc: a lot of their career writing Python,
Dan Gerlanc: I find Rust easier to understand than Go code,
Dan Gerlanc: so it was somewhat more natural to me to go through the Rust code,
Dan Gerlanc: but
Niels Bantilan: Yeah.
Dan Gerlanc: it wasn't also the most complex project.
Dan Gerlanc: So
Niels Bantilan: I see. but yeah, to your original question just around like data transfer
Niels Bantilan: and and GPU optimization, a couple of things we do there around
Niels Bantilan: yeah, using Rust powered libraries for general data transfer.
Niels Bantilan: So in Flyte, like you have tasks and you have composed multiple tasks
Niels Bantilan: to for a larger application. Each task runs on its own pod and its own container.
Niels Bantilan: So you need a way to like take the output of that one pod and save it
Niels Bantilan: to blob store and then load it up into the next pod to do the next job.
Niels Bantilan: and so that with that we used RustFS
Niels Bantilan: To quickly transfer the data. I think our bench internal benchmarks,
Niels Bantilan: I'm not sure if we publish them,
Niels Bantilan: but it's like I think it's the 500 gigs and four seconds in terms of download time,
Niels Bantilan: which is which is pretty good,
Niels Bantilan: I would say.
Dan Gerlanc: Yeah.
Niels Bantilan: And then for GPU loading, we
Dan Gerlanc: Is that writing to blobs
Dan Gerlanc: to something like S3 or is it writing well wow?
Niels Bantilan: Yeah. Yeah.
Niels Bantilan: S3, it's any S3 compatible
Niels Bantilan: object store. I forget the upload times.
Niels Bantilan: I think they're a little yeah,
Niels Bantilan: I forget the the the exact numbers there.
Niels Bantilan: But in terms of GPUs, we invested a little bit and it's using the same kind
Niels Bantilan: of RustFS backend, but we have a system that will automatically load weights from
Niels Bantilan: S3 into the GPU without having to download all the weights to disk first.
Niels Bantilan: And there's a
Dan Gerlanc: So that will use like the direct
Dan Gerlanc: to GPU memory way of doing it.
Niels Bantilan: Yeah, yeah,
Dan Gerlanc: Yep.
Niels Bantilan: exactly. one fun anecdote around there with agents was that there was
Niels Bantilan: an off by one error in the like so you you save a bunch of like tensor chunks
Niels Bantilan: to object store, right? And in object store also you keep like a metadata file,
Niels Bantilan: like you think of a JSON file that just is sort of an index of you know which
Niels Bantilan: Tensor chunk maps onto what part of the the model architecture when
Niels Bantilan: you when you load it up into memory.
Niels Bantilan: there's like this nasty off by one error that I just like could not find.
Niels Bantilan: And this was pre me being agent-pilled.
Niels Bantilan: So I was just like still sifting through the code and like trying to debug it and
Niels Bantilan: The symptom was basically the LLM was just outputting like garbage,
Niels Bantilan: garbage, like symbols and random stuff,
Niels Bantilan: right? so there's clearly something wrong.
Dan Gerlanc: Just a hard thing to debug,
Dan Gerlanc: right? It's like what's what's going on here?
Dan Gerlanc: Yeah.
Niels Bantilan: Yeah.
Niels Bantilan: so then I I think at the time I forget what I was using,
Niels Bantilan: but I just gave it the context,
Niels Bantilan: I gave it as much of the logs as I as I had access to,
Niels Bantilan: and then, you know, within five minutes it was like,
Niels Bantilan: yeah, here's here's the like the minus one you forgot to add to this like
Niels Bantilan: one part of the code. so yeah,
Niels Bantilan: that I think that was the beginning of my agent pilling story or journey.
Dan Gerlanc: Yeah, I feel like there's things where something like that,
Dan Gerlanc: right? It could be a bug where you say,
Dan Gerlanc: Okay, this might take me an hour or five days or two weeks to figure out.
Niels Bantilan: Yeah.
Dan Gerlanc: Yeah, the agents that they can move so quickly through that kind of stuff.
Dan Gerlanc: It's pretty amazing
Niels Bantilan: Yeah,
Dan Gerlanc: to see.
Niels Bantilan: I think that's the thing that worries me is I've I've stopped I've I've started just
Niels Bantilan: like skimming what the agent's findings are.
Niels Bantilan: Like 'cause it'll output a report.
Niels Bantilan: It's like, this this is all what happened,
Niels Bantilan: right?
Niels Bantilan: And yeah, I've I've started getting to the habit of just like not really trying
Niels Bantilan: to fully understand what went wrong.
Niels Bantilan: And that I feel like that that worries me 'cause I think I've already lost
Niels Bantilan: the the hard earned developer skill of just like finding a needle in the haystack.
Niels Bantilan: and now it's like I'm already slipping in terms
Niels Bantilan: of my understanding of like what actually went wrong.
Niels Bantilan: So yeah, it's I don't know.
Niels Bantilan: It's it's a push and pull. I I don't know if it's yeah,
Niels Bantilan: I feel bad about it, but at the same time it's like I have like so many other things
Niels Bantilan: to do. context switching, 'cause sort of like wear multiple hats
Niels Bantilan: at at Union that just like I'm just incentivized to move on.
Dan Gerlanc: How do we fight back against
Niels Bantilan: Ha ha
Dan Gerlanc: that or I mean, should we, or what is the part that
Dan Gerlanc: I guess yeah, what is there an answer to that?
Dan Gerlanc: What or do we just delegate this more
Dan Gerlanc: and more control to the agents?
Niels Bantilan: I I don't know. I feel like I am if my only job was
Niels Bantilan: to be an engineer, I feel like I would still want
Niels Bantilan: to deeply understand the issue even even if if I'm not the one to find
Niels Bantilan: or even fix fix the bug. but yeah.
Niels Bantilan: It my role is just so much context switching that it it
Niels Bantilan: Like I feel like more of a technical PM who had a coding background.
Niels Bantilan: who still does it for fun. And I I still like sometimes roll up my sleeve
Niels Bantilan: and like write from scratch. for like fun stuff.
Niels Bantilan: But yeah,
Niels Bantilan: it's it's getting harder and harder and
Niels Bantilan: I I would just like for anyone listening,
Niels Bantilan: I would just say, you know, hold on to that as much as as you can,
Niels Bantilan: 'cause you still need to understand stuff.
Niels Bantilan: Like I think that's the last bastion of of what we
Niels Bantilan: can do as humans.
Dan Gerlanc: Yeah, I think in personally I know on systems
Dan Gerlanc: or code where I'm more familiar with the area.
Dan Gerlanc: So I'll look at what it's doing sometimes and be able to say,
Dan Gerlanc: hey, from a structural or understanding point,
Dan Gerlanc: here is a better way to to do this.
Dan Gerlanc: Because I'd understand it better if you had this abstraction or not,
Dan Gerlanc: which
Dan Gerlanc: I feel like is an area that's still LLM's,
Dan Gerlanc: AI agents aren't they're not lazy,
Dan Gerlanc: so they they don't care as much about creating
Niels Bantilan: Yeah.
Dan Gerlanc: reusable abstraction.
Niels Bantilan: Yeah. yeah, I mean when I do I mean I I I do still
Niels Bantilan: do a lot of code review and reading of what it's generating,
Niels Bantilan: right? So
Niels Bantilan: it it does all sorts of dumb stuff enough so that I'm I still don't like hundred
Niels Bantilan: percent trust it. like an example is it just I
Niels Bantilan: I had a variable called like memory key and like the original script that
Niels Bantilan: I had called it memory key underscore something else.
Niels Bantilan: And it kept the it kept that indirection.
Niels Bantilan: So it just like reassigned the constant
Niels Bantilan: To memory key or like with the old value just because it it was like too lazy
Niels Bantilan: to refactor the places in the code base where it it had the old variable name
Niels Bantilan: or the old constant name. and so you just had memory
Niels Bantilan: key underscore something equals memory key,
Niels Bantilan: and and it's just like why why'd you do that?
Niels Bantilan: but you know,
Niels Bantilan: the the code worked, so as far as it was concerned,
Niels Bantilan: it it was fine.
Dan Gerlanc: Have you within Flyte and Union
Dan Gerlanc: set up like agentic reviews or checks for security things like that,
Dan Gerlanc: in CI or in other automated parts of the system?
Niels Bantilan: we have in some of our repos we do have GitHub Copilot enabled.
Niels Bantilan: It catches some good stuff. we haven't set
Niels Bantilan: up bespoke things or integrated with other third-party services.
Niels Bantilan: and that's I think mostly like we just haven't felt the
Niels Bantilan: the strong pull to do so. our internal agentic systems are
Niels Bantilan: Are a lot around like internal operations.
Niels Bantilan: we have one that's like
Niels Bantilan: the the inspiration of for the mascot we have internally is like
Niels Bantilan: the Butter robot Rick and Morty.
Niels Bantilan: So it all it does is change node pool configurations.
Niels Bantilan: So it only has access to one repo.
Niels Bantilan: It can only
Niels Bantilan: read and write like a specific set of directories in there like YAML files
Niels Bantilan: of like customer node pool configurations.
Niels Bantilan: So customer asks for hey I need to bump my limits for this instance type
Niels Bantilan: and then so Nody is its name and so it it will make a PR that
Niels Bantilan: we have to review it. I think
Niels Bantilan: As of today it's been online for like half a year and it's like closed couple like
Niels Bantilan: hundred plus PRs for these node pool configurations and you know they're they're
Niels Bantilan: customer facing, so it's it's critical and we do code reviews for that
Niels Bantilan: and make sure everything passes our like internal tests before merging it.
Niels Bantilan: it'll get things wrong. Our tests will usually catch it and you know we'll
Niels Bantilan: fix it and
Niels Bantilan: That kind of like makes it into the context,
Niels Bantilan: its context, because it has access to GitHub and,
Niels Bantilan: you know, the Git repo itself.
Niels Bantilan: So it can kind of like see the cases where the PRs were were edited by us.
Dan Gerlanc: And how do users interact with it?
Dan Gerlanc: Is directly just whether as a skills around it or just opening
Dan Gerlanc: a starting a session in that repo?
Niels Bantilan: for for customers they just they ping us on Slack.
Niels Bantilan: and so we can activate it with like a special word.
Niels Bantilan: we also have a form.
Dan Gerlanc: So you can activate
Dan Gerlanc: it through s you have it tied into Slack.
Niels Bantilan: Yeah,
Niels Bantilan: yeah, yeah. so we just reply to the thread,
Niels Bantilan: the customer thread where they made the request and we just at Nody
Niels Bantilan: and then it'll kinda wake up and do its its thing.
Niels Bantilan: we also have like an official node change form that our customers have access to.
Niels Bantilan: so that form will directly kind of trigger
Niels Bantilan: a webhook on Flyte. So the Flyte agent then wakes up and and does its thing.
Niels Bantilan: we have another one.
Niels Bantilan: We our naming convention, I I guess I I am to blame or take credit for it.
Niels Bantilan: so we have another one called Doxy,
Niels Bantilan: which syncs up our docs with code changes from various repos.
Niels Bantilan: so PR comes in from our SDK,
Niels Bantilan: and then Doxy will go and see is this a meaningful change that we need to document?
Niels Bantilan: and if so, it will
Niels Bantilan: Write the page and then make another PR to the docs repo.
Dan Gerlanc: So for these in in these kinds of internal workflows,
Dan Gerlanc: that's is that where that in day to day software engineering you think it's been
Dan Gerlanc: most impactful?
Niels Bantilan: Yeah. I mean a a ton of other areas.
Niels Bantilan: It's really just like a matter of time investment to the initial like hump
Niels Bantilan: of setting a thing up, and a little creativity of like,
Niels Bantilan: okay, what what kinds of problems map onto agent capabilities?
Niels Bantilan: it's a it's a big space, right?
Niels Bantilan: So I have or
Niels Bantilan: We have internally like a PRDs repo where we work on our PRDs.
Niels Bantilan: We have a go-to-market repo for like the technical side of the go-to-market team
Niels Bantilan: to take in all the context, write blogs,
Niels Bantilan: write code examples that support the blogs.
Niels Bantilan: we I ha I have like a
Niels Bantilan: local skill that just like has a laundry list of all of my previous writings.
Niels Bantilan: So I just say, you know, hey, like,
Niels Bantilan: assume my tone and voice. And so it it it gets gets it part of the way,
Niels Bantilan: and then I go and edit on Notion.
Niels Bantilan: and so there are like parts of it that
Niels Bantilan: it I can outsource pretty effectively to get
Niels Bantilan: like writer's block out of the way and all that stuff.
Niels Bantilan: I mean this is not literature,
Niels Bantilan: right? So it's not like I'm
Niels Bantilan: I do have fun in the kind of creative writing process,
Niels Bantilan: but again,
Niels Bantilan: it's sort of like a race against time.
Niels Bantilan: So it's like given the choice between facing a blank page and just like writing
Niels Bantilan: the outline and going through my whole writing process,
Niels Bantilan: it's like easier to get the initial thing and I'll make substantive changes
Niels Bantilan: actually. I've there are some times where I've just like
Niels Bantilan: effectively rewritten large chunks of it.
Niels Bantilan: But the core the core flow and the story and like narrative it gets right.
Niels Bantilan: It's just like wordsmithing and like the taste part of it,
Niels Bantilan: I would say, that I I still put my my hands,
Niels Bantilan: my fingers in the pie.
Niels Bantilan: But it's it's pretty widespread.
Niels Bantilan: Like I would say, at least for my role,
Niels Bantilan: it's like it goes from coding to
Niels Bantilan: like customer customer solutions,
Niels Bantilan: customer success, and and marketing.
Dan Gerlanc: And how often do you find you're reaching out to use Flyte
Dan Gerlanc: to actually orchestrate these different agentic workflows?
Dan Gerlanc: Because that's I feel like a big challenge folks have.
Dan Gerlanc: How do you put
Niels Bantilan: Yeah.
Dan Gerlanc: this all together and track what's happening and
Dan Gerlanc: put the compute behind it beyond just a kind of prompt and response?
Niels Bantilan: Yeah, so Nody is a Flyte powered agent and I think that
Niels Bantilan: was that was like our first that that was born out of a hackathon actually
Niels Bantilan: and then it we just kind of productionized it and it's sticking around.
Niels Bantilan: we're in the process of making the Doxy agent
Niels Bantilan: I guess what we'll call a background agent or a long long horizon agent that kind
Niels Bantilan: of just wakes up when it's needed.
Niels Bantilan: I I would say our flow so far has been someone has a problem
Niels Bantilan: and they will start off with Claude or whatever local agent harness they they're
Niels Bantilan: working with. And I have to hand it to the those projects,
Niels Bantilan: right? It's very easy to integrate all the sting all the things with it.
Niels Bantilan: So you have like Notion, Linear,
Niels Bantilan: Slack, GitHub, all the stuff, and it's like a point and click
Niels Bantilan: kind of OAuth experience to connect your like local system with it.
Niels Bantilan: And so you can get really s started really quickly
Niels Bantilan: and prove pro prove the concept or like prove the value of the agent.
Niels Bantilan: Like somewhat this is like the manual version of of agent agentic development,
Niels Bantilan: I guess. You start local, make sure it works,
Niels Bantilan: you get some value out of it.
Niels Bantilan: And
Niels Bantilan: We don't have yet like a good heuristic of like,
Niels Bantilan: okay, when do we graduate this,
Niels Bantilan: or should we even port this over to Flyte?
Niels Bantilan: generally the shape of things that we productionize
Niels Bantilan: in Flyte are roughly speaking,
Niels Bantilan: like what activates the agent.
Niels Bantilan: If I really do need
Niels Bantilan: like a human opening up a terminal and like giving the thing context
Niels Bantilan: for a particular task. I think that's it's a it's a good time to just keep
Niels Bantilan: it local at that point. When you have things like
Niels Bantilan: GitHub events, if if you have like external programmatic events that should wake
Niels Bantilan: it up, that's generally good signal to put it into Flyte.
Niels Bantilan: So if like a Slack event
Niels Bantilan: wakes it up and based on that event you can just like
Niels Bantilan: gather all the context needed to complete the task.
Niels Bantilan: or it's a GitHub PR, and same thing.
Niels Bantilan: Then it's it's easier to set up,
Niels Bantilan: or I think conceptually it it should be going to Flyte.
Niels Bantilan: I think the challenge is the whole integration piece.
Niels Bantilan: And so I mean that's kind of signal for me that Flyte should have a story
Niels Bantilan: for that to make it like really nice to
Niels Bantilan: either use a third party service or build it internally where you know,
Niels Bantilan: it's a whole thing. Like you need to make a Slackbot or you need to
Niels Bantilan: get make a GitHub app and you know it's it's not too bad,
Niels Bantilan: but it's still enough friction for it to for me to just defer to my terminal
Niels Bantilan: and be like, okay, well let me just do this on my terminal because it'll
Niels Bantilan: be kind of quicker on the sh in the short term.
Dan Gerlanc: Right versus provisioning s system accounts with API keys for every service there.
Niels Bantilan: Yeah.
Niels Bantilan: So I I see the value of all sorts of like MCP services and and other types
Niels Bantilan: of services that support the the agent harness.
Niels Bantilan: it's it's like, you know, it's for convenience.
Niels Bantilan: It's like what all of us in tech kind of know well like the trade offs of of that.
Dan Gerlanc: Do you think this will be a direction Flyte will be focused
Dan Gerlanc: on and supporting these kinds of workflows versus kind
Dan Gerlanc: of the more tradition, I guess more I always think of like more traditional data
Dan Gerlanc: engineering, ML
Niels Bantilan: Yeah.
Dan Gerlanc: engineering workflows, which can have very different shape and time frames
Dan Gerlanc: or much bigger compute needs and like some agentic.
Dan Gerlanc: workflows, it's the compute is all going to the to the model versus what you're
Dan Gerlanc: doing locally is not a lot.
Niels Bantilan: Yeah, I think we're I think we're getting to the place with agent agentic
Niels Bantilan: applications where we have a pretty good profile of like I would say,
Niels Bantilan: you know, just hand wavy like eighty percent of the use cases is like
Niels Bantilan: Most of the compute is outsourced to an LLM provider.
Niels Bantilan: If you're lucky enough to self-host,
Niels Bantilan: like congratulations. You can like host an open weights model that's good enough
Niels Bantilan: to serve your needs. you know,
Niels Bantilan: here GLM 5.2 is great and I've played around with it a little bit on
Niels Bantilan: one of our instances. Very slow,
Niels Bantilan: just because we don't have a very big one.
Niels Bantilan: And the the compute that the agent itself needs a separate from the
Niels Bantilan: LLM is like very, very light. So I think that's like I would
Niels Bantilan: say maybe eighty percent of the the mass of the applications right now.
Niels Bantilan: where I think Flyte is differentiated and
Niels Bantilan: We've see started to see some rumblings of people at least expressing demand
Niels Bantilan: for this. Is the agent becomes the workflow orchestrator for your traditional
Niels Bantilan: ETL pipeline or your traditional ML pipeline,
Niels Bantilan: your your hyperparameter optimization workflow,
Niels Bantilan: right? It's no longer hard-coded.
Niels Bantilan: and you give the agent a bunch of tools and the tools are
Niels Bantilan: the point solutions in or like the the nodes in your traditional DAG.
Niels Bantilan: this time the agent can like compose them arbitrarily.
Niels Bantilan: the benefit of Flyte is it it's type aware and it uses types
Niels Bantilan: in Python to understand like how to connect the pieces.
Niels Bantilan: And so the agent is given all of that information to compose the tasks.
Niels Bantilan: We have a thing called code mode.
Niels Bantilan: We use actually Pydantic Monty,
Niels Bantilan: which is an awesome project. folks haven't seen it,
Niels Bantilan: it's really nice. But it's like a Rust-based
Niels Bantilan: sandbox, which I know is an overloaded term,
Niels Bantilan: but it it's a way for you to run Python code.
Niels Bantilan: It's like a Python interpreter in Rust for like a strict subset of Python
Niels Bantilan: that prevents you from doing network calls or doing any IO
Dan Gerlanc: So the agents can write Python code and you can
Niels Bantilan: Yep. Yep.
Dan Gerlanc: run it in that sandbox in an easy to stand up environment.
Niels Bantilan: And we've Yeah, and
Niels Bantilan: we've hooked it up 'cause cause Flyte has a notion of files and directories.
Niels Bantilan: This is how you express like writing arbitrary bytes from that one container
Niels Bantilan: to the next, as I said earlier.
Niels Bantilan: also with data frames, so like parquet files,
Niels Bantilan: other data frame formats. And so the the power of like Pydantic Monty plus Flyte
Niels Bantilan: is that
Niels Bantilan: You can actually do IO indirectly because Pydantic Monty gives
Niels Bantilan: you the customization utilities to be like,
Niels Bantilan: okay, these are like allowed symbols and these are like allowed functions that
Niels Bantilan: you can write and
Niels Bantilan: you can actually compose this tasks that run on a separate pod,
Niels Bantilan: right? So the actual like control flow logic is still happening
Niels Bantilan: in the Pydantic Monty sandbox.
Niels Bantilan: It's just that the compute is happening elsewhere and the
Niels Bantilan: I/O is indirectly happening mediated by Flyte.
Niels Bantilan: So like Monty doesn't know things are happening in terms of I/O,
Niels Bantilan: but Flyte is just handling that under the hood.
Niels Bantilan: so you can actually
Dan Gerlanc: It just knows here's
Dan Gerlanc: your types, input output types and that that's cool.
Niels Bantilan: Yeah. Mm-hmm. Here's like the variable
Niels Bantilan: that is like basically a pointer to this thing in S3,
Niels Bantilan: right? and so yeah,
Niels Bantilan: code code mode I'm I'm having a fun a lot of fun playing around with 'cause it's
Niels Bantilan: it's a a nice kind of abstraction where tools are perfect.
Niels Bantilan: To to say, here are your building blocks,
Niels Bantilan: here are your nodes in the DAG.
Niels Bantilan: but there is no DAG anymore.
Niels Bantilan: Here's the agent. The agent's gonna come up with an execution graph
Niels Bantilan: to compose the the tools. And if you have any like prescriptions about
Niels Bantilan: how the tools should fit together,
Niels Bantilan: here's like some skills. And skills are just like text that specify how
Niels Bantilan: to use the tools in some sequence or in some way.
Niels Bantilan: so
Niels Bantilan: these are all like built into the Flyte SDK now and and are open source as well.
Niels Bantilan: so I think to circle back to your question,
Niels Bantilan: I think if an agent needs like
Niels Bantilan: disaggregated or heterogeneous compute where the the loop itself runs on
Niels Bantilan: a CPU is like not very memory or compute intensive,
Niels Bantilan: but the tools are
Niels Bantilan: It's like, hey, build me this model and do HPO on it,
Niels Bantilan: and you know, here's your metric that you need to optimize.
Niels Bantilan: this is basically auto research that I think Andre Karpathy popularized,
Niels Bantilan: which is like fancy hyperparameter optimization where like the search space
Niels Bantilan: is text over the code that it's writing.
Niels Bantilan: you can do everything in between.
Niels Bantilan: You can do like the very structured,
Niels Bantilan: config-based.
Niels Bantilan: Like here's like a model or set of models and here's like the JSON config
Niels Bantilan: for like the hyperparams for it,
Niels Bantilan: all the way to like hey, here's like an initial seed training file,
Niels Bantilan: edit it until you know you make the number go down to like
Niels Bantilan: a certain certain number.
Dan Gerlanc: Yeah, I think that's it's cool and it's it's interesting because like what Flyte
Dan Gerlanc: will take into account for is and for ML kind of problems,
Dan Gerlanc: which is always a big challenge is where is your data?
Dan Gerlanc: How does it get from one place to another?
Dan Gerlanc: Like if you're on AWS you can't be sending
Dan Gerlanc: gigs and gigs of data over the internet or I mean you can but it's gonna
Dan Gerlanc: be expensive
Niels Bantilan: Yeah.
Dan Gerlanc: and potentially slow, so
Niels Bantilan: Yeah. I mean that's how we're kind of conceptualizing this notion of AI runtime.
Niels Bantilan: I think we've seen peop other people talk about it in the market.
Niels Bantilan: we agree for the most part. Our specific take
Niels Bantilan: on it is basically durable workflows,
Niels Bantilan: like all the tasks and durability in this case means
Niels Bantilan: You have a 100-step pipeline, step ninety-eight fails.
Niels Bantilan: There the system understands that all the previous steps succeeded,
Niels Bantilan: and so you're not gonna have to recompute all that.
Niels Bantilan: You just start from the where you left off.
Niels Bantilan: that's like one piece of the durability puzzle.
Niels Bantilan: the second part is serving,
Niels Bantilan: so it's like you're not you're not only wanting to
Niels Bantilan: Yeah, so take the RL case, right?
Niels Bantilan: You're not only having this kind of like long-running compute-intensive training
Niels Bantilan: job. At every checkpoint, you also want it to spin up a separate inference server
Niels Bantilan: to do rollouts, right? So even in the model training case today with RL,
Niels Bantilan: like you'd still need online serving.
Niels Bantilan: and obviously the the the majority cases like here's
Niels Bantilan: a model endpoint that you like hit or your customers hit.
Niels Bantilan: and then the last piece is multi-silicon.
Niels Bantilan: So it's like that means multiple clouds,
Niels Bantilan: the GPUs and TPUs and whatever other processing units come up in the future.
Niels Bantilan: So like having a system like Kubernetes that like mediates
Niels Bantilan: the like container c container and compute orchestration behind all of it.
Niels Bantilan: so that's how we like think of this like AI runtime component.
Niels Bantilan: It's yes, you it it is agents as well.
Niels Bantilan: but I think moving into the future,
Niels Bantilan: agents are gonna be also compute bound at the tool level as well.
Dan Gerlanc: Yeah, like if you're compiling Rust code,
Dan Gerlanc: you're pretty compute bound.
Niels Bantilan: Yeah. Yeah,
Niels Bantilan: yeah. Yeah, exactly. so yeah,
Niels Bantilan: it's it's been it's a fun journey so far.
Niels Bantilan: Like we started in around twenty I think the team the Flyte team spun
Niels Bantilan: out of Lyft in like twenty nineteen and I joined in twenty twenty one.
Niels Bantilan: Or twenty twenty and then I joined in twenty twenty one.
Niels Bantilan: so it's been five years and less like the evolution in this space
Niels Bantilan: has just been like
Niels Bantilan: pretty nuts. like several just like existential moments for me,
Niels Bantilan: at least personally, of like, what the heck are we doing now?
Niels Bantilan: You know? yeah.
Dan Gerlanc: What
Dan Gerlanc: what do you mean like with agentic software engineering
Dan Gerlanc: or just with L LMs generally and our role
Dan Gerlanc: as software engineers
Niels Bantilan: Ex
Dan Gerlanc: or ML engineers?
Niels Bantilan: Yeah, exactly. That the latter the latter point.
Niels Bantilan: It's just like if if one gets caught up in the narratives,
Niels Bantilan: the Frontier Lab nar narratives too much,
Niels Bantilan: it it sort of just like takes you takes you for a loop and like just like
Niels Bantilan: questioning what what is the point of like having built
Niels Bantilan: all these skills up. But then when you start putting things into practice
Niels Bantilan: it becomes clear like, there's there's a role for me here actually.
Niels Bantilan: And you know, so that's I've I've settled to the point where it's it's
Niels Bantilan: it's clear to me that there's still like a role for a human ML engineer,
Niels Bantilan: AI engineer to build systems and build them like with assistance.
Niels Bantilan: but yeah like
Niels Bantilan: Creating a a auto research Ralph loop is it's it's not gonna get us to AGI.
Niels Bantilan: Like there's a there's a reason like Anthropic and OpenAI
Niels Bantilan: are still recruiting people. as much as they they talk about
Niels Bantilan: the other stuff around like recursive self-improvement.
Niels Bantilan: to to get that system you still need humans,
Niels Bantilan: right? So
Dan Gerlanc: Well and they're building out all these massive consulting arms.
Dan Gerlanc: Like if you're
Niels Bantilan: Yeah, yeah.
Dan Gerlanc: at if you're at AGI or if you've solved all the problems,
Dan Gerlanc: I I would think you wouldn't need to hire all those people to do that.
Niels Bantilan: Yeah, yeah.
Niels Bantilan: So yeah, I mean I I the the experience with Flyte has been actually quite different
Niels Bantilan: compared to Pandera because Pandera has just been a pure pure win
Niels Bantilan: in terms of agents, just because generally like you know,
Niels Bantilan: I have a two-year-old now, family and it's it's like time is very scarce.
Niels Bantilan: And so like the late nights I would spend till three in the morning,
Niels Bantilan: like building out a thing in Pandera,
Niels Bantilan: fixing bugs, maintaining the project,
Niels Bantilan: just just becomes a lot more tractable.
Niels Bantilan: and there isn't like a huge commercial pressure behind it.
Niels Bantilan: So it's purely just like Yeah,
Niels Bantilan: it's like my exercise of like,
Niels Bantilan: hey, I'm just giving this purely back to the community.
Niels Bantilan: People are benefiting from it,
Niels Bantilan: people are using it. It's useful and kind of boring in a sense.
Niels Bantilan: Like it's not it's not gonna get 10,000 plus stars.
Niels Bantilan: You know, it's it's like it's not like an agentic thing.
Niels Bantilan: So it's I'm glad it is where it is now.
Niels Bantilan: And I'm like like happy at the level it's it's gotten to.
Niels Bantilan: And I'm still excited. I think I have an
Niels Bantilan: angle that I want to work on that's goes a little bit more in the agent space,
Niels Bantilan: so like vector databases and like validating things that are not that
Niels Bantilan: are like quasi-tabular in the sense that it's a table,
Niels Bantilan: but there's like a vector inside of that or like images inside of that.
Niels Bantilan: so there are projects like DAFT and a few others out there that
Niels Bantilan: provide like a data frame like structure
Niels Bantilan: for multimodal data.
Niels Bantilan: Obviously I mentioned vector data databases and RAG use cases that
Niels Bantilan: I think are like interesting areas to get into for the project.
Niels Bantilan: And if not for adoption, maybe just as a intellectual exercise to just
Niels Bantilan: see how far I can take the kind of abstraction and like the framework that I've
Niels Bantilan: built. So and and to do it in the in a way that doesn't like bloat
Niels Bantilan: the dependencies and all that stuff.
Niels Bantilan: So I think there's like still a interesting engineering challenge there that still
Niels Bantilan: motivates me.
Dan Gerlanc: When is that is that on the pipeline?
Dan Gerlanc: Right into the roadmap
Niels Bantilan: Ha ha
Dan Gerlanc: right now, when when do we have that to look forward to?
Niels Bantilan: so actually a contributor just shipped the Narwhals backend.
Niels Bantilan: for I guess folks who are not super familiar,
Niels Bantilan: Narwhals is sort of like it's like that xkcd comic of like the standards.
Niels Bantilan: It's like but I think it's done a really good job.
Niels Bantilan: So I think it is actually interoperable
Niels Bantilan: data frame library that kind of like subsumes is like subsuming various
Niels Bantilan: of the other frameworks and wrapping it into like a really nice Polars-like API.
Niels Bantilan: and so we just released
Niels Bantilan: 0.32.0 for Pandera and
Niels Bantilan: That will add support for I think the Ibis,
Niels Bantilan: Polars, and PySpark schemas so that you can like use a narwhal's backend
Niels Bantilan: to validate those schemas. and so we'll you know we'll play some catch-up
Niels Bantilan: to like add pandas to that just because Pandera's pandas functionality
Niels Bantilan: is just like way more since it's more mature.
Niels Bantilan: so there's a little catching up to do on that front first.
Niels Bantilan: before getting to, you know, to throw out a few ideas out there of like LanceDB
Niels Bantilan: is a cool like Rust-based vector database.
Niels Bantilan: a few others out there.
Niels Bantilan: Part part of me like maybe is like,
Niels Bantilan: will Narwhals add support for?
Niels Bantilan: LanceDB or whatever, right? I think with LanceDB you c you can export it
Niels Bantilan: to Polars and then validate that.
Niels Bantilan: So there's some there may be some quick wins to be had there with some trade offs.
Niels Bantilan: But yeah, overall I think I'm curious about sort
Niels Bantilan: of the the place Pandera can play in like the Agentic AI stack.
Dan Gerlanc: Yeah, I think especially in the data science
Dan Gerlanc: w world of agentic AI. And I think that would
Dan Gerlanc: be the kind of place where you'd want your agent using Pandera
Niels Bantilan: Mm-hmm.
Dan Gerlanc: as part of
Dan Gerlanc: your data validation and it's going to the agent can
Niels Bantilan: Mm-hmm.
Dan Gerlanc: make corrections if it gets errors from the validation and iterate there.
Niels Bantilan: Yeah, yeah, exactly.
Niels Bantilan: I yeah, that's the my decision to not make HTML report Pandera error reports
Niels Bantilan: is I'll do it eventually, I think.
Niels Bantilan: I think it's kind of unavoidable to have like a nice,
Niels Bantilan: pretty artifact. so there's a project called Pointblank,
Niels Bantilan: the great validation library that I think was ported from R into Python.
Niels Bantilan: there
Niels Bantilan: Main differentiator is is differentiator is part besides the API,
Niels Bantilan: like you express validation rules differently,
Niels Bantilan: is you have beautiful like HTML reports,
Niels Bantilan: which I think, you know, I think is definitely coming from the R
Niels Bantilan: DNA of like visualization. Whereas Pandera just like spits out of plain text,
Niels Bantilan: like it's it's not beautiful, but definitely it's agent ready.
Niels Bantilan: I mean it's a very concise
Niels Bantilan: blob of text that says which columns failed and like what were the failure cases,
Niels Bantilan: what were the types that failed.
Niels Bantilan: so yeah,
Niels Bantilan: it's you know, it's I think
Niels Bantilan: I haven't had a finger on my pulse recently of how agents are starting
Niels Bantilan: to be used in data science. I know Eric Ma's work,
Niels Bantilan: he I mean he's like running a tutorial in SciPy this year about it.
Niels Bantilan: and so yeah, I I wanna like make sure Pandera is kind of ready when that
Niels Bantilan: adoption starts to take off so that like you know it's it's
Niels Bantilan: you know
Niels Bantilan: whatever th it means to be agent compatible,
Niels Bantilan: like Pandera is is sort of that soon.
Dan Gerlanc: And you can use the agents to help you write pretty HTML reports too.
Niels Bantilan: Yeah, yeah, exactly.
Niels Bantilan: Yep, yep. Just use Jinja, write some HTML,
Niels Bantilan: whatever like templating it needs,
Niels Bantilan: it just uses Jinja. So yeah,
Niels Bantilan: it's I think markdown is still good for like the human AI interface.
Niels Bantilan: but I think from a display perspective,
Niels Bantilan: like HTML, you know, you just put whatever interactive elements in there you want.
Niels Bantilan: it's it's pretty pretty nice.
Dan Gerlanc: Yeah, I've seen that as the back
Niels Bantilan: So
Dan Gerlanc: and forth now between people saying,
Dan Gerlanc: you should use HTML as the output format for agents,
Niels Bantilan: Yeah. Yeah.
Dan Gerlanc: but it's
Dan Gerlanc: like your diffs your your diffs start to have a lot of stuff in there
Dan Gerlanc: you might not care about, at least related to the content,
Dan Gerlanc: things like that. So I'm
Dan Gerlanc: still kind of in the markdown camp as well.
Niels Bantilan: yeah it's it's nice to just write in it.
Niels Bantilan: I think that's it's anything that that is for human eyes,
Niels Bantilan: I think will will have some staying power for maintainability.
Niels Bantilan: like human in the loop maintainability.
Dan Gerlanc: Niels, thanks so much for joining us today.
Dan Gerlanc: It was great to have you.
Niels Bantilan: Thanks, Dan. Great to be here.