AI Workflows in Agno: Building Deterministic Agents

Enterprise data is full of sensitive information: different teams, different access rights, different rules. When you ask an AI agent a simple question and get "access denied," it's not a permissions bug. It's a design problem.

Pascal has been exploring how to tackle this using Agno, an agent framework built around deterministic workflows. Instead of letting a single agent roam freely across your data, Agno lets you build specialized agents, each with its own access rules and instructions. Workflows orchestrate these agents with guardrails that keep humans in the loop when it matters.

In this episode, Pascal Knapen, CTO at Dataminded, demos the full flow: from a natural language question, through an access check, to a verified answer. We explore how skills differ from workflows, how Agno handles dynamic agent creation and deployment, and how LLM-based evaluations act as a quality judge for agent responses.

Additional Resources:

• Demo code: https://github.com/datamindedbe/demo-technology-exploration/tree/main/demos/agno-workflows

Intro music by Aleksandr Karabanov from Pixabay

• (00:00) - Intro: AI agents and enterprise data
• (01:50) - Two ways to give AI access to your data
• (03:23) - Skills vs Workflows
• (04:45) - Demo: AI with controlled data access
• (07:45) - The workflow is deterministic - the agents aren't
• (09:13) - Demo: evaluations & reliability
• (11:00) - Code walkthrough with Agno
• (17:45) - Why Agno? An honest take
• (19:46) - Identity & exposing as an API
• (20:24) - Takeaways

Creators and Guests

Host

Jonny Daenen

Head of Knowledge at Dataminded

Guest

Pascal Knapen

CTO at Dataminded

What is Technology Explorations in Data & AI?

Deep dives and practical demos on the technologies shaping modern data and AI development. Join the Dataminded team as we explore, unbox, and critically review the latest tools, from building AI agents and RAG systems to optimizing cloud costs and accelerating data pipelines. We cut through the hype to show you what actually works in real data engineering practice, complete with demo code!

Jonny Daenen (00:00)
You ask your agent a simple question. What's the average order value? It says access denied. Not because you're not allowed, but because nobody told the agent who you are, what data you can see, and what rules to apply to you. That is the reality of AI agents and enterprise data. Powerful, but unpredictable by default. My colleague Pascal has been exploring how to fix that by combining specialized data agents with deterministic workflows using a framework called Agno.

In this episode, he'll show us how it works from a natural language question to an access check to an answer. I am Jonny and welcome to Technology Explorations at Dataminded.

Jonny (00:46)
Hi everyone. Welcome to technology explorations at Dataminded. In this series, we give you an initial look into new or interesting technologies. And today we'll have a look at Agno, a tool that allows you to build workflows to shape your agents and talk to your data. And for this, I've invited Pascal, the CTO of Dataminded. Welcome Pascal.

Pascal (01:06)
Hi, Jonny.

Jonny (01:07)
Before we dive in, could you quickly introduce yourself and your role at Dataminded?

Pascal (01:10)
Well, as you mentioned, Jonny my name is Pascal,

I'm CTO of Dataminded. That means I'm responsible for the technical vision I'm also responsible for our products here at Dataminded We have two products. One is Conveyor, which is ⁓ a data engineering workbench. And on the other side, we have an open source project it's called Data Product Portal to start treating data

more as a product.

Jonny (01:36)
Yeah.

And speaking of data as a product, if we extend that to the AI scope today, we see a lot of organizations that want to talk to their data products or talk to their data. And this is also the scope that we will cover today, right?

Pascal (01:50)
No, absolutely. And that's actually how I encountered

So if you want to actually enable talking to your data, how do you approach this, right? Do you give an AI agent access, full access to all your enterprise data? At least I would be very concerned.

But this is indeed what we see people do. You have Claude running on your local machine. use MCP to connect to various tables, schemas, semantic models to answer questions. But what's an alternative? Basically, could also

instead of talking directly to the database, you could also be talking to individualized specialized agents that know about the data, the underlying data, the rules, the semantics and everything that's related.

then I can look at agents more like people with their own skills, with their own speciality, and it also allows them to actually limit your scope. So limit the amount of tools they need access to, limit the amount of knowledge they need to be able to gather. So it creates a little boundary of isolation rather than having one giant intelligent super being. You create multiple smaller intelligent

Jonny (02:58)
Yeah, you get an assistant for every data product So you flip the model actually, instead of having this MCP and giving it your personal credentials, you actually give the AI the right credentials, but you need to have permissions to deal with the AI in that case,

Pascal (03:07)
Yep.

Yes, indeed.

what happens is, course, what if you don't have access, right? But there might be an agent that knows the answer. I think this is where workflows come in.

It's all about getting deterministic results with workflows, a user might ask any kind of question and then we could actually figure out what agent can actually answer this question, And then of course, to ask is, do you actually have access to use this agent or

to use the underlying data. And if you have access, the agent can just go ahead and answer that question for you. If he doesn't have access, you can request it.

A workflow is really something very deterministic. You write it in Python, it's really do this, then this. It's basically a state machine. And I think it's very good when you have predictable, repeatable and complex pipeline, right? it's very hard in a skill to express things like conditional things, like, you you have to...

do this in these circumstances, but if you then reach this condition, you need to do this and this, and you can have very complex state machines.

Skills is for me a handbook. It describes how it has to do things. Yes, you can also give it a bunch of tools, but it's up to the LLM to still defer when to use it, how to use it, and do these kinds of things.

a workflow is way more deterministic, you have to go through these steps. ⁓

Jonny (04:30)
Yeah, it seems you flip the

whole skill thing. With skills, have markdown and then you have a few Python scripts. And here it seems you have this framework and then underneath that you have the agents.

Pascal (04:37)
Yes. Yes.

yeah, absolutely.

And for this case, I'm actually using Agno.

what you see here, Jonny, is agentOS. It's part of their commercial version, but you can also use it freely.

this thing it just runs in the browser you can just sign in and what you can then do is actually just sign up an agent so i have already done this it's the dpp agent and it just connects to something that's running on my local host right it's just a fast api running on my local host

And because it connects there, it basically exposes all the AI agents and workflows that I actually had built in this.

And so what I've done is that you'll see here I have actually three data products and each one of these data products basically have their own specialized agents. I can see here what's their configuration, it has access to a bunch of tools like a SQL database so it can actually answer questions about the SQL database, it has a system message and so on. And so I could ask here something

like what is the average. ⁓

order size, We took like a model of the little organization that sells camping gear. and now that agent is going to basically call my agents that are running on my own computer and it's going to do this. And, it says like, you know, I can help you, but it's first checked access. what it's telling me is that I actually don't have access to query this data.

wondering if I'm in this real situation how would I solve this? instead of just having a random tool somewhere that can approve this I would probably model this as a workflow and that's exactly what I've done. I'll show it first from user experience and then I'll show it a little bit from how it actually looks like in code. So what we see here is that our workflow actually consists of multiple steps. When I ask a question he's gonna identify the agent that's probably gonna

to be able to answer my question. It's going to check if I already have access. If I already have access, it's just going to answer it. If I don't have access, he's basically going to look like, am I allowed to grant this person access? In this case, he's just going to say yes. But I've also built like more complex system where basically we're going to submit an access request to an external system, wait for actually sometimes a data owner to actually approve that access. And then you can still come back to this workflow and still

later on actually get the answer to this question. So let's ask the same question what is the average order value? so when I press this, you'll see actually this workflow get actually kicked off.

And it's going to execute step by step. So immediately it identified, this is the, the sales transaction ledger agent. tells me the reason why it actually chose this one. It's then going to check if I have access in this case, I didn't have access he's then going to see like, can I grant him access? Yes. And so he granted me actually access. And now basically then he's going to answer this question. And then you see here,

the average order value is it's very easy to build like various different kinds of agents.

Jonny (07:45)
And so the agent that you ask the question, it's more like a top level agent that distributes tasks to other agents based on some conditions I understand, right?

Pascal (07:55)
it's not even an agent. It's really just a deterministic workflow. It's a deterministic state machine where in each step agents might be involved. As you see the first step, it actually uses the ⁓ identify agent to actually figure out which other agents can answer the question that I have.

Jonny (07:59)
Okay.

Okay, so your top level flow is always deterministic. Those are the steps that you showed in the right side of the screen. Identify agents is the first one, then checking access, granting access if needed, then answering the questions. And each of these has an implementation, each of these steps. And so Agno is more like a workflow system that is an orchestrator of agents.

Pascal (08:18)
Yeah, yeah, yeah.

Yes, indeed.

Yeah, so I call it an agent framework. Nowadays you have so many out there and most of these actually allow you to build like...

Jonny (08:43)
Yeah.

Pascal (08:46)
AI agents and most of them also allow you to build workflows. And I think there's always a lot of emphasis on these, on these agents. while I do think workflows are equally important because for some very important cases like granting access. You want determinism. You don't want the AI agents starting to drift or decide by themselves, like on some basic rules, like what it can do and what it can't do.

Jonny (08:59)
Okay.

Yeah, exactly.

Yeah.

Pascal (09:13)
very

easy to build workflows. ⁓ It has lot of bells and whistles included. you can have memory, evaluations, something I really like. If I want to test something I do reliability and I want to say how reliable or how accurate is actually whatever I'm doing. ⁓ around

80 I think it was right. You can say like you know I want three iterations, run evaluation, okay I have to give it a name, Now it's going to run that same query over and over again and it's going to check whatever the agent responds if we get actually consistent results.

So you specify how often you wanted to run it because you know agents are non-deterministic, right? So this time I'm asking it not to a workflow. I'm asking it to an agent. So that means that once in a while an agent might be able to give a different result.

so we didn't get a full 10 on 10, right?

That's actually very cool to see, right? I wasn't expecting this because most of the time when I've run this in the past, I get a 10 on this question. But it's very cool here to see that you get this non-deterministic ⁓ answer

Jonny (10:19)
Okay.

Pascal (10:25)
Now I can look at each individual runs and I can kind of see,

actually the result was still correct. But because we were using AI as a judge, of course, like the evaluation can also decide the non-determinism on this thing. So he scored it like 10 in the other two cases. And basically for the evaluation, he scored this one a nine while it actually still produced the correct results.

Jonny (10:37)
Yeah.

Okay, so this is the non determinism in the evaluation part, which is done being an LLM. And I see, see.

Pascal (10:52)
This is the, yeah, yes, yes. And so

these are always things you have to take into consideration when working with agent systems.

Jonny (11:00)
these agents, you also defined yourself, right?

Pascal (11:03)
They're all just living in code. Let me quickly show you how workflow looks like. ⁓

Jonny (11:08)
Alright.

Pascal (11:10)
So a workflow is quite easy, right? You give it a name, you give it a description, and then you basically describe the steps, right? And the step can be, an agent, In this case, it just calls the identify agent. I can set a condition and then he's gonna run a Python function to see, and that's just gonna be to return

true or false to decide can I progress or do I have to stop and then like if it succeeds like it's gonna execute these steps and of course there's like else step so in case it would fail I could also give it an action you know and when all these conditions is done he's gonna go to this one to the next one and that's how it basically decides what to do. ⁓

Jonny (11:44)
Yep.

And in this case,

identifying an agent is a step, but that's also an agent that does the identification or how does that work?

Pascal (11:57)
Yeah,

yes, indeed. This is the identify agent,

I have specified ⁓ somewhere which, agents are available, what their capabilities are. You know, it gives a brief description of what kind of questions the agent can answer. And basically it has some instruction to see how it actually has to figure this one out. And then basically it has an output schema, right? I can say like, I want structured outputs,

I want the agent's log and I want to reason why you actually chose this one. And this is for me to be able to debug if he does something wrong. You give it a model and then basically the output of this step is gonna be just this agent selection.

Jonny (12:38)
So your workflow can use this because it's structured output to make decisions like you do in your second step.

Pascal (12:42)
Yeah

Yeah

in my case, my next step is the check access elevator, right? Which basically for me just checks in a local file that keeps track of which agents every user has access to,

Jonny (12:49)
Yeah.

the sake of the demo,

is sufficient, I think.

Pascal (12:59)
Yes,

yes,

Jonny (13:00)
And so the last agent you have here is the one that answers the query. Did you need to do something special there as well?

Pascal (13:04)
Yep.

so there's this session state that goes around, right? I could get the step input, The step input just means the input, the original question The session state decided like which agent was selected to answer the question. So the only thing I do here is like figure out which agent I actually have to call.

Basically making the call and then streaming it back to my UI, the answer.

Jonny (13:29)
And where do you do the database query in this case?

Pascal (13:32)
So for each data product, I have a specialized agent. it's actually on the fly that I decide which agent to call.

Jonny (13:32)
it.

Okay, so you generate dynamic agents for data products.

Pascal (13:44)
I dump a YAML file somewhere, it basically generates me a completely new agent for a data product. If new agents come in, these are added, each one has like their own credentials to actually log in.

They have their own instructions, they have their own description, they have their own name. They have access to OSI files. OSI is Open Semantic Interchange. It's this new standard that is emerging to actually define your semantic models. I'm using that specification to actually describe the semantics of the data

Jonny (14:09)
I'll

Yeah, I see.

Pascal (14:16)
So these are statically created, but what I'm doing in a workflow on the fly is dynamically decide which agent to call.

Jonny (14:17)
Okay.

And here your agent is essentially fed with information about the data product, how to connect and how to answer questions.

Pascal (14:30)
kind of go.

Jonny (14:31)
you do

have this YAML file describing how the agent should respond and what it should do in this case.

Pascal (14:37)
Yeah, there's some generic instructions that I always add, but this kind of gives specialized information about this agent.

Jonny (14:45)
Yeah.

And seeing this, so you now have your Python file. This reminds me of how Airflow works. You define a DAG in Airflow of tasks. Here you define also a flow, a workflow with some agent coupling. How does this end up in Agno then? Is this something that's automatically synced or how do you get it in the UI?

Pascal (14:52)
Mm-hmm. Yeah. Yeah. Yeah.

So you create this AgnoOS object, Basically, you give it agent, you give it your workflows, I have enabled tracing to figure out what's going wrong in case things go wrong.

this file basically just defines a fast API which is called app and then of course like if I go into my Docker file you know I'm just using uvicorn

So this is running locally. I could just curl this right from here. But of course we all want a beautiful UI And you know, this is where this goes on, right? I basically say like, I have this agent.

Jonny (15:28)
⁓ yeah.

Pascal (15:36)
This is running on my local machine. So it connects directly from the browser to my Docker container running on my local system.

Jonny (15:43)
Yeah.

And can you have multiple?

Pascal (15:45)
Yes, yes, You see here, I have a Workflow agent and I have a DPP agent.

also from a deployment point of view, it kind of makes sense, right? Because we don't want all these agents being deployed by one Python process. we want to group them in a logical unit for deployment.

Jonny (15:56)
Yes.

And then it makes sense actually to have when you deploy a data product, also co-deploy an agent in its own container, for example, and make it available in one place, which in this case is Agno OS, it.

Pascal (16:10)
yeah,

Jonny (16:11)
so you register your container in this case, in this UI, as you showed, will it automatically pop up in your chat interface that you have here?

Pascal (16:20)
Yeah, so it will automatically show all the agents, teams and workflows. So here you see, if I want to connect a new one, If I have a new container, just say, I give it a name, you can also put some additional authentication on it. And then basically here all your different...

Agent OSes just appear here and then you can select which one you want to use and within each you basically expose agent teams and workflows.

Jonny (16:43)
And so

if you would now change the code, ⁓ rebuild the Docker image and run it, that would be reflected in the UI quite fast, I assume.

Pascal (16:52)
I typically have this task to reset my access, just task down. It's just killing all my containers, task up.

Jonny (17:02)
I haven't used this

Pascal (17:03)
And so basically whenever I change something, and I'm back up and running.

Jonny (17:07)
Yeah, and then the UI will detect the changes on the fly. Yeah.

Pascal (17:08)
He will first say like, know,

something's wrong and then you just refresh and then, you know, everything is green again.

Jonny (17:16)
And what I wonder, you have these workflows that are essentially multiple steps that can take a long time. What happens if something does take a long time? Like if we would ask approval, let's say we send a Teams message and somebody needs to respond,

Pascal (17:21)
Yeah.

So, yes, so you have a database attached to it.

So it will keep track of the status of your flow. And then you could come back and you can look at your sessions and come back to like a previous session.

now we actually come to the key question why I chose Agno I could have built this on Databricks. I could have built it somewhere, but

I wanted to kind of explore with an agent framework, like what I could build myself and also gain a little bit of a better understanding how and all these things actually were behind the scenes.

like Agno It's just a Python framework, It has a lot of good things actually going for it. it actually has excellent documentation. It actually also has an agent on the site, documentation site, that you can ask questions,

And there's a lot of things I like about Agno I've also hit some limitations here and there, which is always good, but it comes with a lot of batteries included. It has agents, has teams, it has a lot of tools. Access to databases, MCP skills, but also things like agent to agent, communication, evaluations.

love that as well. Like the fact that you can test an And in the end, like it really feels that built by engineers, it's all built on fast API,

It has like a very nice feature that I like in here.

something they call callable factories. It just means that to a lot of the parameters of a...

of an agent or a team or a workflow, you can just pass a function instead of defining it. And that then at runtime be called with the runtime information. And then you can decide at runtime actually which tools you're making available. this is basically to limit your context window, right? To keep the amount of input tokens as small as possible.

Jonny (19:14)
Yeah.

And based on what do you then decide? Is it based on metadata or also on a previous conversation?

Pascal (19:22)
you could look at the history if you want to. But basically it is deterministic Python code. You could use an agent there as well, but I'm not sure that's going to save you tokens.

Jonny (19:25)
Yeah.

Okay, yeah.

No,

Pascal (19:34)
depending on what your name is, your role, it knows your identity, right? So it could basically query some additional system, ask, what your permissions are based on those permissions, give you access to these kinds of tools.

Jonny (19:45)
Yeah, indeed, that was indeed a question I had, you were logged in into Agno, I guess. So it gets information about your role and your identity,

Pascal (19:50)
Mm-hmm.

Mm-hmm.

It gets information about my identity, that's basically something that's Agno just passing in, right? Because I'm logged in into Agno OS.

Jonny (20:02)
Yes.

does Agno then also allow deployment of agents? now you built this, you're developing the agent. Suppose you want to give it to me without giving access to Agno. Is there some way to expose the agents as an endpoint or using a UI?

Pascal (20:08)
Mm-hmm.

Yes. Like it's, it's just a fast API in the end. you get just a fast API object. You can plug any UI you want into that API.

Jonny (20:24)
So Pascal, this was a really nice overview of ⁓ Agno and what it can do What are your main takeaways after trying this out?

Pascal (20:31)
It is indeed new ⁓ technology and if I would be looking at a production system and I'm already using Databricks or Snowflake, I would probably look more into these systems first, especially for the talking to your data. I wouldn't like go and hosting my

Jonny (20:45)
Yep.

Pascal (20:53)
they have these things built in. What I do find interesting if you want to create like these workflows, right? And then I do think Agno is a very nice framework to use. And I think it's really built with strong engineering principles. It really comes with batteries included. It still has a little flaws here and there.

Jonny (21:14)
Yeah. it was good for me to get a view on what is a workflow and especially how it compares to skills. My take is indeed that workflows are the flip side of skills where skills are mostly markdown driven and can call some tools in Python, for example. the workflows are rigid frameworks where you plug in dedicated agents

Is there some code you could make available in our repository?

Pascal (21:36)
Yes, it's completely self-contained. So I can definitely share that. Yeah, yeah, absolutely.

Jonny (21:38)
Okay, so we can put that in our public repository. Very nice.

All right. So Pascal, thanks a lot for this enlightening demo. It was really good to see how Agno works and how workflows work. So thank you everybody for watching and give Pascal's code a spin, test out Agno and get back to us with any questions. And we'll see you next time. Bye bye.