Subscribe
Share
Share
Embed
Short on time but curious about how intelligent technology and AI are reshaping the enterprise world? Fear not, my friend. The Intelligent Enterprise is your short, sharp pause from the chaos of business as usual.
Every other week, Creative Thinker and Business Strategist, Tom Stoneman, meets with a business leader for a simple, real-world activity outside of tech, to talk about what’s going on inside it. It could be a coffee, a walk, some DIY, who knows? But it’s a chance to take a break, reflect and get to the heart of a single challenge in enterprise transformation with the help of leading experts in digital transformation, change management and workplace innovation.
From AI implementation strategies to debating the next wave of tech leadership trends, this is a show for CIOs, CTOs, and anyone navigating the fast-moving world of enterprise solutions.
The Intelligent Enterprise is brought to you by Digitate, creators of Ignio, the award-winning AIOps platform that keeps tech running smarter, faster and more autonomously. Join us for a coffee, run, or maybe even a dog-walking session, and get the inside scoop on what’s next for the enterprise world, by getting outside of it.
Tom Stoneman:
Hi, I'm Tom Stoneman, and this is The Intelligent Enterprise, where every two weeks, we take a break from the chaos of enterprise life and get inside a big idea by getting outside of it. Each episode, we meet an industry expert who helps cut through the noise from all the updates and rollouts while exploring one of their favorite break-time activities. It might be over a kayaking session, a game of ping pong, or some time at the gym, something that gives them some head space when they're deep in a problem.
Today, Ash Ashutosh, CEO of Pinecone is back back for part two of AI, The Hidden Layer. In our last conversation, Ash walked us through the rise of agentic AI and why it demands a completely new kind of infrastructure. He also introduced us to Pinecone Nexus, a knowledge engine built for a world where machines, not humans, are doing the asking. Since then, Nexus has hit the market, and results are turning heads across the industries, such as financial services, insurance, and legal. Today, Ash is here to get into how it all works and help us think bigger about what today's lightning-fast AI advancements mean for the enterprise of the future.
Ash Ashutosh:
There was this notion of how many people report to a manager? Five, seven. But why? Because that's how much a human person could really manage. What if you suddenly had a brain that was 100x more capable? Those boundaries are gone.
Tom Stoneman:
Let's get inside the future of enterprises by stepping outside of them. It's funny, I was listening to the radio this morning, and they were talking about a pretty drastic number. I think 60% of CEOs were negatively concerned about AI, the negative impacts of AI on their organizations, and that's dropped now to below 50%. So, people are starting to understand this, understand how to put the strategies in place, I think, so that they're not just willy-nilly throwing things in there and having issues. So, when we spoke last, you had just announced Pinecone Nexus. I'd like to understand how Nexus uses... I think you have a new compiler and a proprietary language to bypass the traditional per-query text processing bottlenecks. And I'd like to... If you could talk about that a little bit about how that actually works.
Ash Ashutosh:
Yeah. We started seeing late last year, where a new kind of user started showing up and using a lot of our APIs. If you recall, Pinecone provides a fully-managed vector database service. And most of our customers come in using the APIs, the humans would query, get a response, query again, get a response, get their task completed. And we saw a very different user called an agent who would issue 1,000 responses in a second. And most of them were pretty dumb, had literally no context of what the information was. And like we talked in the last session, it's fundamentally because you are asking an agent to speak a language that was designed for human beings where you issue a query and get a response back. So, the first thing was to address the problem of, can we define a language for agents? Can we define something that the agents can actually be very efficient in speaking? It's like me trying to go explain to someone in some language I'd never heard of, but I'm going to try to babble my way through.
So, the first thing we said was, "Let's build something that is a language, a declarative language we call knowledge query language, KnowQL, that was designed explicitly for an agent to say, 'This is my task to do, and this is the format of the response I need. Give me a very clear structured data,' because I understand structure. I'm a machine. I understand structure." And there were other elements we added to that language. Make sure there's a certain level of accuracy. Make sure there was some security. Make sure you don't exceed certain budget, whether it's of time or some other expense that's built into what happens underneath.
The second part was, okay, now that I have an agent communicating to me in a very clear language, great. I have this massive systems of record out there, all kinds of data. My data warehouse data, my emails, my Slack, my call records, my notes. There's a ton of stuff. What we do with that KnowQL interface is to capture how. How is an organization performing this particular task? What we have is this reasoning agents that compiles to say, "Given this data, how do you create very, very specific knowledge artifacts that capture how the work is done?" And frankly, you couldn't have done this year ago. You needed reasoning models to happen.
Now, in some ways, the old way of doing things, people do this today. People either bring in ETL pipelines and brittle process to get there, or they just say, "Claude Code, go to this data and answer my question." And by the way, oftentimes, they would fail. Success rate was 60, 68%. You would burn through millions of tokens. But most importantly, the second time you asked the same question, you would do that all over again. So, not only did we create the artifact, but we pre-compile the artifact so that after your first build phase, as we call it, now, you get this 30 to 40 times faster. You ensure that the accuracy is precise. And most importantly, we are seeing closer to 98 to 99% reduction. And the more complex, the bigger the dataset is, the better the outputs are.
Tom Stoneman:
That is so impactful. While you were talking, near the end there, I started thinking about things that I know you guys talk about all the time, I'm sure, but footprint comes to mind. So, energy usage, big thing right now. Being able to do things that much quicker to be able to retain and use less tokens, less energy use, right?
Ash Ashutosh:
And accuracy. And I think one of the things we've got to be careful about is if we started at the beginning saying, "Where was the most interest coming from?" Financial services, insurance, and legal. All of those need very accurate results and very explainable results. Because if I went through Claude, if I went through any frontier model, each time, the answer might be a little different. I would hate to be on the other side of the table getting different answers.
Tom Stoneman:
Absolutely. I mean, in marketing, I might be able to get by with a percentage or two, right? Here and there, but not those guys.
Ash Ashutosh:
Yeah, that's fine.
Tom Stoneman:
Ash recently wrote an article on LinkedIn called Token Apocalypse, and he's not being overly dramatic. Here are just a few examples of the cost crises enterprises are facing. The CTO of Uber burned through his AI budget halfway through the year. Microsoft pulled back from a frontier model entirely, and Accenture has recently revealed soaring token bills. All of this has led to companies heavily throttling AI, rationing tokens, or banning trivial tasks like converting PDFs to presentation slides. I asked him how Pinecone Nexus addresses these runaway cost problems and why the answer isn't reducing agents. It's about smarter retrieval.
Ash Ashutosh:
Agents and AI are unbelievable technologies that truly transform and have an opportunity to transform almost everything. And number one, it would be an absolute shame and sin for an individual or a business leader not to adopt them, not to use them. And we've seen that. In the last few years, most organizations have gone through an experimentation phase to see what is possible? What can I do with this thing? Now that they're out of that phase, the next part is to figure out which of these things that I've tried out truly make business sense, truly get to the point where my use case has an ROI? So, now, you get into a very different set of discussions. We talked about governance, accuracy part, explainability part. We talked about speed. What I cannot do is to say, "My CFO, good news. I built 1,000 agents, and I know that you hadn't given me a budget to December. I just burned all of them in May. But gosh, it was so amazing. I have done such an amazing job, but I need more budget."
And we've seen this example. I think if you just look on the social media, you'll find many CXOs who are really struggling with how do you make sure you unleash the creativity and the productivity that comes with agents? And how do you balance that against what's really a true ROI? And so, that was our whole point. The point is in the absence of trying to do something different and just using brute force that the agents are amazingly smart, you have to come up with something newer that'll allow you to get the ROI that you need. By the way, I am guilty of it too. We all just find it so addictive to go back and unleash and manifest our creativity with these wonderful tools. So, once you move from experimentation to production and you realize this tokenomics is something we have to address, that's where what we've done with Pinecone Nexus is a good example of really digging and saying, "What exactly is this thing burning tokens on? And can I do something better?"
Tom Stoneman:
Ash has always found his clearest thinking when he's building something with his hands. In his previous episode, he told us that if something's broken, he wants to take it apart. And if it doesn't exist, he wants to build it. I asked him what he's been working on lately.
Ash Ashutosh:
Okay. I admitted a little while ago that I've become a vibe coding addict. Now, I almost feel like this Superman who can take on any problem. Just tell me what the problem is, and I'll vibe code my way out of the issue and solve it for you. But to support that, obviously, there's a new toy or a Mac Mini that's been recently introduced into my area of toys to start playing with. And this is less about fixing, more about actually starting to do some things that change the behavior of how I work. You're getting to the point where I've started building an AI desktop. And so, I've been working on building a desktop, an AI desktop just works for me. It's not designed by whoever the vendor of my laptop is, the vendor of my software is. And the way I work on my own laptop and the way I interface with the people around me is my own. And I need something that is hyper personalized for me. And that might be the biggest fix I can make.
If you want to talk about regular stuff, in some ways, the weather has gotten nicer in Boston, and it's time to go fix the tree house because we've had quite a bit of snow, and we did that part. And there's nothing more fun than getting your hands dirty and start going up the treehouse and hoping and praying I don't fall down.
Tom Stoneman:
It's kind of funny. In a past life, I had owned a commercial recording studio. I've since closed that all down, but I'm starting to disassemble it. And I love going... That's one of my favorite parts of all of that business was going in and building everything and getting it to work, right?
Ash Ashutosh:
I think, Tom, I've noticed this. I've talked to a lot of my colleagues. The ease with which we can all build with AI is almost creating on one side, an addiction, on the other side, a fatigue of a desire to go back and build something physical. And if you notice, there are several companies that have come out now building tiny, assemble-your-own combustion engines. Build your own something with the hands. I think there are a lot more companies coming out because people do want to feel like there's something physical. There's something that you've built, something that you probably just took it for granted. And once you start putting it together, you realize, oh my Lord, this is an amazing piece of technology that gives you a satisfaction. So, I think it's just not me. I think it's happening... And not you. It's happening everywhere. A lot more people are getting back to the-
Tom Stoneman:
I think so too. And that's a great segue into a few more things we wanted to discuss with you. So, Ash, you've organized Pinecone in a way that's really focused on initiatives, people running things end-to-end with AI filling the gaps. A few months on now, I mean, what have you learned from that so far? What's working? What's challenging? What do you see happening in the future?
Ash Ashutosh:
This is a very interesting topic that has emerged probably around few months, six months ago that we started noticing the work we do as individuals has changed. And so, the organization that's required to support that work needs to change. If you go back, I would say the org structures that we all created, that we all live in, what are the artifacts of the kind of work where each of us was? It was about making sure that a specialist received some information, did their part, then passed that information onto another specialist and so on, so forth until you got from one end to another. And because we were constrained by human's ability, both in terms of physical and our ability to understand it from a mental perspective, you had a hierarchy. There was this notion of how many people report to a manager? Five, seven. But why? Because that's how much a human person could really manage. Beyond that, you start not being able to take care of that one, of the team that you're trying to manage.
Now what if you suddenly had a brain that was 100X more capable? 500 people? In fact, what if every one of those people were able to do a lot more and had access to all the information and all the knowledge? So, this entire notion of silos that we created to support the concept where there was a human being, there was a person who was creating certain kind of information that was being passed onto the next one, and there was some curation required before I gave this information to somebody else. Those boundaries are gone. Now, you can give me the exact same document, and I can use one of the AI tools to say, "Okay, summarize for me the key points that Tom is trying to tell me. Actually, convert that into a poem because I actually like to listen to things in poems. Or better still, convert that into an audio text."
I am able to receive information the way I understand it, and you don't have to worry about curating that information anymore. And this notion of having to have somebody take some information, put it into this beautiful PowerPoint, and try to understand exactly how does Tom understand and absorb information? I don't need to do all that stuff. So, if you fundamentally change the premise that the capacity of quote, unquote, "The person," which in this case, an agent, and the ability for us to interpret information and knowledge the way we can and it's freely available, then maybe the way we run an organization is very, very different. And that's what we've been executing on. We fundamentally left the two bookends of the business on the engineering and on the sales side. Those have been focused on deeply changing how they do their work. But everybody else across the company has changed how they facilitate the work that's done by owning a single initiative.
Because working in a silo has no advantage anymore. There's no point in me being in an organization X where I am peanut butter across everything and I really don't own anything. And it's just fascinating. What is exhilarating for someone now in the company is they actually own, they are responsible. They have become owners of some parts of the business as opposed to being just somebody who's part of a vertical that just supports everything. It requires a very different mindset. It requires a different sense of curiosity, and accountability, and agency, but I think that's what AI allows us to do.
Tom Stoneman:
To close out, I want to put a question to Ash that we ask every guest on this show. Can you imagine something AI will be able to do either in the very near future or further out that's never been done before? For Ash, the answer lies in the physical sciences, and he thinks it's closer than we expect.
Ash Ashutosh:
In chronological order, I think we already see one of the things AI is able to allow people to do is to manifest their ideas into actual actions that they can actually deliver to whoever they're trying to work with. This is unbelievable. Being able to manifest, removing the barriers. That's almost like a given. It's obvious. It's something that's happened a year, year and a half ago, but I still see people just shocked and surprised that they're able to pull this off. But then, there are bigger things. I think the complexity of so many things that our human mind and collective human mind simply cannot fathom the dimensionality. I mean, we just talked about something like the basic AI construct called a vector. We understand one dimension. We understand two dimensions. We even understand three dimensions. And we can even add in a dimension of time and say, "Look, I can understand four dimensions."
Let me talk about vectors that are 1,500 dimensions. How in the world can I even imagine what that means? But AI does that all the time. And so, you apply that to very complex problems, even simple things like human body. You've seen massive, massive progress that's being made on the entire biology side. So, I think the biggest ones would be areas of physical sciences that we had just never been able to address because the complexity and the dimensionality of that is so big. If that changes, then it is going to change. It's not too far off, by the way. This is not a 10-year plan. We're talking about a plan that is in a couple of years. And then, there is, who knows? We don't even know. Our mind is so small, at least mine. I cannot even fathom what this creative people for humans can do with AI.
Tom Stoneman:
I was quite the visionary, I think, when I was in fourth grade. There was an assignment we had where we were asked what we thought the world would be like in the year 2000. Of course, back then, that seemed like almost unfathomable.
Ash Ashutosh:
It's something that old people live in.
Tom Stoneman:
Yeah. Does that year even exist? But the thing I put down, I was sure that we would be living side by side with machines, I said, working together. And I was completely dismissed, of course, because I was a kid. So, I should have written the script then and gotten Terminator going, but I did not. Ash, thank you again. We've run out of time. We're at the end for this session. We hope to have you back soon, but thank you so much for being here today. Again, it was just a great, great time talking to you.
Ash Ashutosh:
Appreciate you hosting me as always, Tom.
Tom Stoneman:
Ash has a way of making the future feel less abstract. The org chart built around human limitations, the five or seven direct reports, the silos, that's not a management philosophy anymore. It's a constraint that's dissolving in real time. What replaces it, according to Ash, is ownership. People who don't just support the work, but run it end to end. Whether that's exhilarating or terrifying probably depends on where you're sitting, but either way, it's already happening at Pinecone. Thank you for listening to The Intelligent Enterprise, a podcast where we get inside big ideas by getting outside of them. I've been your host, Tom Stoneman. Please remember to follow the podcast and leave a comment or review wherever you get your shows. See you next time.