Subscribe
Share
Share
Embed
Come join us as we discuss everything open source with guests that are pillars in the industry. Welcome to The Secret Sauce.
Welcome back to the show. We're about to learn the secret sauce.
Jeff Huber:Yeah. What's up? How's it going?
Bdougie:Cool. So you are actually, I don't even know what your actual tighter, title, co founder of Chroma. We'll go with that. Yeah, that works. Excellent.
Bdougie:So Chroma is a thing that we're gonna get into. But let's start with, like, who Jeff is.
Jeff Huber:Yeah. Sure. So my name's Jeff. Let's see. I've spent a bunch of time in the Bay Area growing up.
Jeff Huber:Spent a good number of years in North Carolina growing up. Okay.
Bdougie:Is that where you're originally from?
Jeff Huber:I was born in Peninsula.
Bdougie:Okay. When
Jeff Huber:I was young, moved out to North Carolina. Did, like, grade school, middle school, high school, college.
Bdougie:Yeah. Were your parents doing tech in the RTP or something like that?
Jeff Huber:Yeah. My dad actually had, gotten Berkeley for his MBA and then worked at HP for a bit and then went to IBM. There's a good giant, like, IBM campus Yep. In the triangle at that time.
Bdougie:So Okay. Cool. So what's what's Chroma? What are you working on right now?
Jeff Huber:Yeah. So Chroma is an open source vector database, And our goal is to give, developers the ability to create programmable memory for AI. And when I say programmable memory, what I mean is, language models are really powerful, obviously. But we want to be able to bring language models into our applications and our user experiences and our internal processes, and we want to make them really reliable systems, that do what they're supposed to do every time, you know, even when the real world throws kind of, you know, messy information at them.
Bdougie:Yeah.
Jeff Huber:And so, the way to do that is to give the language model more information than it knows about in its own weights. So if language model's seen in which stuff on the Internet, but doesn't necessarily know about your company's data. And so a vector database is the bridge between the information about your team, your tools, your data, and makes that accessible to the language model so that it can make the best decision possible, at that prompt time.
Bdougie:Yeah. And then I I I've known that you've been working on this for a little bit, the last 14 months. And you kind of you were, like, pre kind of a huge wave that we're seeing right now in AI. So, like, we we coming out of, like, this AI agent craze, and everyone's, like, raising money. But now there's, like, a a sense of we get to build a thing that builds a thing underneath.
Bdougie:Do you wanna talk about the origin of Chroma? Like, how you sort of, like, started working on this problem? Why you started working on this problem, and how you got to this solution?
Jeff Huber:Yeah. So Chroma started about 18 months or so ago. So my cofounder, Anton, and I started just chatting about our experiences as engineers. We have both spent quite a bit of time doing applied machine learning. We call it applied AI.
Jeff Huber:So building models and putting them out into the real world, and then seeing how they get mugged by reality and then trying to figure out how to improve them so that they're more robust. And, you know, that that cycle, that feedback loop is incredibly painful to this day. Yeah. There's basically almost no tools, extremely little tooling to figure out. Okay, here's my model.
Jeff Huber:It's, you know, hitting 80% today. We have to get to 95% in order to have it be really be a useful tool. How do we get how do we cross the bridge of that 15%. Just throw more data at it and let's hope, you know, hope and pray that it gets a little bit better. Yeah.
Jeff Huber:And so I think that that that feedback loop we had just seen and felt so viscerally and so painfully ourselves, and we wanted to cross that gap. So that's the gap between broadly when you use anything with AI, the gap between a sexy demo and a robust production system. And it's it's a tough gap. So that was the core pain point that we connected over and started talking about. And then, in my previous role, I started to build out a bunch of developer tools around AI, computer vision, machine learning.
Jeff Huber:We were open sourcing some things and I just felt like, open source was awesome. And I wanted to build an open source community. I wanted to build an open source commercial project. And so, you know, that was something that was, like, true from the very earliest days. Was whatever we do, it's gonna be open source.
Jeff Huber:And, you know, we'll figure out everything else
Bdougie:later. Yeah. That's amazing too as well. And, like, I I've always been adjacent to the ML space and, like, never really sort of dug in, but I was very aware of the struggles of getting started and, like, the GPUs and, like, getting hands on things to have enough power to then get data into it and then train it. Mhmm.
Bdougie:And then only being not even halfway there to having something usable. Actually, at GitHub, I had a colleague who because we got acquired by Microsoft, we had free Azure compute. So my colleague, he, basically, just got uses GPU credits to then build a a language, not even my language model, but, like, to build some ML to then look at Octocats and, like, sort of, like, is this a hot dog type of deal?
Jeff Huber:Yeah.
Bdougie:Just build that for Octocats. And
Jeff Huber:Okay.
Bdougie:He spent, like, 6 months, like, finding all the sort of taking pictures on his phone and then sending it to the model. Yep. And it was, like, it was cool, but, like, it was a lot of work for I don't know where that went. It was just like a little side side project to spend compute because we had it. Yep.
Bdougie:But yeah. The barrier of entry of getting the AI has completely shifted. Like, the thanks to I I I don't know if it's thanks to OpenAI. I'm happy to get pushed back on that, but they have provided enough tooling that people can grasp, like, what they can do with it now. So now we have the influx
Jeff Huber:Yeah.
Bdougie:A bunch of startups on top of,
Jeff Huber:right.
Bdougie:Frankly open AI.
Jeff Huber:I mean, we we got started. You know, there's there's no stable diffusion yet. Yeah. GbD 3 was, like, technically out, but nobody really knew what you could do with it. You know, CHI GPT came out last November.
Jeff Huber:And, you know, I think there there's a phase change from before you had to build an ML model to do a specific thing. Yeah. So it was like, is this tweet mean or not? You know, is this photo of a bird or a dog or a cat? Where how many you know, put a bounding box around the cars in this photo.
Jeff Huber:Right? And it was a again, it was a huge lift. It was quite difficult to build these models. And then if you built them, they'd only do one thing. Yeah.
Jeff Huber:And then, there's this phase change now to where we have general purpose models. And, you know, the general purpose models can do a lot of things out of the box. And so, you know, all of a sudden what before was only the the domain of AI experts became in really a very short time, you know, the domain of application developers who truly, on a weekend, can pick up some of these tools, plug them together, and deliver some pretty meaningful new and powerful user experiences. So it's super cool.
Bdougie:Yeah. And so, like, the trick we're seeing right now is we're, again, building the thing thing that builds the thing. So now we have things like Chroma where correct me if I'm wrong. I could build my own chat gbt with my own data pretty quickly with the tool like Chroma. Right?
Jeff Huber:Yeah. Definitely. Yeah. I think as a developer, you could pick up the GPT 4 API. You could pick up Chroma and then you could pick up a bunch of documents you have.
Jeff Huber:It could be a common one people start with is, like, their support knowledge base or it could be a bunch of Wikipedia information or private notes. And probably in 2 hours, you can have a chat experience where you can ask questions, you know, do natural language q and a just like you would in Chat GPT over those documents, but it knows about your data. Yeah. So you can say, hey, what is my company's return policy? And it'll be able to retrieve the relevant information and use that in answering your question.
Jeff Huber:So it doesn't just make or hallucinate an answer. And, that's a pretty useful thing, obviously.
Bdougie:Yeah. Extremely useful. I think on the on my Uber ride over, we have, Becca, who's at our at Open Sauce, writing a bunch of content around open source and getting started. And she was actually had the the problem because she's written, like, I don't know, like, 40 plus articles, since she joined in May, which is absolutely amazing. Shout out Becca.
Bdougie:But now she's got the issue of, like, oh, have I said this before yet? Or where did I reference this in another place?
Jeff Huber:Right.
Bdougie:And because it's all on the web, you could open up, like, you know, the local, markdown files. So they can, like, get pull it up in Versus Code and be like, oh, let me just command find to see if I mentioned this string of
Jeff Huber:words. Mhmm.
Bdougie:But sometimes it doesn't work when, like, you use different phrases or tone. So I was, like, oh, you should just, like, vector databases thing and, like, start start searching, for a stuff you've done before and even, like, generating new posts based on the existing post because that's what we're doing right now. We're gonna remix a lot of the posts we've done on, like, dev, put it on our blog Mhmm. For, like, some of the the the number one hits, the stuff that gets a lot of traction. Yep.
Bdougie:And then attach that to, like, a feature. So then our blog, like, leads in our features where the community stuff is, like, not feature focused. Yeah. So anyway, this the world is like amazing at this point.
Jeff Huber:Yeah. I mean, the obviously language models are really powerful, really useful. I think also, it's worth mentioning that in just the past, you know, couple years as well, embedding models have gotten to the point where they are also general purpose Yeah. And and quite good. And so, you know, what embedding models are trained on is semantic similarity.
Jeff Huber:And so, they're trained to put similar content close to each other in this, like, higher dimensional embedding space. And what that allows as a developer is this ability to do fuzzy search. Yeah. So instead of having to search by an explicit string or, you know, variance of this explicit string like you would with text search. You can say, hey, where did I talk about that thing where, you know, that developer was, like, having trouble with something?
Jeff Huber:Yeah. And the vibe, if you will, will be embedded. And then you'll do, like, vibe search and you'll find that you'll find the documents that Yeah. That share that vibe. That share that semantic similarity.
Jeff Huber:And, and that's pretty cool too.
Bdougie:Yeah. And it's the term that, when I was doing all my research to build the thing that we're building right now, like, semantic search is the like, it gets attached to vector databases. Yeah. Is that, like, the term that people would probably be looking for if they're if they wanna build something like this? Yeah.
Jeff Huber:I think the exact terms haven't landed. There's retrieval, vector search, vector retrieval, semantic similarity, semantic similarity search. I mean, I can keep going. There's a good ton of them.
Bdougie:Yeah.
Jeff Huber:You know, the macro term that we really care about and we've been thinking about is this idea of AI memory. Yeah. And so, again, the question becomes, you know, if, Becca asks, hey, why did I talk about this thing? How how should that search process be constructed such that we find all the relevant information that possibly the language model would need to know about
Bdougie:Yeah.
Jeff Huber:And then put that in the context window so the language model can see it and then use it.
Bdougie:Yeah.
Jeff Huber:And so in some cases, that's gonna be semantic similarity powered by embeddings. In other cases, it might be, you know, hey. Okay. Becca mentioned, like, Twilio, the search for stuff related to Twilio too because maybe that's what she wants to know about. And so I think there's maybe this you know, the broad goal here would be to, again, create programmable memory for AI, and vector search is a really powerful tool in the toolbox.
Bdougie:Yeah. And there's a there's a current conversation that what? Honestly, I think we have the conversation to close now of, like, the death of Stack Overflow. I don't know if you saw those tweets Yeah. From that.
Bdougie:Mhmm. Literally this week. Yeah.
Jeff Huber:I
Bdougie:think that was kinda going the bat of, like, there was a different situation for Stack Overflow and why the downturn. It wasn't specifically AI. That too. I situation for Stack Overflow and
Jeff Huber:why the downturn.
Bdougie:It wasn't specifically AI. I saw that too. But then in that same vein so, like, the the first tweet was, like, oh, look at the trend. Chat gbt killed Stack Overflow. Overflow.
Bdougie:Not so much the case when you sort of dig into it, but then at the end of the week was, like, yesterday or the day before. Was it AI overflow? Or I don't know what they called it, but Stack Overflow, Shifter, the cheap TV, like like, experience.
Jeff Huber:Yeah. Yeah. I saw that too.
Bdougie:Yeah. So, like, now we're seeing a trend of, like, the incumbents of, like, all these I was gonna call them startups. Like, cyber flow is no longer a startup. But basically, all these companies now applying AI as part of their feature suite, which is, like, mind blowing. So, like, going back to, like, Becca's example, like, if we're only writing on Hashnode or dev.2, the world probably looks like Hashnode and dev.2 also have their their vector DB to then have the question of, like, I wanna find all, what it whisper generated post, to be able to find out more details of examples in this.
Bdougie:And, like, not everyone puts on hashtags and you can't sort of, like, find the thing you're looking for.
Jeff Huber:Yep.
Bdougie:Is every company gonna have their search experience now powered by Vectr?
Jeff Huber:Just a few questions that kind of package in there. So one question would be about, like, data licensing. Yeah. And with the advent of of AI models, large corpuses of data just became way more valuable. Yeah.
Jeff Huber:And we'll let the the courts decide about, you know, who it should be allowed to use what. Yeah. You know, I'm not a lawyer, I a n a l. Right? I think that the other question mark here is around, like, will there be, you know, one chat interface to rule them all?
Jeff Huber:Like or will we still go to, you know, distinct web properties and then interact with a custom experience that web property has has given to us. Maybe a simple example here would be flight search. You know, well, 5 years from now, what percentage of flight search will be done through the ChatGPT interface versus what percentage of flight search will still be on KAYAK, or elsewhere. Yeah. That's that's one simple way to kind of hone the question.
Jeff Huber:And I think there was a moment in time earlier this year where, you know, it see certainly either seemed like or the the the feeling was maybe that, you know, so much of the web's traffic would coalesce to this one magical box kind of in a way that, oh, maybe that Google did before, you know. And, it feels like the jury's out on that. Again, I think that I'm I'm not gonna predictions of this time scale are the the death valley of predictions. You're always wrong. So I won't hazard a prediction, but, you know, it doesn't seem as obvious.
Jeff Huber:It doesn't seem like, for example, you know, Chachiki plugins have quite found product market fit. Now, that could be, you know, one small tweak away as you know with these things, you know, you don't wanna you don't wanna overstate how far off they are. It could be one small change. But again, yeah, it's kind of an open question. And I think that, like, I still think that, you know, people with passion and dedication will be able to create better experiences for end users.
Jeff Huber:And then if those experiences are even 10 or 20% better than, like, the mainline experience Yeah. And users have the muscle memory of going to the URL bar and type in k a and then it auto completes KAYAK and they click enter Yeah. Then I think that, like, users will still do that. So if I had to hazard a guess, I would guess that we would see basically a multipolar world Yeah. When it comes to, you know, AI experiences interfaces.
Jeff Huber:I think also one other comment, it's not really obvious that chat either can or should be the final UI for for frankly anything. Yeah. You know, it's like, it is pretty magical in some ways, but it also has a ton of downsides. And, there are a lot of really interesting, you know, formal researchers and also, you know, Twitter researchers, which I don't mean as a as a as an insult. It's actually a compliment.
Jeff Huber:Yeah. Who are doing, I think, really incredible interesting groundbreaking work around, like, AI user experiences, generative UI, this kind of stuff. Yeah. So we'll see how all that shakes out.
Bdougie:Yeah. And, like, our our our first AI feature we shipped inside of Open Sauced, which is take a PR generated description. Mhmm. So as trivial as as it gets, like, we centralize around that to learn as a team to, like, okay. This is how we could interact with this new ecosystem.
Bdougie:Yep. So I'm a 100% with you. Like, there's things we could do more at AI and sort of extract it. Like, there's maybe a different interface where we sort of apply, like, pattern matching. Like, when when a VC looks at your your company or project, like, oh, yeah.
Bdougie:Does it fit the bill? Or it looks at the founder. Does it fit the bill? Mhmm. But I actually wanna take a step back because the other thing I wanted to I I wanted to mention is the not even mention Ask.
Bdougie:I don't know if you see, like, Elasticsearch as, like, a competitor because Mhmm. Like, when I think of, like, okay, all my data in one place, and I know GitHub spent a couple years Yeah. Rebuilding search Yeah. They did. On top of Elastic.
Bdougie:And it it mostly works today. And I because I spent all that time, like, I'm a power search user on GitHub. Yep. But not everyone else is. So is there a world where, like, even Elastic sort of gets their their lunch eaten?
Jeff Huber:It's a good question. Again, I would wouldn't necessarily get too too spicy. The hazarding guesses about, you know, other other companies. You know, I think for us, we feel like vectors and latent space more broadly. Obviously, a vector represents a point in latent space.
Jeff Huber:Yeah. This higher dimensional Like, quick
Bdougie:side note. I looked up what latent space meant because the podcast Oh, yeah. Were shot at Alessio, and I'm, like, I should probably look up what this means. Yeah. Because I keep hearing it in the podcast.
Bdougie:A genius name for a podcast for sure.
Jeff Huber:Yeah. So, you know, we for the audience, we humans live in sort of primarily three-dimensional space. We reason in 3 dimensions. Obviously, time is the 4th dimension. Yeah.
Jeff Huber:And then, you know, if you get back in embedding from OpenAI, I think it's 15 36 dimensions. And so what does it mean for something to be in 15 36 dimensions? It's a point in, you know, higher dimensional space. And so, I think, like, you know, for Chroma specifically, we feel like what's really amazing about latent space is you're basically taking data and you're hoisting it into the brain into the model. And that point represents how the model understands that piece of data.
Jeff Huber:And what's interesting about it is that it is still a map. It is a geometric data structure. And we think there's a lot of meaning and information and how the points relate to one another, and a lot of meaning and information, you know, where points are far apart or where those holes in the embedding space or the latent space. And so well, there's no information at all. And so, you know, these kinds of questions, I think that we are really obsessed by and, you know, we will uniquely build a system just to answer these kinds of questions.
Jeff Huber:And so, you know, everybody wants to have, you know, maybe this, like, take about all search will be eaten by vector search. I I don't think that's necessarily true. All databases will be eaten by vector databases. Again, also not true. It doesn't tend to be the case that, like, you know, previous technologies get totally wiped out.
Jeff Huber:I mean, even, like, your classic horse and car dichotomy. Right? Like, okay. Yeah. Maybe that's an extreme case because it is primarily a a a car's game these days.
Jeff Huber:Right? Yeah. But, their horses are still around. So, yeah, I think that, like, the right tool for the job, you know, is always, is always an important question.
Bdougie:Yeah. I I went to this a couple years ago with GraphQL versus REST, and I think everyone's kind of, like, pick their position in that sort of space. I think that's still being figured out.
Jeff Huber:But Right.
Bdougie:At the end of day, like, REST is always the tried and true thing that I always I keep going back to rest even though I love GraphQL. But there's certain situations, like, where GraphQL is just not needed. It's just more of a complexity. I
Jeff Huber:think with GraphQL and rest is, like, you know, you play another story much better than I do,
Bdougie:but, you
Jeff Huber:know, it kind of seems like GraphQL was meant for a very specific use case Yeah. Which was minimizing the data footprint of data over the wire for latency limited mobile experiences. Yeah.
Bdougie:And then Pretty much it. Yeah.
Jeff Huber:All of a sudden people were like, oh, this makes my life easier as an application developer for doing web apps where you're not really latency bound. And, as you know, like, just all kinds of it was just it was sort of it was aggressively adopted too fast. So, I guess, my macro point here is, I don't know that GraphQL was really 10 times better at anything
Bdougie:Yeah.
Jeff Huber:Than REST is today or just an HTTP interfaces today. Yeah. And I think it is true that vector search is 10 times better at things that'll last, you know, last year classic text search Yeah. Than it does today. So maybe that my argument would be that these are these two interactions are from apples and oranges.
Bdougie:Yeah. Yeah. So then my question would be, I'm a CTO. I'm just an engineer starting a new project or existing project rather. I've got a Postgres database.
Bdougie:I know I wanna advance my feature suite through perhaps having a vector database. Yep. Is there a world where my main database and my vector database live as siblings or is there like a sort of parent child relationship? Yeah. When it comes to vectors?
Jeff Huber:Yeah. I think it depends on the use case. So a lot of data that we're seeing that is being loaded into vector databases are not currently in relational databases. Yeah. It's textual information.
Jeff Huber:It's images, you know, long form text, not like someone's first name, last name. Like, first name, last name makes zero sense to embed. You should not embed first name, last name. No. Semantic similarity for Brian.
Jeff Huber:It's like, okay. I guess that means something, but nothing useful. Yeah. Semantic similarity for, like, long form text or images or these sorts of things, so, you know, soon video and other modalities. Again, things that are not already in a relational database.
Jeff Huber:I think that's the that's the first point. You know, second point is that like, you know, there's there's depends on what you're trying to do, whether you need the right tool for the job, you know. You can use a hammer to do a lot of things that a hammer wasn't designed for. Sometimes you can get away with this and then you just don't get away with it. You know, there's already a large, there's already a lot of evidence around people using tools like Elastic for text search alongside existing relational databases, because they're just better at different things.
Jeff Huber:Yeah. And, like, I think that the same path will likely exist here. And we can give you more, like, technical algorithms level of, like, why and where and, you know, what kinds of resource contention we're concerned about, etcetera. But Yeah. I think the macro point is that, again, you know, systems that are designed to do one thing very well end up doing it extremely well, relative to the systems that are designed to do everything pretty well.
Jeff Huber:And, you know, you can't have it all. So
Bdougie:Yeah. Yeah. That's true. And I I think we'll probably figure out the space that each of these tools fit in with the sort of modern tech stack. I'm I'm curious to more use cases for like a vector DB and like Chroma itself.
Bdougie:Like, what are you seeing? Because you're seeing lots of folks leveraging Chroma today. So, like, what are the use cases that we're seeing outside of just chat gbt?
Jeff Huber:I mean, the primary the primary use case again is this use case that for the industry to provide is retrieval augmented generation. Okay. So you retrieve data to make better the output of the model. Yeah. And I think that another way of thinking about that again is, in a sense, you are programming model.
Jeff Huber:You're providing data that changes the execution path of that model's output. So that's the primary use case today. And it's kind of again the chat your data use case. You know, we like to say that this is the newspapers on the Internet use case. You know, we are already kind of doing it.
Jeff Huber:It's kind of an obvious thing to do, but it's also very valuable, and it should be done. You know, all newspapers are now on the Internet. Like and, you know, maybe there's still some print publications floating around.
Bdougie:But Yeah. Well, yeah. Some newspaper newspapers didn't survive the the transition. But, yeah, there Yeah. Newspapers you can you can sort of search and find.
Jeff Huber:Yeah. So so it's a rag or retrieval augmented generation, I think, whilst quote easy, also probably makes sense over almost any textual corpus of documents makes sense to exist. Yeah. So that's the first comment. 2nd use case that we're starting to see emerge and I think again, we we feel like we are extremely early here.
Jeff Huber:I mean, we live in a bubble kind of here in San Francisco and this is all the new we've talked about. But, you know, we went to a hackathon even just a week or 2 ago and it was a hackathon of, like, an AI company. We thought, oh, most this will be kind of more bubble whatever. And people were coming up to us and saying, like, so what is what is this technology do and and why would I use it? And it kind of, like, made us step back and reconsider, you know, from either okay.
Jeff Huber:Maybe we're in the first one percent of this to no. Maybe we're, like, this is the first point one percent of this. Yeah. And I think that's, you know, probably true. It's, like, just extremely, extremely early.
Jeff Huber:And so, you know, all the use cases, all the native use cases. So the use cases that could not be done before. Those use cases are obviously really exciting because they could not be done before. You could kind of do check search over documents before. You could kind of able to chat up before.
Jeff Huber:It wasn't as easy. It wasn't as good, but you could still kinda do it. What are the use cases that could not be done before? And I think that the one that, like, people are right that agents are do matter. This idea of, like, this little entity, this little intelligence, which you can, give instructions to, you can give feedback to, it learns by watching what you're doing.
Jeff Huber:You know, hopefully, most state runs in your computer, so it's not too creepy or Orwellian. But that idea of like a little agent, I think is pretty attractive because what it enables is like the personalization, the intelligent personalization of certainly any digital product you interact with and then later probably any physical product you interact with. And so a very kind of trivial example of this, I'll give you 2. So one would be, like, you know, a little agent that sits over your email inbox
Bdougie:and you wanna just give it feedback. Hey. Like, next time I
Jeff Huber:get a calendaring invite from somebody at a sales company, archive the email. You know? Hey. Next time I get a an an, email from, like, an investor, you know, like, market for the like and you wanna be able to get that natural language instruction and then have it respected and have it executed. And you also wanna give it feedback.
Jeff Huber:Hey. Please don't do that again. You know? Yeah. And have it respected.
Jeff Huber:So that's one. I think another more physical use case is, like, you know, again, this is your classic, like, talking to your washing machine thing. Yeah. Like, should we be able to talk to our washing machines or our dishwashers? I don't know.
Jeff Huber:Like, it's it seems like, you know, the Jetsons, it kinda just seems trivial. But I think there is something to it where you can say, you know, say to your washing machine. Again, give it, like, natural language instructions to have it do what you want do next time. You experience this all the time with products, digital or physical, where it just doesn't do quite what you want it to do.
Bdougie:Yeah. Oh, we saw this with the Echo devices. And, like, it could do the thing if it was, was, like, trained to do the thing already, but you couldn't train it.
Jeff Huber:Exactly. So that ability to teach and, you know Yeah. And the ability to teach the model something and then for it to remember it, I think unlocks a lot of really exciting use cases. So, that's that's, you know, one bucket. And then the 3rd bucket is, like, you maybe see the experiments online with, like, agent to agent interactions.
Jeff Huber:People putting, like, virtual worlds and having agents to create kinds of crazy interactions and stuff. I think, yeah, we're still early in figuring out where the utility is on that. Yeah. But this agent to human interaction stuff, again, it's like maybe, like, it's kinda like the movie Her. Maybe if that's scary or bad.
Jeff Huber:I think in reality, it'll be a lot more boring and just useful.
Bdougie:I'd be more scared if it was a movie Megan instead. I
Jeff Huber:haven't seen it. I haven't seen it. Yeah. That's Okay.
Bdougie:It's like her with a physical rep it's pretty it's wild.
Jeff Huber:I'll check it out.
Bdougie:I I think about that movie a lot. Okay. I want to be honest. Maybe, maybe maybe space it out there. But, yeah, the yeah.
Bdougie:I was just I mentioned, like, the Echo devices because I feel like they they missed they missed an opportunity. Like, there was a time where everyone thought we wanted to talk to devices. Right. But now we're in a world where everyone thinks we wanna chat to devices. Mhmm.
Bdougie:But if I could basically say, hey, you know, I eat a baked potato and a piece of chicken breast every day. Like, that's my model. Now, what can I do outside that model? It's like, maybe that's rudimentary, but, like, I I look forward to the world where I can basically pattern match my life through and and memory models and stuff like that.
Jeff Huber:Yeah. I think a lot of the, like, technology waves of the past, whether it be things like augmented reality or maybe more like VR, whether it be certainly sort of like chat interfaces in home speakers. Yeah. Smart Home before that. I think a lot of them were directionally correct and good ideas, but the timing was wrong.
Jeff Huber:Yeah. And I think that it is quite likely that either now or, you know, at least 2 years or 3 years from now, all the base technology will be readily available to really make those useful magical experiences and not frustrating ones. Anybody who's interacting with smart home technology will tell you, no. Just get the light switches. You know, you don't wanna have it to be very fancy.
Jeff Huber:Yeah. But I think that now we can have our have our cake and eat it too. It can both be simple and So Yeah.
Bdougie:So I I wanna your cofounder worked at Facebook previously or or I guess Meta now. Mhmm. Meta launched their llama 2 thing. Microsoft is invested in llama 2 or not even invested in llama 2. They're participating in all these other different models.
Bdougie:Yep. So do we see an acceleration even more so that now we have all these incumbents now participating and providing, like, free computer discounted compute?
Jeff Huber:Mhmm.
Bdougie:I guess the the the real question there behind that question is, like, what do we see next? Yeah.
Jeff Huber:It's a good question. There's a lot of different ways you could go. You know, any of these are speculative at best. Yeah. You know, I think that you should expect to see the community pick up open source commercially friendly models like llama 2 and really squeeze all the juice out of them.
Jeff Huber:That means making them easy to fine tune. It means getting structured outputs out of them. It's already happened. You know, I think that the the the idea of a general purpose reasoning machine, which is outside of the idea of maybe artificial general intelligence or artificial superintelligence, but just like general purpose reasoning Black Box Yeah. Is incredibly exciting.
Jeff Huber:I think that, model distillation will be important. So, the idea is, like, can we take large models that have memorized large corpuses of the Internet or pseudo memorize these things and maybe have some emerge capabilities in terms of their reasoning power? Can we distill out of those large models, small models that maintain all the reasoning power but minimize the bias towards hallucinations or to, like, mix stuff up. Because you've sort of wiped out the weights that were, you know, memorization focused. I think, yeah, if you're an AI researcher, this is not the exactly what's happening under the hood.
Jeff Huber:Yeah. But this idea of distilling out reasoning capability is quite exciting. So distillation is really important as well. Small models. Small models have the benefit too of being cheaper to train, cheaper to run, maybe easy maybe even having better interpretability as well, which interpretability is always a big deal.
Jeff Huber:I think another another bucket of, like, really interesting things happening is, multimodal models. So you saw earlier this year with the GP 4 Vision demo
Bdougie:Yeah.
Jeff Huber:The ability to plug in a picture and understand the real world. And I think that that ability you know, it'll be the first version and it'll get much better very quickly. That ability will likely bring intelligence more into real world context in a way that never has really before. And I think, actually, there was announcement from Google, like DeepMind, Google Robotics even just today or yesterday on a on a similar note. It was sort of like a a vision language transformer model.
Jeff Huber:Again, that kinda can bring may may actually I mean, planning, sensing sensing and planning. Sensing recognition and planning are the biggest problems in robotics largely. There's also a bunch of other, you know, sure like batteries and motor stuff. But, those might be not solved but get much better in very short order. So multimodal is really exciting.
Jeff Huber:Robotics is really exciting. Small models is really exciting. I think that we'll continue to see open source continue to, like, make more and more strides.
Bdougie:Yeah.
Jeff Huber:And it wouldn't surprise me that if we think about the current workloads that go through GB 3.5, GB 4 today, or Cloud 2 or Palm or whatever, it wouldn't surprise me if we see as good performance from open source models in the next year. Yeah. You know, that being said, people always want the the latest and greatest. The latest and greatest will be always a little bit more reliable. It'll be a little bit a little bit or a lot more powerful.
Jeff Huber:And then there's also this question of like, well, there are economies of scale. And if you're running a 100000 GPUs or more, you know, maybe you can actually run these things cheaper than you can even run the open source model yourself. Yeah. And so I think that's another interesting cost curve that people don't talk about enough. It is a real one.
Bdougie:Yeah. It's a yeah. It's a fascinating time too as well. And I now we have, I guess, they were talking about, like, g t b 4 and, like, even, like, g t b 5, whatever that comes out, like, the danger of that. Now we have, like, the federal government now Mhmm.
Bdougie:Creating this sort of cohort of folks to for safety and AI. Yep. What's your take on that? And, like, that speed of acceleration, but also, are we gonna hit a ceiling of governance literally, like, figurative Yeah. Governance in a term that could be both ways.
Jeff Huber:Yeah. I think that, it's not obvious to me that the current trajectory leads to, you know, a singular intelligence which is smarter than all of humanity combined. Yeah. Yeah. I think that people, you know, humans are not good at thinking in exponentials, but humans are also not good at thinking in in s curves.
Jeff Huber:Yeah. And, you know, this is, like, true about COVID, for example. You know, it's true about take take any sort of, like, mass market meme or like panic over the last like 5 years, 10 years, 15 years. It's usually an exponential event that people have adjusted to to some degree, you know, you see these like tweets on Twitter about if you know, one person's printed to 3 people, then a 100,000,000,000 people will have it next week. You know, it's like early in COVID.
Jeff Huber:Of course, it's not a 100,000,000,000 people in the world. And, I think that to some degree, that's what's happening, you know, now as well. Or another another joke here is that if you look at, like, historically, you know, there's the famous figure of Malthus who predicted that the world would run out of food and there'd be mass famines. And then the irony is that if you look at the data, Malthus was right every single year up until Malthus, like, published his book. And then there were innovations in, like, chemistry and and, biology that meant that that was no longer true.
Jeff Huber:Yeah. And we're able to really get to, like, mass mass farming. So, yeah. The p the p doom stuff, which is like
Bdougie:AI will it's by itself, you know, take over and kill all of
Jeff Huber:humanity, I think is, self, you know, take over and kill all of humanity. I think is kind of just a deus ex machina. It's a secular apocalypse. It's probably rooted in the fact that, like, you know, modern society has, you know, rid itself of a lot of religion. But, you know, human beings being human beings being religiously minded, you know, pick up worse religions.
Jeff Huber:Yeah. This is certainly one of them. So that's one comment. I think the second comment is around the the likelihood that bad humans use this technology for evil. Yeah.
Jeff Huber:So it's not that AI itself is gonna take over take it over, take over the world, but but maybe a bad human picks it up and does something evil with it. I think most of the sophisticated people will tell you that that is a much more real and present danger. You know, and you see these graphs in the New York Times where they compare language models to nuclear bombs, which are probably this came out yesterday, which was like kind of ludicrous. But, but yeah. I mean, it we we don't know.
Jeff Huber:We don't know yet.
Bdougie:Yeah.
Jeff Huber:We don't know yet what this kind of technology put into the hands of people that have evil intent, like, what it will mean. And then we also don't know what the cat and mouse game will look like either. Yeah. You know, there are plenty of bad people out there that have plenty of tools today to do really bad things. Yeah.
Jeff Huber:And, you know, bad things do happen. But on the on the net net, people are pretty happy to live in this this millennia versus previous millennia. And so, my overall bet will be that that trend continues.
Bdougie:But Yeah. Well, speaking of good tools, trychroma.com. That's your URL.
Jeff Huber:Yeah. T r ychroma.com.
Bdougie:Yeah. Yeah. Yeah. So folks, if you're interested in checking out this technology, we have, like, 100 of thousands of impressions soon. Like, by the time this video is, like, hits that, it'll be, like, of course, we've got all those folks.
Bdougie:So, like, and subscribe. Thanks for the time, chatting about Chroma. And, folks, stay saucy. The secret sauce of the podcast produced in house by Open Sauced, the open source intelligence platform to provide an insight by the slice. If you're in San Francisco and interested in being a guest on the show, find us on Twitter at saucedopen.
Bdougie:And don't forget to check out Open Sauced at opensaucedot pizza.