Subscribe
Share
Share
Embed
Come join us as we discuss everything open source with guests that are pillars in the industry. Welcome to The Secret Sauce.
Welcome back to the show. We're about to learn the secret sauce. Zeke wanted to chat about your open source journey because you've actually pretty prolific. And I don't know if you've, like, considered yourself behind the scenes, in front of the scene, but, like, you've done a lot of stuff. So, why don't we start with, like, who is Zeke first?
Zeke:Who is Zeke? Wow. Okay. Well, in the context of what we're talking about here
Brian:Yeah.
Zeke:You know, of course, in my regular life, I'm a dad and I have a family and I live in Berkeley. But for work, I mostly work in AI these days, doing this startup called replicate. But yeah, I've been working in, open source software development for a long time. I've been doing web stuff since, you know, the early nineties. I just watched that, Kelsey Hightower Yeah.
Zeke:Kubernetes talk that you did last month, and had a lot of parallels there in terms of like, not going to college, but being really interested in computers from a young age and figuring out how to, make web apps, make web web pages and get paid for it.
Brian:Yeah.
Zeke:So I've just had this sort of like long career of just climbing higher and higher and finding more and more interesting opportunities. And I think part of it is just that I wanna be creative. And when I find barriers to creativity, I wanna kind of break them down or simplify things. So I know a lot of people that you have on your show are like, you know, really talented software engineers. I'm not actually a very good engineer.
Brian:Oh, interesting.
Zeke:Yeah. I'm not good at solving problems. I'm not good at fixing bugs. But I am good at breaking things down and making them simple. And so, the last 10, 12 years of my career have been really focused on that.
Brian:Yeah. Yeah. I mean, that's a good way to put it too as well. And, like yeah. Because I I look at the stuff that you've done and I'm like, oh, man.
Brian:You've, like, touched so many different things. And I've actually looked at your PRs before in the past and be like, okay. I'm gonna learn from this. Because, again, like, I've I've seen your work in Electron, most recently. Some stuff internally at GitHub, like, you'd helped build out, like, that that you built out the docs stuff.
Brian:Right?
Zeke:Yeah. I helped internationalize the docs and open source the whole docs site for GitHub.
Brian:Yep. Yep. And, Yeah. So I know you're from most recently from there, but, like, let's actually take a tour down memory lane and start from the beginning.
Zeke:So,
Brian:like, did you like, what was the interest in eventually, like, learning the code?
Zeke:I think in the very beginning, it was like, there was this product called Macromedia Flash in the late nineties. Action script. Action script. Yeah. So Flash was like this program where you had a timeline and you had a stage and it was sort you sort of like Photoshop in the sense that you could put things on different layers but in then in addition to the layered aspect you had this time element where you could, animate or transition things on different layers.
Zeke:And so that was kind of a that was a visual that was a GUI that you could use to create, you know, animations and games and things like that. But then I realized that you could actually write scripts to do that same thing which and when you when you take something that's kind of a manual process and turn it into something that's managed by a computer program or script, you have so much more flexibility. So I spent a lot of time just making crazy fun animations with with ActionScript in the late nineties, early 2000. And that ActionScript was part of ECMAScript which is the same sort of language family that JavaScript is a part of. So even though I wasn't really doing any legitimate browser coding at the time, I was sort of getting the foundation for being able to write code that would ultimately work in browsers.
Zeke:So I was kind of a scrappy developer for a long time, did PHP and MySQL and Windows and I built my first like shopping cart application in the early 2000. And I wasn't very good at it and it probably had a lot of, like, security holes. And I think we were even storing credit card data in a way that was, like, you know, not PCI compliant and stuff, but learning a lot along the way. And I worked for a bunch of like, branding agencies and so I got to be exposed to different kinds of like design, like industrial design and print design, graphic design, stuff like that, which I'm still not especially good at, but I love I love design and I love love being part of teams that do interesting design. And I also really love language.
Zeke:So, sometime in, I think, right around 2,009 to 2,010, I watched this TED talk by a woman named Erin McKean who was the head of the Oxford University Press in the United States. So she was like the Oxford English Dictionary, boss in the US. And she gave this TED talk about how she was so frustrated with, the state of dictionaries and how, you know, the web is just exploding and that's where all the real sort of like valuable language information is. But then we have these like crusty old books that small teams of people maintain that are supposed to be like the sort of canonical representation of what language is. And she's so she she was kind of bugged by this.
Zeke:And there was, someone in the audience who, was, sort of a, you know, prolific investor, early investor in Facebook and was like, maybe you should just turn this into a startup. So she moved to the Bay Area from Chicago and started this company called Wordnik. And the goal of Wordnik was to build APIs around English language and build the world's largest English dictionary. And so she had this concept of what she called a free range definition, which was, it was the idea that, when you learn words, you learn them from other people saying them. You don't learn them from opening up a dictionary and reading the definition and being like, oh, okay.
Zeke:Now I get it. You hear people saying it.
Brian:It's like contacts.
Zeke:Yeah. And you start to absorb it. So a free range definition is basically a word out there in the wild somewhere on the Internet that has enough context around it to define the word. So she hired all these, you know, natural language people, linguistics people with, computer science backgrounds to start harvesting these free range definitions from the Internet and using them to compile this big data thing. And at that time, I was really interested in, Ruby on Rails and I was kinda starting to level up to being like a legitimate software developer, like writing tests and, you know, being able to write scalable applications and stuff.
Zeke:But I was so pumped about the idea of working for for Wordnik. So I moved to the Bay Area. I moved to San Francisco on January 1, 2011 for my first venture capital funded startup job. And that was really when it started to kind of like take off for me and I started to make a lot of connections and do some really interesting work. Ultimately Wordnik was not a successful business.
Zeke:The idea behind the business was to make APIs that could be used by game developers or educators yeah. To incorporate word data into their apps. But the problem was they're just not people just don't expect to have to pay for that sort of thing. Like, there are other sources for finding that kind of thing. So, we didn't have a great business model.
Zeke:But one thing that we did do well is we built an internal open source project called Swagger. And Swagger was basically a JSON file or a YAML file that we use to describe the APIs that we had internally. So we had one team that did, like, they maintained the API and they wrote everything in Java and Scala and they maintained the MongoDB database. And then we had the web team and they wrote everything in Ruby on Rails and they were more interested in JavaScript. And there was some really, like, cultural differences between those teams.
Brian:It
Zeke:was like the XML team and the JSON team, like Yeah. Butting heads. So it was hard to get the APIs we wanted because the API team didn't necessarily speak our language. So we created Swagger as this way of having, like, a shared contract for, like, here's the API I want before we even did any development. Just does this look like what I what you something you can build and something that I can consume.
Zeke:And it became a a contract that we could use to to design before actually building. But we didn't actually know what Swagger was gonna turn into. So now Swagger is called open API, and there's a bunch of, like, you know, politics around why that got renamed. Basically, some company tried to,
Brian:you know Like, own the the trademark. Own it.
Zeke:And everyone was like, well, no. It's actually just like this open source thing. So we're gonna fork it and call it open API, which I think is awesome.
Brian:So it's a good name, though.
Zeke:Yeah. So the the reason the name came from there was a thing called from Microsoft called the web application description language, which was like an XML thing. So people call it Waddle. So the idea was my idea was like, why why Waddle when you could swagger?
Brian:Okay. Did you name Swagger? I did. Oh, wow. Okay.
Zeke:Yeah. You
Brian:heard it here first. Well, I don't know. I honestly had no idea, but, yeah, I've been using Swagger open API for, like, a very long time. So cool.
Zeke:Yeah. We had no idea that it was gonna turn into what it what it is now. But, so my boss, Tony Tam, he wrote the original spec, which was, like, this very long document with all the, like, capitalized musts and shoulds and all that sort of stuff. Like, the legalese of, like, how to really use this thing. I think I wrote the 1st JSON schema and then I also wrote the, the swagger UI which was like this thing where, it was like, oh, well, we have a bunch of JSON data.
Zeke:We can just render it on a page and see what it looks like. And then the next and so then we had documentation for our API. And then it was like, actually, well, we know all the data types because we just wrote them into this JSON file. So then if it's like, you know, a multi option list, then, you know, we can turn that into a dropdown. And we can actually turn the thing into a form where you can run the API right in the browser where you put in your API key or you authenticate or whatever, and then you're testing the API without having to write any code.
Zeke:Yeah. And I was like, it was like an accident, you know, just kinda, like, happening.
Brian:It it's wild because, like but I I I mentioned we started this podcast before we started shipping any code for open sauce.
Zeke:Yeah.
Brian:And, at that point, we've actually built an API first for open sauce, and we built it in open a open API.
Zeke:Yeah.
Brian:I I'm Freudian slip nonsense
Zeke:on that. To not say open AI or open API.
Brian:But, yeah, API dot opensauce dot pizza is swagger. And, you can go test and get endpoints and data and see our future features before we get UI on it
Zeke:Yeah.
Brian:Today, which is a lot of my conversations. They're like, hey. Here's an endpoint. Or here's the API, like, linked directly to the form. Just authenticate, and you're good.
Zeke:Yeah. It's amazing because, like, if you actually just hand over an open API schema to developers, that's actually often what they want. It's just like, I don't want all this, like, ceremony around what your API can do and all this documentation and tutorials and stuff. Like, those things are good, but sometimes you just want the raw reference data. And the open API schema is, like, the closest thing you can get to just, like, this is what this API does.
Zeke:Like and, later that became a project that I worked on at GitHub, but that was that was many years later.
Brian:Excuse me. But yeah. Yeah. Last thing. So, like, quick like, take a step back.
Brian:Like, so Aaron started word wordnik. It didn't work out. Yeah. But what's interesting is, like, that natural language processing is the thing that's, like, it's common knowledge and folks are doing it now. So I was curious, like, did Aaron go back to the well or anybody ever solve that problem?
Zeke:Yeah. So, one of the really we used to have these, like, sort of hack day 20% time kind of things where we just do, like, experimental stuff at Wordnik. And one of the people who worked with us, named Will Fitzgerald who, came to us from powerset which was this very interesting search company that was it was basically the predecessor to Bing, like Microsoft bought them and then they became Bing. They had this, like, I think it was a it might have been a Google engrams thing. It might have Bing.
Zeke:I can't quite remember. But they made a story generator which used this concept of engrams. So, a bigram is a pair of words, 2 words together. A trigram is 3 words together. I know if there's a word for a tet tetragram maybe for 4, pentagram, I guess.
Zeke:Although that means something else usually. Anyway, arbitrary number of words paired together. So you can imagine if you're scouring all of the text that's ever been written by humanity, you could start to collect these pairs and these triplets. So when you're doing, like, a Google search and you start typing something and you get the autocomplete, it's telling you suggesting the next
Brian:word that's based on the statistical likelihood of that word following the words that
Zeke:preceded it. Yeah. So likelihood of that word following the words that preceded it. Yeah. So, this guy, Will, made the story generator
Brian:where he gave it he would
Zeke:give it the first four words of the story or the first five, and then it would just ramble based on that moving one token over at a one word over at a time and generating the next word. And so it didn't make any sense, but it read like one long continuous sentence because all the previous four words for every word always made sense. Yeah. So it didn't go anywhere. It didn't it was kind of meaningless.
Zeke:It was just an experiment, but I found it really fascinating. And now we are here we are 12 years later and we can dig into this, but we have this thing called, you know, we have these language models and most of them are based on this idea, this thing called transformers and credit to the 33 brown one blue YouTube series. I think that's what it's called. Yeah. It's got the little funny animated pie character.
Zeke:They just did a, a video about how transformers work that is so helpful because I've been trying to figure out how this actually works for a long time and I'm starting to understand it. But the gist is, the way language model works is it's doing that same thing. Like, based on all the data that I've consumed, the most likely word next is and there's a, you know, there's, like, a list of probable of, suggestion of possible, words. But the thing that is crazy about transformers is that it is also looking at the surrounding previous words in the context to know what version of that word you're talking about or contextually what is the most relevant word. So it's the engram thing plus contextual awareness.
Zeke:Yeah. And that's basically, like, how language models work. And, like, that's the sort of, like, fundamental piece of all of this, like, really amazing innovation that we're seeing in AI right now. So for me, it's really cool to see, like, the, like, really rudimentary version of it that was happening 12 years ago that was, like, state of the art. And that was a 5 word context window.
Zeke:And now we have, like, table stakes is, like, 200,000 word context window, and then Google Gemini has a 1,000,000,000 or whatever. So Yeah. And I am getting off on a tangent here, but, like, thinking about the sort of arc of my career, like, it's pretty interesting to see these things come back around after all these years.
Brian:Yeah. And it's like it it sounded like because, like, we'll we'll get to other things as well, but, like, you've kind of been seeing the earliest, like like, action script, echo script. You also got to Node. So, like, you you mentioned Ruby on Rails.
Zeke:Yeah.
Brian:But I know you I know you from the Node community. So, like, how did you get from okay. Now we have the swagger stuff. Yeah. Like, how'd you get the Node ecosystem then?
Zeke:Yeah. So, JavaScript's been around since the nineties. There was, like, a brief time where Microsoft made a sort of competitor to JavaScript called Live Script, and there was a sort of tenuous moment where we didn't know what's gonna happen and people had to write 2 different versions of the front end code to make their web app work. We didn't call it web apps in those days. Yeah.
Zeke:But, like, then, like, Netscape kind of folded and they managed to, like, you know, as they were going down in flames, they managed to, like, set up this new protected entity called Firefox, which was this, like, new open source browser that was, like, gonna come out of the ashes of Phoenix or whatever it was called. That was Yeah. Netscape, the, like, engine. I forget all the terminology, but Firefox sort of, like, crystallized this thing of, like, I don't know. Maybe it was, like, 2,005 or something.
Zeke:They it kind of crystallized the idea that JavaScript was gonna be, like, here to stay and then it was gonna be powerful and useful. But that was still years before Node. So, I was doing Ruby on Rails development in the, like, 2,007, 2008, 2009. And, of course, if you're building a web application, of course, you have the back end, but if you want anything to happen inside the web browser, you have to write some JavaScript. And so when Node came out, I remember having this feeling of, like, oh, that's, like, what the cool kids are doing.
Zeke:And I'm still over here doing Ruby on Rails stuff, and I'm kind of missing out. Although it was still really early. It was like that was like node 0.4. Yeah. And it was this kind of like this infantile project.
Zeke:But I could see the promise of it because it was an opportunity to write code that would have shared logic between a server and a web browser, which is, like, great. Now we don't have to have, like, the kind of back to, like, the swagger wordnik problem. We don't have to have, like, the team that owns the API and the team that owns the front end because we can write the code in the same application using the same, you know, in the same language using the same tools. So to me, it was, like, obviously, everyone's gonna go in this direction. But then I went to work at Heroku and, I sort of joined Heroku out of spite because at Wordnik, we didn't have, the opportunity to deploy production services unless they went through the, like, Java they had to be part of this, like, Java monolith, basically, and I didn't want to write Java.
Zeke:I didn't wanna figure out how to write Java. I wanted to write apps in whatever language I wanted to use and deploy them on Heroku. But they at the time Heroku was sort of seen as, like, a toy or I mean, it still kind of is seen as a toy by some people, but Salesforce. Prototyping. Yeah.
Zeke:So I was so interested in Heroku and how they made it really easy for sort of not very good web developers like me to be able to build an application, easily deploy it, scale it, connect databases and logging services and email sending services and stuff to it with, like, very minimal configuration. So I went to Heroku kind of, like, with this feeling of like, I actually don't like doing this stuff. I'm not interested in like I'm not like a systems developer, but I just love how great this product is and how it's made me able to be like a full stack developer, even though previously I was just making web pages. So I assumed that when I got to Heroku, all those Ruby developers would be like, yeah, Ruby is actually not that cool anymore. We're gonna start doing everything node.
Zeke:But that was not the case at all. It was either everybody was still loving Ruby because it had this beautiful syntax and was just so, you know, such a humane programming language to look at with your eyes and interpret. But everybody was moving to go because go was like this really fast, safely typed, easy to test, hard to break programming language that could, you know, outperform. You know, it was like kind of Ruby and Go were sort of on the opposite ends of the spectrums in in terms of their, like, performance capabilities. Like, Go is really, really fast and Ruby is really slow.
Zeke:So all these people who were working at Heroku building the actual foundation of Heroku were really stoked about Go. And I was like, okay. Go is cool, but I don't wanna write that because I wanna write web application. I wanna build web apps. So, Heroku initially was a Ruby platform, but they had started to pivot into being a polyglot platform, meaning that you could run Yeah.
Zeke:Any kind of code out there. So there were like these things where if you push if you did, like, git push Heroku master and your code had a package JSON file in it, I would be like, oh, that's that looks like a node app. That's not actually a rails app. So let's send it to this other build process. And so those things were called build build packs, and those were open source.
Zeke:So I was like, well, we had this node build pack that this really talented developer David Dollar had built as like a prototype, but it wasn't maintained and there wasn't any team maintaining it. So even though I wasn't good at that stuff, I was like, this is a huge opportunity because, like, Node is blowing up. So I became the maintainer of the Node. Js build pack at Heroku. So that meant I devoted, like, a year of my life to getting good at bash scripts, basically.
Brian:Yeah. Not web apps.
Zeke:Not web apps. No. Like, I keep getting kind of farther and farther from my web apps, which is interesting. But, that got me really connected to the Node community. So we had, like and it was one of the most sort of, like, fulfilling times in my career because I was working on a project that, like, people internally didn't care about or weren't really precious about because it was still, like, Ruby was still, like, dominating.
Zeke:But I could see the writing on the wall and it was, like, we had something like, I don't know, 5,000 deploys an hour of node apps on Heroku. And so I could change one thing about the build process, and then all this data would pour in and I could see how much I was, like, slimming down the build times. And then thinking cumulatively, like, wow. Okay. So that's like 5,000 deploys an hour.
Zeke:If I, like, save if I shave, like, a minute or 2 off of each of these builds, it's like that's adding up to a lot of developer time, like a lot of time that is no longer wasted. So I felt like I had this huge lever for being able to, like, really improve developers' lives. And that was going really well for me. And I had this kind of like private little niche at Heroku where nobody was I was like the node guy, which was kind of a problem too because I'm like, well, I'm actually not very good at this stuff. I just think it's cool.
Zeke:You know? Like, I'm not an expert. But meanwhile, the Node and NPM ecosystem were kind of falling apart because Joyant had not been a very particularly organized steward of the project.
Brian:And This was the IO. Js time?
Zeke:Yeah. Exactly. Yeah. Well, just before that. So so the NPM registry was maintained by a company called Nojitsu.
Brian:Yeah.
Zeke:And they were based in New York, and they had this kind of they were trying their hardest to keep the registry alive, and it was filling up with lots and lots of, NPM packages. And the traffic was just getting out of hand. And they had this sort of bespoke database configuration where they had this database called CouchDB, which was a really sort of interesting project that was kind of very web forward. But it was a little bit like a house of cards in terms of the way the registry was designed. And the the the registry was going getting so insanely popular because of the ease of using NPM and publishing packages with NPM, that something needed to happen.
Zeke:So Isaac Schluter, who was the, creator of NPM, turned it into a company. He raised he raised capital and started NPM Incorporated in Oakland. This was in, like, 2014, I believe. And I was like, okay. Well, that's the place to be because, like, Node is the that, you know, at that time, I just felt like NPM and Node are sort of like at the epicenter of web developments, like, where it's all happening.
Zeke:Everybody's typing NPM blah blah blah all day long every day. So even though I had this really good thing going at Heroku, I just felt like I have to chase that opportunity and see where see where that leads. And ultimately, working at NPM was not a great experience for me.
Brian:It was
Zeke:tough because, again, it was sort of like the wordnik thing and that there was no business model. Like, the idea was, okay, we'll sell private packages. But GitHub had had already kind of people had already kind of solved the problem of creating NPM private packages by just putting their code on GitHub. Like, still to this day, you can you can type npm install p dougie slash pizza, and that's just gonna, like, go straight to Git and install that as if it were an NPM package right from your GitHub repository. So that's always been part of NPM.
Zeke:And so when people wanted to use a private package, they would just point it at a repo or a branch in a repo where the build of their stuff was. And as long as they have a secret configured so that, you know, the NPM and get client can authenticate to that, then you don't need private repos. So the whole, like, basis of this is how we're gonna make money didn't actually make a ton of sense. Meanwhile, usage of NPM is going through the roof, and it just didn't make a ton of sense. And actually, I wanna talk there's like a there's a core there's a corollary here, I think, with Hugging Face
Brian:that we can get to at some point. But Yeah. Yeah. It's it's interesting because I just ran into Isaac yesterday at an event in SF, and we're gonna have him on the podcast as well. They talk about that journey and the story because now they're working on Volt, you know, which is a a whole different approach to problem with Darcy Clark.
Brian:So and I think, Roy and a few other NPM folks are there. So I don't know how much if you overlap the the the sort of I was say, like, the folks who were the last torch bearers of NPM Right. They all have now started Volt. And, it's it's a fascinating experience, but, also, I lived through it. Yeah.
Brian:And, I also joined Netlify a couple years after that where I remember the Heroku build pack. Like, for for Node, like, it worked at the time when I was, like, building, like, Node apps and eventually React. But then it didn't, like, quite need like, you kinda did too much for, like, a React app.
Zeke:Uh-huh.
Brian:So then Netlify started. And Netlify and Vercel, Zee, and a bunch of other stuff like, Surge, all built, like, these build tool agnostic deployment platforms which, like, Kapolei, like, shifted course away from being the run build packs on on at least for node apps on Heroku.
Zeke:Yeah. Totally. I mean, it was funny at Heroku, there were so many static web apps that people were deploying on Heroku. And it was like so we're running a server just to, like, host this, like, rinky dink web page. Or we could actually just be, you know, putting it on a CDN somewhere and Yep.
Zeke:Pointing a, you know, Apache editor or something like that and not having to actually run any servers for this.
Brian:But at that time was that did you leave post, Salesforce acquisition?
Zeke:No. I actually started after the acquisition. Oh, you
Brian:started after the acquisition?
Zeke:Yeah. So I was there during a time when everyone was sort of in this celebratory, now we don't have to do anything kind of mode. Yeah. Like, there was there was still no management. There were there was just like a flat organization even though there were, like,
Brian:12 How big was the company then?
Zeke:100 people or something. 100. Oh, okay.
Brian:I think
Zeke:I was, like, 85th person, 86th or something like that.
Brian:Wow. So, yeah, at that point because I know post Salesforce acquisition, things kinda felt like it hit, like, a wall when it came to Heroku. So very interesting timing. But yeah. Anyway, I wanted to share the context of my life during that time.
Brian:Totally. But you mentioned Hugging Face. So, like, the Hugging Face has well, you wanna explain Hugging Face? You don't have to technically explain how it works. But yeah.
Zeke:Yeah. So Hugging Face has kind of become the de facto place where people store model weights when building AI models. So weights are kind of like the, you could think of them as this sort of artifact that's generated from the training process when you create a model and train a model. The result is, like, this big pile of numbers in a file that is ultimately the binary that you're using when you run a machine learning model or an AI model. And so Hugging Face is kind of like, on the surface, it looks a little bit like GitHub, but the collaboration doesn't really happen Yeah.
Zeke:On hugging face. It's more like people are still writing their code on GitHub and then pushing the results of their model training process to Hugging Face as weights files. So in a way, even though they have, you know, they have, like, an inference platform so you can actually run code on Hugging Face, It hasn't really taken off to the degree that it has just become the de facto sort of, like, registry in a way for AI models. So, what ultimately happened to NPM is, like, Microsoft bought NPM as, you know, like a charity move because everybody needs it, but nobody expects to ever pay for it. It's just like we all need it to build web applications.
Zeke:We need it to be alive. We need it to be able to scale, but we don't have any expectation that it's ever gonna be a profitable piece of the puzzle. And I think hugging faces is sort of in a similar position in that, like, people have come to expect it as like, oh, that's where you put your model weights. And when I wanna build my model and I wanna deploy it somewhere, I wanna pull the weights down for Hugging Face, and I'm gonna put it in my own container somewhere and run it.
Brian:Yeah.
Zeke:So, I'm not I'm not an active Hugging Face user, but I see how much people are confused by it, because it has so many bells and whistles and so many different sort of, like, capabilities. It's hard to know exactly what the offering is. Like, people are like, what is it?
Brian:Yeah. Yeah. I mean, I I look at it as, like, that is, like, where you similar push to Heroku and, like, it's it's hosted somewhere. And then they do have, like, a bit of a chat interaction now that they shipped the, well, last summer or maybe they came out of beta Yeah. Recently.
Brian:But, yeah, I'm just honestly, I I use it mainly to find out, okay, is there something else that's out there that I could add to what I'm trying to build?
Zeke:Yeah. I mean, you can kinda use it to get a sense of, like, the pulse of things and, like, what where, you know, where the action is. But then when it comes time to run that code
Brian:Yeah.
Zeke:You can do it on Hugging Face, but most people are finding alternative ways to do it. Yeah. So, of course, I'm biased because I work for a company that
Brian:Yeah. That replicate
Zeke:lets you run AI models in the cloud and containers. So
Brian:Yeah. And I I I find that approach really nice because I I get that approach because I can just use the npm package to point to a model and start throwing stuff at that and getting the context in there and then presenting the results in the context of my little ricky dicky side project.
Zeke:Yeah. Totally. So, yeah, kind of fast forwarding a little bit. I I went to work at GitHub for a while, did some really interesting things there. Met you, of course.
Zeke:Yeah. Had a good time. But in the interest of time, I wanna kinda jump forward to, how I ended up working at replicate and the connection there. So, I was at GitHub working on internationalization stuff. So we worked on docs.gidhub.com, and it's now in a bunch of different languages.
Zeke:And there's a whole pipeline for translators participating in that process, which was super hard and super interesting and took, like, two and a half years to get it kinda dialed. But, of course, you know, Microsoft purchased GitHub and for a long time, things were still fun for me. But after after some time, more and more of the sort of, like, the agenda or the priority was around moving all of GitHub's services to Azure off of AWS or GCP or wherever else I experienced some of that when I was still there. So that
Brian:was less interesting for me, and
Zeke:I was sort of open to new opportunities. And then, Ben Firschman, the CEO of Replicate, approached me, and I think he just knew of me from the open source world and, started telling me about this idea that he and his friend Andreas were working on. And it was like they wanted to build GitHub for machine learning. And I was kinda like, well, what it like, what is that? What does that even mean?
Zeke:And I was skeptical because I was kinda like, I don't really know what AI is. This was like 2 this was actually spring, like, this time of 2021, so 3 years ago. But I could tell these guys were really interesting and really smart. So it was kinda like, okay. Well, also they've done some pretty cool stuff in the past.
Zeke:Like, Ben wrote this thing called the command line, interface guidelines, which is like this really cool long tome about how to build CLIs, just with all the sort of like old school 19 seventies UNIX philosophy of standard in and standard out and all that kind of stuff. Small sharp tools, again, connecting back to Heroku. So it's kinda like, okay, this guy's a kindred spirit. And they see that machine learning is like this big messy world where it's starting to show some real potential. Deep learning is becoming this thing and people are like, wow, this is these are these these models can do some really interesting things.
Zeke:However, the process of building them and distributing them and running them is really hard. And so we spent, like, the first part of my time working there talking to a lot of, researchers who are, like, you know, academics working inside of universities or working for, you know, big companies like Google or Facebook who are trying to build machine learning capabilities, build models, and then share them and being kind of stuck because they're SSH ed into some university cluster that can only run the model in a certain way. But, like, they don't know how to build a web front end to, like, send it to their adviser to be like, click on these buttons and then you'll be able to see what my cool model does. Yeah. So it was kind of, like, similar to the Heroku thing of, like, well, you're a machine learning engineer, but, like, that doesn't mean that you can actually show your work to anyone.
Zeke:Yeah. So the idea of replicate was about literally giving people a way to replicate their work, to, like, take someone else, some other academic's work and being being able to reproduce their results without having to read their PDF academic paper and try to reverse engineer it or look at their GitHub repository with some janky old code that has, like, no read me and no tests and try to turn that into something that actually produces results. So we wanted to solve that problem. And it's very connected to so Ben often talks about this idea of, like, what if we could do the same thing with AI that NPM did for, like, distributing and packaging software for running in Node or in the browser? Like, if you could just, like, you know, NPM install a vision model and all of a sudden now you have this model that you can, like, give it a picture and ask questions about that picture and it could just answer them.
Zeke:And you don't have to know how it works. You don't have to know how to install it or how to run it. Yeah. So that's kind of the vision. So right now, replicate is like a, it's a cloud hosted company.
Zeke:So we run, public API models public models that have their own API as well as private models. And then there's an open source component to it. So there's a tool called cog. And cog lets you, define a simple Python API around your machine learning model and it wraps it up in a container, like a Docker container with a standardized, HTTP web service and a Redis web service. So every model that you package up in cog, you can put it in a container, run it on your own infrastructure, or run it on cog's cloud or on Replicate's cloud knowing that it's going to have the same API interface as any other model on the platform.
Brian:Yeah. That's amazing. Yeah. I wish we we jumped on a call and chatted about this last year when I was playing with it, because it makes so much more sense than, like, only the last, like, 100 and 20 seconds you explained this.
Zeke:So Okay. Yeah. We need to work on our docs and our messaging for sure.
Brian:Yeah. Yeah. Yeah. Let me know if you need any help with that. I've got experience.
Brian:Nice. But yeah. I mean, this is yeah. I mean, your your journey is, like, full circle. Like, you've kinda touched a lot of different things, seen a lot of different inflection points.
Brian:And even, like, you've seen, like, trends. Like, I know you were a skeptic on this, AI thing, but you're you're here now.
Zeke:Not anymore.
Brian:But even, like, the note thing, the the go thing, like, you've you've seen a different inflection point. So I was curious, like, what's your secret sauce for, like, making decisions, for, like, tech stacks and technology? Like, if you could concise that into, like, a nice sound bite.
Zeke:I've become less concerned with the tech the underlying technology or the underlying programming language because it's all getting more and more abstracted over time. So, when I went to work at replicate, I told Ben, I was like, dude, I don't know how to write Python. He was like, that doesn't matter. I don't know Docker. That doesn't matter.
Zeke:I don't know Django. It's not really a problem. This I don't know if I'm necessarily answering your question, but, like, I feel like over time when I was younger, I used to have I used to feel this sort of like fierce devotion to a certain tool or a certain programming language like Ruby in particular. Ruby people were just like zealots. Right?
Zeke:Like, there's no other way to write code except for Ruby because it's so beautiful. But over time you realize that there's like a time and a place and the context is important for you deciding what tool you wanna use. So I know you had Evan Yu.
Brian:Yeah. We're here talking about this. Yeah.
Zeke:And it was like you were talking about, you know, veet and bun and all this stuff. And I'm like, I've heard of those things, but I don't even know what they are. You know, like, the the the landscape moves so fast that, I would actually be kind of in trouble if I had to make a decision right now about what, what, like, stack to use or what frameworks to use, like, in the JavaScript world. Of course, we do a lot of Next JS stuff at replicate. We have, like, you know, sort of kindred spirits with, Vercel in terms of, like, trying to build a platform that just makes it really slick and really easy to build stuff.
Zeke:And of course, people at Vercel, they had a DevRel team who used to build all kinds of really interesting I think that's how you found out about it. Yeah.
Brian:That's how you discovered Replicate, which is like a good like, their AI SDK. Yeah. Like, a lot of that those ideas came from that, I guess, season that was happening when they were just building stuff every week. Yeah. And, yeah.
Brian:I mean, it I I think it's actually a really good way to, like, take a step back and say, like, it's not about, like, you know, these teams that you have to be on. Yeah. There's, like, that one meme of the curve of, like, it it depends. It depends. But in the middle, it's like, I don't know what to choose.
Zeke:Right.
Brian:And, but, eventually, it's just like, yeah. Yeah. It doesn't matter. It's like, as long as you're building cool things Yeah. And they work in production
Zeke:Yeah.
Brian:I think that's really the goal.
Zeke:One of the things I really like to do is, so I use language models to write code a lot now. I know some people are like, I'm still better at writing code, but I'm like, no, this thing's definitely better than me. But one of the things I like to do is just sort of take a gamble and say, write me a script to do x. And I don't know what language this is gonna pick. Like, it's maybe it's gonna pick Python, maybe it's gonna pick JavaScript.
Zeke:It's probably gonna be one of those 2. It might be bash. But I just sort of roll the dice and let the language model decide what programming language it is. Of course, there are certain situations where there's already something I'm working on and I'm like, this is a Python situation. Let's talk about this.
Zeke:But I I find it fun to just be like, roll the dice, see what the language model gods have to say about the best way to write a script to do x, you know, take all the screenshots on my machine and, you know, put them in a directory or, you know, turn them into a movie or something like that.
Brian:So Yeah. I mean, and and it's getting better.
Zeke:Yeah.
Brian:Yeah. It's it's getting better really quickly. So we'll we'll see what g t p, GPT, 5 is gonna give us. But, yeah, the meta stuff, the llama stuff is, like,
Zeke:looking more together. 3 is really, really good. Yeah. I know there's, like this is really, like, a contentious topic, but, you know, there's, Andre Karpathy, who was the head of AI at Tesla and
Brian:then went
Zeke:to OpenAI a p AI. See, I'm doing it now. OpenAPI. OpenAI a couple times back and forth. He has this pinned tweet which says, you know, the hottest new programming languages is English.
Zeke:And I totally subscribe to that thinking. It's like, a lot of my work now involves starting with writing a markdown file that outlines my intentions for building something. And the more clear I am in describing the thing, the more likely a language model is to be able to produce exactly the result that I want. And a lot of people are using these sort of, like, chat based interactions where you ask a language model to write you some code and you say, oh, no. Can you, like, do this a little differently?
Zeke:And then, you know, it's like this back and forth that goes on and on. And ultimately, sometimes the language model starts to forget things about the earlier part of the conversation because the context window is too short. So they become forgetful. So I have kind of a different approach to building those things, which is I write the read me file. I write the markdown file.
Zeke:I run it through the language model. I see what it produces. If it's not what I like, I don't then have a chat with it. I go back to the markdown file and change it to make sure that it doesn't make that same mistake that it made the first time. And I just end up with a document that is ultimately more and more clear about what my expectations are.
Zeke:And then I run that and it generates code. And from there, it's like an application that I'm that I'm using. But it's just getting better and better at that.
Brian:Like Yeah. You might have described the get, GitHub Copilot workspace.
Zeke:Yeah. Totally.
Brian:It's that's basically what what they're what they're shipping now, but, like, not not meant to be a plug for for that, but try it if you get access.
Zeke:Yeah. I did get access. I got a little bit overwhelmed when I saw the UI because it has sort of, like, this idea of, like, it's, like, strategize, plan, execute, something like that.
Brian:It was
Zeke:a little bit, like It's very JIRA. Yeah. It was a little bit, like, I don't know what this means. I get the gist of it, but, haven't yet jumped jumped in and actually give it a try. Cool.
Brian:Yeah. Well, I'm still waiting for my access. But speaking of which, yeah, let's let's wind this down. Appreciate you coming up, all the way from Berkeley into Oakland. And, folks, stay saucy.
Brian:Secret sauce of the podcast produced in house by Open Sauced, the open source intelligence platform providing insights by the slice. If you're in San Francisco and interested in being a guest on the show, find us on Twitter at saucedopen. And don't forget to check out Open Sauced at opensaucedot pizza.