Michael Sharkey (00:00:00):
Mark Zuckerberg went from reptilian overlord conspirator to down to earth jiujitsu champion releasing opensource software as the big tech villa villain redemption arc. Nobody saw coming Chris to open ai. Just announced some pretty major updates to their a API including function, calling in the chat, completions a api. And for those that don't know anything about this, we'll do our best to explain it to you in terms that you can fully understand. But they announced this function, calling in the chat completions a api, lower prices on G P T 3.5. They're saying, uh, a 25% cost reduction on that model. There is also lower prices on the embeddings model, 75% cost reduction on the embeddings model. They're saying the versions of G P T four and GB T 3.5 turbo are now more steerable as well. And I think the big, the big thing that excited me outside of the function calling, which we'll get into in a minute, is the 16 K context size for G P T 3.5. What do you make of these announcements?
Chris Sharkey (00:01:09):
I think that, I mean, I'm excited by all of them. Firstly on the context size, the thing about that is, even though you and I have been playing with much larger context sizes through philanthropic and Claude, I think a lot of people who are developing AI-based applications at the moment are using 3.5 turbo because it's cheaper. Most people have access to it, whereas G P T four CLA are more difficult and you gotta get lucky and timing and things to get access. So this is the first time we've had mainstream availability of a larger context size window, um, outside of running your own open source models, for example, or having access to one of the others. And so 16 K, that's double what you can get on G P T four now, unless you have the 32 K, which I don't know anyone who does. So 16 K is huge, and I think a lot of the things we've spoken about in the past where it opens up applications in terms of what they can do, this is gonna open it up for everybody. So I think again, we're gonna see an explosion in the applications built on top of it.
Michael Sharkey (00:02:07):
When they say more steerable versions of G P T four and G P T 3.5 turbo, what does that mean?
Chris Sharkey (00:02:13):
It means it's better at following instructions and doing what you want it to do. So less hallucinations, less sort of, uh, outputting in formats that you don't like. Uh, and then you have to, you have to coddle it into doing things like that. Um, there's something I wanna talk about in a minute, um, which is Microsoft's guidance syntax, um, which we'll talk about soon. And yeah, this ability to have a steerable model is becoming more and more important, where even though we want the abstract abilities of the AI in its creativity, we want it to output and behave in predictable ways so we can build them into applications. And so when they say steerable, I think they mean it's more predictable in the way it interacts with you.
Michael Sharkey (00:02:54):
Cause this is, this has been a big problem. I mean, we've faced this and other developers out there, uh, we've seen them experiencing the same problems as well. Where you would, you, you would, the, the user would ask the AI a question. And the example they give is, you know, what's the weather in a, in a various location? And then you would want to take that, uh, input from the user and turn it into a specific data format that your app could consume and then return a value to the api. Yeah. Like going and fetching the weather. But previously, one of the challenges with that was it was poorly structuring the, the Jason or, or the, the YAML or whatever you were using and that was causing errors and, and caused it to be fairly unstable.
Chris Sharkey (00:03:42):
Yeah, that's right. So what it outputs in text, because it's a, you know, it's designed to output in text, and so you inevitably have to pause that text into something useful. And so if you ask for J S O N and then it puts a bunch of extra quote marks in there, or heaps of new lines or heaps of curly braces or whatever, then it breaks the J S o. So you know, you're writing all of this code to sort of, you know, massage it into a pausible format. Then if you try a different format, like you mentioned yammel, that's really good, but then it adds in an extra colon or an extra whatever, and that's difficult to pause. And so I think a lot of the code people were writing to get their apps up and running was all this stuff to, you know, molly co coddle it into being exactly what you want it to output. And therefore you have to add all this extra stuff into the prompt to tell it, Hey, don't do this, don't do this. Don't quote mugs, don't do whatever. And it's all this like, you know, a naughty child kind of thing where you're like, please don't. So now they've adjusted it, um, to, to be better at doing that. And specifically specifying the output format
Michael Sharkey (00:04:41):
For people that haven't worked with this technology or, or, you know, aren't that technical. I think the best way to describe it is like, you're essentially begging the AI to please, please, please just stick to this format. Don't do anything else. Don't say anything else. Just structure it this way. I beg you. Yeah.
Chris Sharkey (00:04:56):
And, and so for example, you'll ask it, you know, really specifically, I want js o valard js O output. And then it'll go, oh, sure, here's the valid JS O output and add a bunch of texts at the start. That's not JS o n. And you're like, come on. And so, yeah. And so what that led to is a lot of retry code and things like that. And when you are hitting up the bigger models, it can take a while to get a response. And so what it means is that you are, you're delaying the response to your user possibly by, you know, 20 seconds or 30 seconds every time it has to retry to get a more valid response. And so I think it also means things like temperature change because you're apt to go for a lower temperature to make its output more consistent, even though you might want it to be more creative with its response. So you lose some of the ability to control the variables there. Cause you're so desperate to make sure you get the right kind of output.
Michael Sharkey (00:05:49):
And I think a lot of this is just developers figuring out how do I use this technology to provide benefit to my users? And as you're doing that, you discover all of these problems with implementing AI into your application. And so what OpenAI thinks doing really well here is identifying those main problems and then going off and offering solutions now through the api. And so that's de
Chris Sharkey (00:06:14):
That's definitely right. And it is exciting to see that in all areas. We're seeing people recognise what the problems are, and then they come out with a comprehensive solution that completely solves it. Like, no, they don't want everyone out there having all this spaghetti code trying to, to meddle with the models. And also the, the fear is, and my fear has always been, okay, you've written all this code and then a new model comes out, um, and then you switch to that model and then suddenly you're not getting your output in a consistent format anymore. This eliminates that problem.
Michael Sharkey (00:06:45):
I think also a good way of thinking it through for, uh, you know, people not as familiar is just this idea of you, uh, currently getting, uh, like when you're working with this technology as well, you're just chatting to the AI or asking the AI to return a certain value. And then now what you can do with open AI is teach it aspects of your app. So when you have functions in your app, you're saying, Hey, here are a bunch of functions I have in my app that you can use in this format, and now it knows it can go and use them in that format. So it's just making it really clear to the ai, you know, what it can retrieve in a clear and consistent way with your application. Yeah.
Chris Sharkey (00:07:28):
The built-in function calling is sort of what I had imagined the plugins would be in the first place. The idea that you tell the AI here are all of your abilities. These are the different things you are able to do if you so wish. And so the great thing about the function calling that OpenAI has released is it actually allows typed parameters. So what that means is if it's expecting a number, you can like your function. I mean, if your function in your system is expecting a number for a certain value, you can specify I want a number. If you want a string of a certain length, you can specify that. So what it means is you can develop the functions in a normal way you would do with normal programming without having to sanitise every single input to check that it's the right kind of value. And if necessary, recall the model, it knows how that function needs to be called and we'll honour that. So that's a really, really powerful aspect of the function calling they've add.
Michael Sharkey (00:08:20):
Yeah, because before, if you're expecting a string, which is just a, a piece of tax, and then you've got a number that's not necessarily gonna be the biggest problem. But if you're expecting a number and got a string, like as an AI model, I can't give you an answer to that. Yeah. It, it can really break
Chris Sharkey (00:08:36):
The answer is four or whatever. Yeah. And so this, this really does that, and additionally it actually means that the ai, instead of it having to spit out sort of an imaginary function call, and then you coerce that into what you need to call it in yours, which is what people were doing, um, it, it actually can call a function. And if you look at the implementation open AI is done, they actually have this interactive model where you can actually run the model again with the answer to that function. So sorry, the result or the output of that function back into the model, and it can sort of pick up where it left off. Now knowing that answer, um, so it's actually a better paradigm to, to work through that sort of agent model where you are working with it and it's able to retrieve information and continue on with its original mission.
Michael Sharkey (00:09:24):
I thought the other interesting piece in it was this idea of extracting structured data from text, uh, in, in a consistent format. So the example they gave is going to Wikipedia and extracting all of the information about people in an article and putting that in a structured way with expected, uh, inputs. So again, stating the variable types around say name, location, uh, I think it's here on the screen. Uh, birthday is a, I don't know why it's a string. Uh, but yeah, giving them, yeah, that makes no, oh, I guess it's yeah. Date
Chris Sharkey (00:09:59):
Feel. Well the thing is like, if you think about our betting example from last week, we were talking about this exact problem where we wanted to extract the, the relevant horse information and things from a, a page, but we wanted to exclude the bookie's comments on the particular horses. That kind of thing would allow you to, to do that. Um, and the other thing I really wanted to bring up though is Microsoft guidance, and I was going to talk about this this week anyway, but prior to knowing about OpenAI releasing the function calling, because Microsoft's released a library called Guidance, which is a really interesting paradigm for working with large language models. And the first thing to note is it's agnostic to which model you're using. So you can use it equally with, with LAMA or Open AI's model or other open source models. It doesn't support philanthropic yet, but I wouldn't expect that would be far off.
(00:10:51):
There's nothing precluding it. And so you can use a sort of generic syntax to say, this is the system message, this is the assistant message, this is the user one, which is very interesting on its own, but it takes it far beyond just having a generic, uh, syntax that works with different models. It has an actual sort of structure to it that works with the tokens to accomplish certain things. So to give you some examples, it already prior to, um, the, the stuff we're talking about with open AI today could force the output format. So you could actually say, um, I want J S O and here's an example of the Js o n I want with templated variables of where you want it to fill in the gaps in the J S O N. And it went so far as doing things like having select, so the example they give on their GitHub is like armour for a character, and it's like select between leather chain, mail or plate, and you actually specify the options so it won't stray outside of those specific options.
(00:11:48):
And then additionally, you can have these sub l m calls within it to generate data at each point of the way. So to continue the sort of soldier analogy, they have the ability to like generate a mantra, generate a strength, generate the items they're having, and they actually have different parameters to the L L M. So a different temperature, a different prompt that can generate each thing within one greater generation. And so it can then progressively generate it through using smaller calls to the, the LLMs and including different LLMs within that to generate one overall thing it's trying to do. And then finally, I know it's a lot, but finally it can actually do similar to what Open AI is announcing today, where it can actually, um, it can actually have that interactive thing where it will pause its generation while it gets say, input from the user to fill in a certain gap. Um, and then once that's filled in, it's able to continue the completion from there. So it's a totally different paradigm working with the LLMs that leads to much faster responses, perfectly structured output, the ability to call functions within your code from it, even without the open AI stuff. Um, and you know, and, and just generally, uh, a different way of, of interacting with the models seems
Michael Sharkey (00:13:07):
Like being model agnostic here too is also one of its key strengths, whereas with the innovations to open AI's models, you know, tho those function, that function calling is limited somewhat to obviously their models. I'm sure it'll be copied .
Chris Sharkey (00:13:24):
Yeah. So I've tried the Microsoft guidance with function calls, and I just, prior to having the open AI thing, I gave a list of, here are the functions you have, and it has a syntax, like a, it's, it's similar to a handlebar syntax, which is the templating language where you can say, here are your abilities. And it was able to call the functions perfectly. But this morning after I saw the announcement, the first thing I did was go on the guidance GitHub and look under the issues to see if someone had requested adding the function support for open ai. And they had so already,
Michael Sharkey (00:13:53):
Yeah, minutes later
Chris Sharkey (00:13:55):
They beat me to it.
Michael Sharkey (00:13:56):
It really shows how quick things are advancing.
Chris Sharkey (00:13:59):
Yeah, that's right. The immediate thought comes to it. So it's very exciting. And probably the other thing worth mentioning about guidance that I found really interesting, it actually has the concept of hidden blocks. So when you're trying to generate content, content within your prompt, you can actually say, don't display the history of this part of the prompt when you are asking for further generation on the model. So the idea is that you will get these sort of artefacts from generating content when you have a ongoing conversation. We've spoken about this before. So if you're in the chat mode and you're chatting with the bot, you don't want a previous answer to cloud. Its thinking about how it answers the late later one if it isn't relevant. And so it has a way of actually saying, well, this block of information here, I need you to do it.
(00:14:47):
I need the output of this, like, say, gathering the user's input or, um, generating an example. But once you've generated it, I want you to ditch the prompt part that led to that output and just have the output. And that way it doesn't muddy the waters for, I guess this is around guidance, right? It doesn't muddy the waters for what the later goals of that prompt are. And so that's really effective in, in getting, in guiding it to the kind of answers you are you are looking at. So I think this, this paradigm of working is really excellent, and I think it's only enhanced by open AI somewhat officially supporting it through the function method calls.
Michael Sharkey (00:15:24):
Yeah, there's some really exciting possibilities for it that you can start to expect in all the applications you use as well that are implementing ai. Uh, namely, if you've ever done searching or filtering in applications, you can just now ask pretty complex questions. They give examples like, who are my top 10 customers this month? And, you know, you, the developer can easily call the database and return those values. How many orders did Acme Inc place last month? And it can return the answer. So it it starts to mean in business applications as well. Uh, in, in our product that, that we work on auto, uh, we have filters to find, uh, a a group of audience you, you might wanna send an email to. And this will allow us, for example, to just have natural language filters in our product pretty soon where people can just search for what kinds of customers in their database they're looking for and it will return a match. So it's, it's pretty impactful to people's day to day this kind of technology. Yeah. And,
Chris Sharkey (00:16:24):
And to give an example of why this is better, because of course that was possible before, you know, and we've been working on things like that, but the issue is for it to output in the filter format that we have, you know, like to, to actually run it through our database to get it out. Um, it needs to be coerced into an exact format that our system supports. Now, to do this prior to, um, guidance and the function calls from OpenAI, you would actually need to have all this code that takes the, like sort of specifies what kind of output we want from the model and then coerce that into our format, making sure it never makes a mistake because it's not a programmatic thing at that point. It's just the AI's output. And so you gotta do all this extra work. This, this just eliminates all of those steps. It means you can tell it exactly how to do it and it will adhere to it. So it's a huge advancement not just in the possibility for new applications, but just the speed and accuracy at which you're able to implement it in existing applications.
Michael Sharkey (00:17:23):
Yeah. So I think if you're a user of these technologies, it just means that you can expect this stuff to be coming to more and more applications by giving developers better tools to go and implement this into many of the apps you use Chris also,
Chris Sharkey (00:17:38):
Uh oh, sorry. And just, and just less like a chatbot. Like it's not all just gonna be okay now, now just chat with it and ask what you want. And it's just like chatting to something, it can actually do stuff.
Michael Sharkey (00:17:49):
Yeah. Like making that AI productive, which we've talked about a lot. A lot of the early use cases have just been l l m rappers chatting to a PDF for example, which gets old pretty quick.
Chris Sharkey (00:18:00):
Yeah.
Michael Sharkey (00:18:01):
So the other thing we wanted to talk about was this, um, Andro cluster released by Nat Friedman and Daniel Gross. For those that, uh, don't know, Nat Friedman was the c e O of GitHub for a number of years. I think he left in 2021. He's also an advisor to Mid Journey, which is I think the leading image generation, uh, technology and AI right now. Um, you've got Daniel Gross as well who helped, uh, release this, um, Adro Atter cluster. He worked on AI at Apple from 2013 to 2017. I read he also invested in startups, should know like Uber, Instacart, Coinbase, GitHub, open Door, Airtable, fever, pretty well
Chris Sharkey (00:18:44):
,
Michael Sharkey (00:18:45):
Uh, yeah. And so they've released Andromeda for their startups. Um, I'll let you explain it, but I thought the best takeaway is they have, they are using 3.6 tonnes of GPUs in this server cluster. So that's a lot of GPUs.
Chris Sharkey (00:19:03):
I mean, you say me, explain it. I can't explain it. It's unbelievable. It shocked the hell outta me when I saw it, first of all, because their website looks like H D M L from 1997, which I actually respect, where it's just like, you know, just literal times New Roman, just whatever the default is, they don't have time for websites and shit because they're installing 2,500 H 100 s. It's like we were talking about a limited supply of these things, how the, like I better not swear, but how did these guys get so many and why so many? It's like, oh, you know what, Nate, what's his name? Nat. Hey Nat, we should build a cluster. It's like, sure, how much money do you have? He's like, heaps, I've done well in all
Michael Sharkey (00:19:41):
These, I'm gonna vote two and half thousand.
Chris Sharkey (00:19:44):
He's like, I've got so much money. Um, I might as well buy 3.6 tonnes of jelly. Like when you're buying and buy the tonne, it's not a deli. You're not like buying meat, you know, like you're not buying 'em by weight are . I
Michael Sharkey (00:19:56):
Just, it's the the funniest stat I've ever seen. So again, this is a big cluster to train AI models. Is that, is that
Chris Sharkey (00:20:05):
It can do anything. Game loan. I mean it can run inference and it can do models. I mean, don't get me wrong. I think this is exciting and cool. I mean like, you know, you talk about investors, you know, sort of, uh, often saying they can bring a lot to the, the table with uh, stuff and not being, you know, necessarily translating to what they can actually do. But these guys, I mean, seriously, that's a pretty powerful thing to offer to their portfolio of companies and it looks like they're gonna open it up for people who want to pay. It's a pretty exciting thing. I'm not a hundred percent sure how, cuz they're talking about like giving s ssh access and you know, sort of very direct access to it. I'm not sure how they handle the sort of time sharing nature of it if they've developed software to do that or they're just gonna have a schedule.
(00:20:47):
I don't really know. It sounds a lot like back in the university days where they'd have, you know, a mainframe and everyone would take turns using it to run their jobs on. Um, it seem seems similar to that, but it would certainly lead to the ability to, for people to train their own models, which funnily enough, we were talking last week about Sam Altman sort of saying, Devon, haha, no one can train a model like we can. Um, and then, you know, we saw a big Indian response from that because he was sort of taunting the Indians saying they can't do it. This, this and another announcement we'll talk about in a minute, sort of say, well maybe people can't, I mean I think they said that running this thing for 16 days would be enough to train something as big as Lama 65, 10
Michael Sharkey (00:21:26):
Days, 10 days. That's all they need.
Chris Sharkey (00:21:29):
Yeah. So it means that you could train some, like if you ran it for a few months, you could train some seriously large things. It's, it's truly exciting and cool. I mean, I'd love more information about it. I'd like to see it, I'd love to see what it looks like. Like where is it physically? Have they bought a data centre as well? Like how much space does that take up? It's, it's just a genuinely fascinating and awesome thing.
Michael Sharkey (00:21:50):
I love how it says on the website, your results may very consult your doctor before beginning a new training routine. , they're even trolling with it as well. The, the, the thing that is truly interesting though is you could just see this is the best marketing for an AI investor on the planet because it's saying, Hey, any AI startup that knows how to use my Androta cluster, come and use it. Reach out if you want access. Oh, if I like the idea you're running on the cluster, not only can you use the cluster, but I'll fund it. So it honestly seems like a great investment strategy.
Chris Sharkey (00:22:22):
Yeah. And the sort of tacit implication is like, you know, if you're a portfolio company, you probably get some sort of free access to it. Like they're not gonna give you capital that they, they then get spent on themselves. Well maybe they will, but um, you know, like it, it sort of, it means by definition they're gonna get access to the best deep learning AI startups, I would assume, or at least have a look at them. Um, so it's pretty, yeah, it's, it's a great strategy. It's damn cool. I I think it's awesome. I hope it works out for them. Do
Michael Sharkey (00:22:54):
You think this access piece is such an important part of like what's happening with AI right now in the sense that, you know, if you don't have access or countries don't have access, like to, to take that into your example and, and to be clear, Sam Altman came out after that and said this has been taken way outta context. Uh,
Chris Sharkey (00:23:15):
Yeah, probably because of the severity of the response
Michael Sharkey (00:23:19):
. Yeah. Okay. Uh, but, but I think what's interesting is, do you, it, you know, can other countries even get this many H 100 s right now? Is this hoarding the technology and that's what's gonna restrict or limit other people to compete? It seems like that is a big, well, I guess,
Chris Sharkey (00:23:38):
I guess my answer is, I don't know because if you'd asked me yesterday, do you think a single private person could buy a two 500 H 100 s? I would've said absolutely not. So it just, yet again, shows I know nothing and I, I'm really, genuinely not sure. And so we also saw that Lambda, which is a company I followed, just cuz I find it exciting, they're an online cloud provider where you can basically either buy from them dedicated hardware, like dedicated GPU servers for tonnes of money, like these a hundred thousand, $200,000 computers I fantasise about. Um, or you can sort of lease them similar to Amazon, you know, where you hire a server by the hour or the day or, or whatever you want. And so they were advertising last week, I think that they have H 100 s available. Um, and then just this week they said that they're, they're down like the number of H 100 s they're almost sold out, but they said, I think they said they have tens of thousands of them or something like that. I need to find the actual announcement I saw. But it was a lot. And I was very, uh, you know, interested that they had so many, um, so I guess that I'm just not used to talking about it at this scale. They said, oh, they only have 4,000 left, so,
Michael Sharkey (00:24:51):
And they're expecting to be sold out. I read by QS three this year so it, all these resources are disappearing real quick,
Chris Sharkey (00:24:59):
Deploying tens of thousands of them. So you just think about all of those GPUs pounding out models or running models 24 7 and what, what can come of that. So I don't know, maybe the hard hardware shortage isn't that bad. Um, but I guess it's just, we always talk about it in that context just cuz we know that the thirst for them is always going to be there. And so inevitably you will run into some limits at some point or you know, maybe it's simply a cost thing because it's, it's not cheap to buy and lease this stuff.
Michael Sharkey (00:25:28):
What kind of things could startups do on Anter that they couldn't do? You know, using other stuff? It's
Chris Sharkey (00:25:37):
Training. It's training the truly large models where, you know, it needs so much G P U Ram, like, you know where, cuz right now it's a time game. Like if you've got a smaller graphics card with a certain amount of memory, you can train a certain level of model. Like you can train Lambda with 7 billion parameters, but if you want to train the larger one, then you need like an A 100 and if you want to train something bigger than that, you need, you know, perhaps a few GPUs strung together. And that adds difficulties in terms of the way it's done it with these H 100 s. Like obviously you can train much, much larger models. So I think it's competing on the model front, the very thing that we were talking about, you know, would people do, uh, you know, this, this enables that. So I think that's the thing. It's more about training models than inference, I would say.
Michael Sharkey (00:26:24):
So in a similar, uh, story about competitive models, France's Mistral AI gets 113 million seed round to take on open ai. According to TechCrunch, it says Mr. AI based outta Paris co-founded by people. Google's DeepMind and Meta are focusing on open source solutions and targeting enterprises to create what C e o Arthur Mench believes is currently the biggest challenge in the field to make AI useful. It plans to release its first models for tech-based generative AI in 2024. It's a lot of money for a company that has no product.
Chris Sharkey (00:27:05):
Yes. But it sounds like these guys have pedigree when you read about them. You know, they've been in the AI space for a while, at least in terms of their experience. And, um, you wouldn't doubt 'em with that kind of money that they can produce something good. It's
Michael Sharkey (00:27:18):
Interesting that if you think about it, they say in the announcement, open source can prove tactical insecurity and we believe it would the case here too. Uh, while OpenAI has the word open in its name, it felt like anything, but they feel like Proprie the proprietary approach was largely shaving up to be the norm. They saw an opportunity to do things differently. It's really nice to see these people dedicated to open source.
Chris Sharkey (00:27:44):
Yeah, it's interesting and it's very interesting to sort of do that, like take a a bunch of VC money, which is inherently private and then talk about making it open. Um, so it's, it's, that's great. I think it's, it's really good that they will approach it like that. I guess they just need to internally think about how does that work out for them privately and for their investors, but not our problem. If they want to come out and build even better open source models, then please do it.
Michael Sharkey (00:28:11):
My view of this right now, how it's shaping up with proprietary versus open source is that to me, the LLMs that people are using in their applications, the AI that we're using today. So think of things like chat G B T, but for developers to add into their applications. It's becoming as commonplace or will become as commonplace as just using an ES SQL database in your application. And as we know, historically, all of those technologies are mostly open source, uh, and then large companies are built on top of those technologies by supporting them and bringing them into the enterprise and servicing them and hosting us based on their expertise of actually building the, the project originally mm-hmm. . And so this will be an interesting competition to see play out between open ai, which is not open and, and this misre and, and others, uh, like LAMA that are going down that, uh, that open source route. And we just see constant advancements from Llama now or at least Innovations. And on the Lex
Chris Sharkey (00:29:19):
Freeman. Yeah, Lama Laa just this week added GPU U support, which is pretty interesting because the whole thing about LAMA dot CBP was it runs on CPUs so anyone could run on it on their computers for enhancement, but what they're doing now is just making it so if you've got a gpu, it can use that too just to make it faster. So it's something that's advancing absolutely rapidly like that, that LAMA community and what, what can be done with it. So yeah, I think that's, um, I, I don't know, I just think it's great to see the rapid advancement of the ai, the open source models,
Michael Sharkey (00:29:51):
But do you think that all paths now lead to open source models in the sense that, like are they, are the proprietary ones going to be able to compete, like will open, will, will open source force them to release their models? Or can you see a pathway where open AI now is just stays proprietary and it's their unique training data and their special source that keeps them ahead?
Chris Sharkey (00:30:14):
I don't really know because I've always struggled to under understand like, you know, the monetization of open source. Like there's been famous companies like Red Hat and then all the database companies, you know, like Mongo DB and various others who make money on the service model. So, you know, the software's free, but you can, you've gotta pay for the service, which, you know, enterprise companies really need and want. So that's how they make their money. And open sourcing makes sense because they get the benefit of the security and up channel advancements from the different software makers. But in terms of the models, you know, if people can copy them and run them themselves and you are putting it out there, then what do you have that you can charge for on top of that other than potentially maybe the hardware or private professional distributions of it?
(00:31:03):
And maybe that's what it is. Maybe. So for example, if we look back at Mistral, maybe the goal is they make an open source model that anyone can run and use themselves, but they say, look, if you want to run with the highest security, if you want it run for your corporate enterprise in a way that you, you can still meet your security certifications, you can still meet your obligations to your customer, then we, we as the experts run it for you. And that's probably what the model is, right? Similar to, we saw Open AI briefly flirt with that.
Michael Sharkey (00:31:33):
Oh, they, I mean they're actively doing it. They're, they're actively training, uh, uh, models with the enterprise right now and, and going after the enterprise. I have old team working on it and I think Amazon have just taken the different approach of saying we're agnostic, we'll host it and we'll caught the enterprise as well. But I just, that's why I I start to think maybe the models become mostly open source and the specialty is like fine tuning and training future models. Yeah, and
Chris Sharkey (00:32:00):
You're probably right. And I would say that that's what the open source based ones are looking at saying, well, we're not gonna be able to outcompete the, the collective might of everybody who's enthusiastic and talented at this working on it. So why not embrace that, invest in it and then add on the services on top that we can, that can, can make the big money. Because I don't think a long-term play is getting everyday Joe's to pay $10 a month to, to chat with a bot. I think that it's the real corporate Oracle style where it's the foundational element in everybody's business that they're paying you epic money for years to come that makes the real money.
Michael Sharkey (00:32:41):
Yeah, it'll be interesting on the price front to see how it plays out because I know as people add these technologies into their apps, the question is, is like, are people willing to pay that additional money to run the advanced AI models or does that just get absorbed into the existing margins of the providers where they need to just absorb it? Because it's an expectation now of the fact that you have to have this technology in your app.
Chris Sharkey (00:33:04):
I would expect it's the latter. I I think that that, so every product's gonna be expected to have it in there now, and I don't think charging people extra saying, oh, this is the AI package, you've gotta pay extra. Unless I suppose if it's, if it's sort of customly trained or has, you know, a huge amount of data hosted for that user, maybe then there's a marginal cost for it saying, you know, you can opt into this if you want and it's gonna provide this extra feature for you. But I agree, I think it'll become more of an expectation than, than some sort of, uh, premium add-on.
Michael Sharkey (00:33:41):
I alluded to it earlier, mark Zuckerberg last week was on the Lex Friedman podcast. It was pretty interesting timing after the Apple Vision Pro came out. Him, uh, popping up on Lex Friedman, uh, to, to talk about that. But I was mostly interested in what he had to say about the story behind Llama getting released and you know, kind of what went on from their point of view there. Uh, and he did speak quite a bit about open source ai. He said that, uh, and I'm paraphrasing quite a bit here, but he said that with Llama, they obviously did release it, uh, like open sourced it to the research community, uh, and they, they didn't want the weights to leak, but it turned out that it was a good thing that they leaked because there had been all this innovation around it. He also said he backs open source in the open source community and they've been incorporating into LAMA a lot of the innovations that we're seeing in the open source community. So it's actually benefiting them, it's making them a all better, it's making it more secure and more efficient, he said as well. So that it seems like the fact it leaked at first everyone thought it was a disaster, but for them now, it kind of proves the benefit these companies can get from open sourcing their models, maybe not their latest models, but their models.
Chris Sharkey (00:35:03):
Yeah. And they can incorporate the changes they're seeing in the community backup stream into the, the internal ones that they're running. So that is interesting and good, and that is the benefit of, of open source, I suppose. And I partly wonder if why the, the success, the ongoing success of LAMA is because Facebook is sort of behind the scenes supporting it. It's with developers or other resources. I don't know. I don't follow it closely enough to be sure.
Michael Sharkey (00:35:27):
It's so funny that, uh, you know, , I, I gotta go out this tweet. Mark Zuckerberg went from reptilian overlord conspirator to down to earth, uh, jiujitsu champion releasing opensource software as the big tech villain villain, redemption arc, nobody saw coming . I'll credit that in the sources, but what a great tweet. I, it is interesting. Mark Zuckerberg's gone from this like evil guy, uh, that everyone thought was a robot. Uh, he might might still be to an article in the information that was released. Mark Zuckerberg is the hero AI needs. So he's literally,
Chris Sharkey (00:36:07):
I wonder if, I wonder if they, sorry you say that. Oh no,
Michael Sharkey (00:36:10):
I was just, I just think it's hilarious that he's gone from this like hated guy to, he's gonna be the saviour of humanity by open sourcing ai.
Chris Sharkey (00:36:19):
Yeah. And part of me one is, is he just going with the flow? Like it sort of played out that way where the models leaked and then they, you know, him and his experts gauge the reaction. They're like, oh, people are actually liking this. Maybe we just run with it. Or if it was some sort of deliberate thing, I mean, it would make sense that he wants to embrace open source. Most of the big tech companies got there because of open source, Amazon, especially Facebook with P H P and they've always given back to, to P H P I think, I don't know, I might be wrong there, but you know, they, they certainly benefited from open source over the years, so it kind of makes sense that he'd, he'd want to do that. But at the same time, clearly it's, it's benefiting them just as much as it's benefiting the open source community. It
Michael Sharkey (00:37:00):
Sounds like they have an agreement though with the people that work there. Uh, and that team's headed by, uh, Jan Lacoon who, uh, I think a lot of listeners would be familiar with, and we'll talk a little bit about him in a minute, but they, he said they intentionally opensource the models and release the research because it's in their agreement with these academics to have them work at Meta. Like they wanna release their work, they want to contribute to science. And so therefore it seems like that was already the agreement. And a lot of these companies are similar like Google, uh, having their researchers there, being able to release their own work. But we know more recently a lot of them have clamped down on it because they see it as now their, their strategic advantage. But he said Facebook we're, or Meta, sorry, we're going to continue to release, uh, this stuff. He also, I thought some other interesting takeaways from it was his vision around generative ai. Um, in general, this idea that he thought it would be baked into all of their products. He called out specifically, they're gonna release chat g b t like features in a messenger and WhatsApp. He also said for creators on Instagram that want to interact with their audience, they would have potentially some form of agent in the future where you could interact with that creator as if the AI is them. And we talked about previously how, oh, like the,
Chris Sharkey (00:38:19):
Like the Sexbot one we talked
Michael Sharkey (00:38:21):
About. Yeah, the before not a sex bot, but that girl creator who made like $2 million a a a week or something from people talking to her. Um, I thought the other one takeaway was that he, in the future with advertising, they're working on implementations where if you're a local dog walker, you can literally go to Facebook and just type like, I'm a dog walker, I wanna reach people in my local area who are willing to pay this much. It goes and creates the ads, it optimises 'em automatically and it helps connect you with the right audience. That seems a pretty cool use of advertising, especially through meta. There was a lot to take away from that, that interview. But they are all in on generative ai.
Chris Sharkey (00:39:02):
Yeah, and I wonder if this is because the whole Metaverse thing, they invested so much money in that and it's not shaping up as quickly as they would've hoped or as well as they would've hoped. I'm not, I'm not writing off the whole vr ar thing yet, particularly cuz of Apple having a play there and just because I've seen good applications of it, I just don't know how I feel about that. However, this seems like a more here and now thing where they can benefit from it, they can benefit financially if they can do the advertising thing like that. And certainly they can benefit in terms of being a major player in the AI space if they embrace it with the size of the audience they have. To
Michael Sharkey (00:39:39):
Me, this is their area of strength. You know, they've got all the data, they've got a whole bunch of users, they've got billions of users using these tools. They can distribute models, they can distribute AI to the masses. He talked a lot about building agents and he said he disagrees with the whole idea of open AI's chat, G B T and Google Bard being a single agent. He, he said, I don't think people are gonna use single agents. We've talked about this before, but there'll be many agents just like apps that people consume to do different things and they'll be specialists. So
Chris Sharkey (00:40:09):
Yeah. Yeah, that does make sense for sure. It's
Michael Sharkey (00:40:11):
Clearly that's the direction, um, meta and Zuckerberg's headed. Uh,
Chris Sharkey (00:40:17):
Yeah, I, I actually wonder about that. I, this is slightly off topic, but something I've grappled with lately is when you've got all the different models that have different strengths and weaknesses and then people index them on different problems in terms of how well they do, how do you get your programme to decide which one to use when, cuz you've got the functions now. So it's deciding, okay, I'll use this function for this scenario, but you almost need a way of understanding the strengths, like which model's gonna give the best outcome? And then having the system decide that, almost like how you provide the functions to the model saying here's all the functions you have, it should be, here's all the models you have and their strengths and weaknesses. And when
Michael Sharkey (00:40:55):
You say models, you mean things like G B T four, G b T 3.5 cla.
Chris Sharkey (00:41:00):
Yeah, yeah. And, and also the open source ones. So if I could say to it, you know, here is the, the sort of, um, graph where you sort of say the vectors of cost versus experience in the different problem domains. If you could give it that and feed that into the thing, it could decide, okay, well this is a cheap and easy one that even this this like, you know, could be solved on a raspberry pie. I'll run it on the cheapest, easiest one. This problem needs a much more robust answer. I'm gonna run it on this or maybe several and get the best answer between them. I just wonder how that problem's gonna be solved. It seems to me like that's the next one where an application developer doesn't really want to think about that. They just want the cheapest and best result they can get. You know, the sort of nexus of those two. I feel like solving that problem would be a really, really big advancement alongside the functions. So it's like you don't really care what you're running as long as it just gets the results. So that's a bit of an aside, but I just, I that, that's one that I always think about where the solution should come from there. It's
Michael Sharkey (00:41:59):
Like a telephone switchboard. It seems like there's an evolution, so it's like select the best functional skill. The next one is select the next large language model that's, you know, cheap as most efficient best result. And that's like the old telephone switch raise. And then the next one on top of that I think is a multi-agent paradigm where it's like, I need to complete this task. I'm gonna go grab these six agents cuz they're the best or, or they, they can do the skills or processes I need to complete that overarching goal.
Chris Sharkey (00:42:27):
Yes, that's right. Yeah. And then you've got the super supervisor agent that's like hiring and firing the
Michael Sharkey (00:42:33):
Agent. Yeah. Like the c e o agent that's literally in, in control. And then that goes down further levels. Maybe we're just describing the future of programming in general or the future of how things are executed in the future. Like that's just how this plays out.
Chris Sharkey (00:42:50):
Yeah, it's quite possible. And then I wonder what that means for the sort of, um, the companies producing the models because then if you've just got one, you are not gonna exactly gonna go out there and build technology that encourages the use of six. You're gonna want everyone to use your one as the be all and end all, or you'll have to have within your little farm, lots of lots of different animal agents that can do the different things and then they encourage you. I mean, I guess that's what Amazon's kind of doing in a way, aren't they? With Bedrock, assuming that, that you can actually use that at some point.
Michael Sharkey (00:43:23):
Yeah. But what you are describing is a tool that knows which really which one at any given time to, to select. But I see what you are saying. It's, it's sort of like servers right now where Amazon's selecting servers that are idle or cheaper to use. It's not that different.
Chris Sharkey (00:43:39):
Yeah, yeah, exactly. And that's why, that's why I think Microsoft guidance is so cool given that it, it can seamlessly transition between models even though it's running the same thing. So you could actually do that experimentation quite a lot quicker and not have to rewrite things all the time to suit the different models.
Michael Sharkey (00:43:56):
I'm curious to see now if this is the, the way things play out, especially in the open source community, if this l l m selection or fat controller tool is the next thing you say, no, you should
Chris Sharkey (00:44:06):
Register that. I
Michael Sharkey (00:44:07):
Really think we should start an open source project called Fat Controller and try and build it. It would be fun.
Chris Sharkey (00:44:12):
Yeah, I like it.
Michael Sharkey (00:44:14):
Um, so another a few like generative AI things we always get paid out for not mentioning the one, this show I just find on a podcast it's hard to demonstrate like video, uh, technology like runway ml, which has had some big advancements lately. One of the ones back again on the meta front, I'm finally getting used to saying meta thanks to this podcast Music gen meta release, which enables you to create, uh, music from a text prompt. And it's, it's really cool. I'll just quickly play a, a one I build, um, is like a podcast synth introduction. Here we go.
(00:45:01):
So it's, it's pretty good. Like it's really fun to play around with and uh, you just type in a prompt. It's available on hugging face, you can actually go and try it out. You can also do things like upload an existing track, which I did with our podcast theme, uh, intro music. And then you can describe the, the, you know, the music still that you want created and it'll copy the melody from that track that you upload. It's really worth checking out. I'll link to it in the show notes, especially if you're listening. So you can go and and check it out. Have you
Chris Sharkey (00:45:31):
Heard on Spotify how people make these, these tracks that last for like seven hours and stuff like hoping it'll, it'll shuffle onto it on someone's Spotify and then it'll leave it there and they get the revenue. Imagine how much content is just gonna be shoved onto Spotify. It's like endless music. Yeah.
Michael Sharkey (00:45:48):
. Well, literally, I mean, it, it only produces 12 seconds. This, this particular model right now of audio. So it's not there yet, but it's not that far away. And I did see today
Chris Sharkey (00:45:58):
That's pretty, it's pretty us it'd be pretty useful for YouTubers and streamers who can't use commercial music on their, on their recordings to have, you know, really high quality background music. I mean, assuming the length gets higher, which I assume it will,
Michael Sharkey (00:46:12):
Well, stock music as well, like if you are, if you're in stock imagery, stock video, all stock is just gone. Like why would you take stock when you can generate it? And then if you put those generations in a library, uh, I mean there's, there's still probably a business there because curating the actual new AI stock music, like not everyone's got the best taste, but Adobe Firefly this week showed off, uh, something at a conference in the uk I believe it was where you could, in your editor. So you're editing a video and you've got a clip of an ocean wave, like some waves crashing to the shore, and then you can literally just right click and say, create a sound effect, uh, effect that matches this visual and it will just create a wave crashing noise that perfectly matches the videos. So yeah, like, I don't, I mean you'll pay for these tools, but the traditional stock providers, I mean that they're, they need these a 100 s to start training, training their, their own models so that they can release this technology. Can
Chris Sharkey (00:47:18):
You imagine like what DJ sets are gonna be like, people are gonna be up there typing prompts into the thing , you know, with their headphones
Michael Sharkey (00:47:26):
Red prompts in results just mixing different like 22nd clips. It would be pretty funny. Yeah, someone should,
Chris Sharkey (00:47:32):
You can have one that like reacts to the crowd, you know, like it's taking the visuals of the crowd and like, it's like if you guys don't hype up, the music's gonna get depressing. .
Michael Sharkey (00:47:40):
I guess it's like those things behind a television, you know, how it can control the ambient light based on the, the film maybe the, yeah, the, the music can match to the mood that evening in the room by taking the visuals of the room with a camera and then just matching the music to like the weather and time of day and the brand of the hotel, for example. That would be pretty cool.
Chris Sharkey (00:48:01):
Yeah, that would be really nice. So if it's like yeah, snowing outside or whatever, you get some cosy firecracker music or something like that. I, yeah, I I mean I maybe it's . Yeah, I don't know who's gonna, I'm not gonna spend my time on that, but I'm sure someone will.
Michael Sharkey (00:48:14):
The the Beatles also, uh, Paul McCartney said that they're gonna release a song that thanks to AI is, is can happen now, uh, which is pretty cool. He, apparently they had John Lennon's voice on an old demo and they were trying to finish this song and then abandoned it in 1995. The song's called now and then, and that they said it'll be released this year. But with ai they were able to remove this buzz, uh, from the recording. So maybe we're not that far away. Speaking of music, Jen, from, you know, if you want more Beatles songs, you just put in a prompt, like, I want more Beatles songs and, and you know, you can just, all these artists that are gone, you, we might see new albums from them. Like they keep releasing Star Wars movies.
Chris Sharkey (00:49:02):
Yeah, yeah. , that's interesting. Wow. Yeah,
Michael Sharkey (00:49:06):
So yeah,
Chris Sharkey (00:49:06):
I mean I could, I could definitely see that. I mean, the record labels usually own 90% or whatever it is of the, the artist anyway, so it's like they can just just pump out new albums as they want.
Michael Sharkey (00:49:17):
I wonder whose album will get the first long dead artist, you know, Kurt Cobain maybe.
Chris Sharkey (00:49:23):
Yeah, if I could sing now , I'd sing as an AI language model. Yeah, I love you .
Michael Sharkey (00:49:29):
Um, but it, it begs a question, I think will original work become more important because everyone's so easily able to replicate existing music or existing
Chris Sharkey (00:49:45):
Yeah, I mean it's quite an interesting question, isn't it? Because it's like, is is it the art that's important or the artist who created it? Because there's oftentimes, you know, that thing where it's like, you know, if you meet your heroes it can sometimes disappoint you. And then everyone, the response to that is usually you've gotta separate the art from the artist. Like you can, like a, a work of art and hate the artist for example, but then for other people, the, the artist themselves is what makes the artwork good. And so it, it really will ha ha that that question will become more and more important because if you just like something, does it matter where it came from or does it, because I think for some people it does and some it doesn't. And I think that will certainly be an interesting thing to see play out, which it seems like it definitely will play out.
Michael Sharkey (00:50:29):
I just am curious about do we need to develop technology that puts AI signatures on everything? Like something that can be detected but maybe not heard in the case of music or something that could be detected in written language but not seen in the writing and so on and so forth so that we understand what is the difference between AI and not and the, the Kansas University researchers release or, or not released, but acc claiming that they can get 99% accurate accuracy in detecting chat G B T fake. So basically if you wrote your essay with chat G B T, they're now saying that they can detect it. It seems like every time they are able to detect it, a new model comes out and then they fall behind again. But that's what I'm thinking is like, is there going to have to be some sort of signature that, like where you can detect if it's human or not.
Chris Sharkey (00:51:26):
Yeah. I guess given that it's working by predicting the next word, that if you could similarly predict the odds on the next word, at any given time, you could selectively go through the output, predict what the next word should be, and if they match, then you can go, okay, that's a percentage chance that it's generated by ai. Um, but you know, I don't know how with open source models and mixing things together, if, if how reliable that's going to be given the amount of variables that can go into the creation of the content.
Michael Sharkey (00:51:56):
Yeah. I don't think there's gonna be good way, it seems like cat and mouse forever now, where
Chris Sharkey (00:52:02):
Yeah,
Michael Sharkey (00:52:03):
That like, that they're like, oh, we can detect it. Oh, we can't. And then it just goes on and on and on.
Chris Sharkey (00:52:09):
Yeah. Yeah, exactly. I don't know. And, and there's always going to be that, that question of it, like I saw a hack and news thread today where someone had written an article on the first three comments were like, did chat G b t write this ? And then there's a debate about it whether the person really wrote it and you know, they said they're using short sentences and they're doing this, and therefore it's written by ai. So I think those kind of debates will happen a lot. And it's gonna be interesting when it makes its way in political speeches. We've already seen it in law cases. It's gonna be, I mean, most wedding speeches will be written by AI from now on, I imagine. So I yeah, it's, it's a tricky one. And you don't wanna be the person standing up being like, did AI write that speech?
Michael Sharkey (00:52:47):
I also think the weird part about it is you're not like humans. Everyone thinks that oh, misinformations gonna spread and it's gonna be really hard to tell what's real and fake, but I think everyone's on high alert right now because of AI being in the media, being everywhere. So you're thinking about, well, did AI create this? Which is probably what spawned on that thread, uh, as well, because everyone is on high alert thinking, well, you know, am I, is this a fake? So I, I think humans are just gonna adapt mostly to that idea of, of fake content or, or misleading content that's out there.
Chris Sharkey (00:53:25):
It might not be the worst thing because it's good to question the sources of the things you read and look into it more deeply. So it isn't it, I don't think it's a bad thing that people are, are looking at the veracity of what they're reading.
Michael Sharkey (00:53:37):
So the last thing I wanted to talk about today was this idea. Uh, so the, just to give you some background on this, there's this open philanthropy AI worldviews contest, and it's an essay competition. And the, the goal of the contest is to surface novel considerations that could influence open philanthropy's views on AI timelines and AI risks. And they posed the question, uh, around, you know, will transformative agi, will we have transformative AGI by 2043? So that's in 20 years time. And uh, I'm talking about a specific paper, which for those watching I'll bring up on the screen, uh, this one is on saying transformative AGI by 2043 is less than 1% likely. And this is a fascinating read, I'll link to it in the notes. And it really brought me back down to earth in terms of what we need to do to get to some agi or how we all imagine, uh, things to play out.
(00:54:47):
And what this paper argues is the, the bar and the difficulty is a lot higher than we might think in terms of achieving agi. And they talk about things that I had never really considered, like wars breaking out and slowing the development down, potential future pandemics regulators just stopping certain areas of advancement as we get closer to a g i also, financial depressions coming into play that could also slow it down. So they look at a number of different factors to conclude, uh, you know, the likelihood and they forecast it by having different events. So for example, they forecast, we invent algorithms for transformative agi and they put a forecast of 60% on that. And then they extrapolate based on all their different forecasters that there's, the joint odds is 0.4%. So very low in the next 20 years, but I've got some highlights in here I did want to call out.
(00:55:44):
So they talk about, uh, we must discover fundamental algorithmic improvements enabling computers to perform brain-like functions. And this one is super interesting. We must quickly figure out how to train computers to do human tasks without sequential reinforcement learning. And in the paper they talk about the fact that it takes humans 20 to 30 years to specialise in something. And we do that with sequential reinforcement learning. So we're, we're trying things, we're failing, we're learning lessons, we're being rewarded when we get things right. And so that sequence or sequential reinforcement learning takes a long time. And you've also gotta have all of these perfect experiences in some format to train a potential A g o I mean,
Chris Sharkey (00:56:28):
But I mean, I've got two counters to that one already. First one is, well, computers are a shit load faster than humans are at learning and they don't get tired. And uh, so even if you had to train 'em sequentially, which I don't believe you do, couldn't you just run 'em at a mass massive?
Michael Sharkey (00:56:45):
Well, their argument is what do you train them on? Because to train them, you've gotta give them high quality training data. There's not there, like, there's no high quality training data on ways that we reason and think, uh, because people don't write that correct. Like Norm, it
Chris Sharkey (00:57:00):
Says the goal is to, to replicate human intelligence. Why couldn't it be a different kind of intelligence that evolves separately?
Michael Sharkey (00:57:08):
Well, I think when they say transformative ai, they're talking about literally having robots walking around on earth that are far superior to us in every way and can do anything we want for us. Yeah. Um, and they also speak about, you know, the, the computational construction of those robots would need to be cheaper than humans. So we've gotta figure out how to drive costs down in, in so many different ways in the next 20 years. Keep in mind it's a 20 year horizon. So they're saying that the cost of labor's gotta be less than $25 an hour, which, you know, across the whole world's pretty high. We've gotta scale cheap and capable robot bodies. And they talk about the reason we need to do that is you've gotta allow the, the AI to run experiments and do things in the real world, or it's not going to vastly outperform humans. Like, cuz it, it needs to be able to try these things rather than being they, they explained, it's like if you chained a child to a chair and just pump them with images and video about the world, like you wouldn't develop the best understanding. You need to go out there and physically touch and see and do things. Yeah.
Chris Sharkey (00:58:14):
Okay. I agree with that for sure.
Michael Sharkey (00:58:16):
Um, it
Chris Sharkey (00:58:17):
Seems like a lot of their reasons are related to money and, you know, world money and how much support gets put behind it economically.
Michael Sharkey (00:58:25):
Yeah. And also they talk about that they're not r like saying that there won't be huge transformations in that time. They give the obvious examples like self-driving cars potentially displacing all the driving jobs, robot bodies good enough to automate routine tasks. So like the, the example they use fulfilment warehouses or retail with nearly no humans, uh, language models as good as reading and writing as humans. I think we're pretty close on that front. Um, conversational a agentic, continuous agis just as intelligent as humans. But I think what they're saying is all of these advancements could come in that timeframe and be, uh, changing to society, but it still is not going to be this transformative agi I in the sense that it, it wants to kill all humans and take over the world. It's just going to enhance all of these areas and be very transformative.
(00:59:17):
But, uh, they argue they just don't think we're gonna get there, um, in, in that period of time. So really, really, uh, interesting paper. I highly recommend reading it. It's a, it's quite a long read, but it outlines all the steps to get there. And I think as when we speculate, like, like we do on this podcast a lot, you saw, I, I don't know about you, but I imagine one day this thing is just like clicks on and then just starts, you know, uh, trying to infiltrate different computer systems and do all sorts of stuff like that and then figure out how to, how it can manufacture itself into a robot. And that's the sort of dream state I have with, with this getting away. But I think this paper sort of brought me back down to earth. I think we've got a while yet, and in that time we're gonna see huge transformation over the next 20, 30 years. It's gonna be probably net positive.
Chris Sharkey (01:00:14):
Yeah. And it's probably something that will gradually take bigger, bigger role in our lives across everything. And so when these advancements do come, it, it'll sort of be the next logical step rather than some massive special day where it just takes over. However, I was curious, do they mention in there, uh, the idea of the AI building its own, like, you know, crossing our intelligence level at somewhere along the way and then making all the remaining innovations itself?
Michael Sharkey (01:00:43):
Yeah, that is, that is, that is an argument in, in the paper. I think they keep in mind they don't think we're not gonna get to this state. They think it's fairly inevitable, but they're saying that not by 2043. So a lot, you know, a lot of these people now are saying it's like a five year, 10 year horizon. Yeah. Um, yeah, they're just saying that like there's, there's a lot to play at. There's so many infrastructure, there's breakthroughs, there's hardware, there's software that needs to be written and all the things that we're seeing take place now. So I still think we're on a path towards something of this nature, but, you know, maybe it's, it's quite far off in the future than we once saw.
Chris Sharkey (01:01:22):
Yeah, yeah. I see what you're saying. And it sort of comes back to what we often say on here is like you talk about the emergent behaviour and things, but then when you actually try to do it and try to use it with current tools, that doesn't always match up to what we sort of fantasise and and talk about in abstract terms.
Michael Sharkey (01:01:37):
Yeah. And they're, they're talking about training in Aion every single task that a human can do. They, they say, but not in 2043, even if we developed AI systems that were just as capable of learning and generalising to arbitrary tasks as humans and well in advance of 2043, we would miss a 2043 deadline to perform a wide variety of complex human tasks because the way we train humans to perform these tasks involves 20 to 30 years of sequential reinforcement learning. So Yeah, I i, it look, it's one perspective. They, they may not be right. Maybe things get here sooner, but
Chris Sharkey (01:02:11):
Yeah, I mean, the only bit I disagree with in that statement particularly, and I haven't read the paper, so you know, it's a hot take kind of thing, but is that I just don't think AI's gonna learn the same way humans do ultimately when it, when it's training on tasks. Like, I don't think if it wants to become a doctor that it's gotta spend 20 years in school, um, and and specialising the way a human does, I think it'll be able to do it in a different way that's much faster than that. Yeah. And maybe that's, they give,
Michael Sharkey (01:02:37):
They give really good examples though, how with robotics right now, when they use the, the deep learning methods to try and train a robot to work it, it basically fails. And Boston Dynamics have invested in, you know, in figuring out how to walk, walk, uh, make a robot walk through trial and error, but the actual models don't, you know, they still can't get a robot to work with these, with these models. Um, and so they're saying we're just not as far advanced as, as we, we think we are, but I think what you were alluding to before is, you know, is there a breakthrough where this thing figures out like a better way to train or learn that skill and then rapidly learn those skills?
Chris Sharkey (01:03:21):
Yes, exactly. So yeah.
Michael Sharkey (01:03:23):
Yeah, maybe there's some acceleration path, but 20 years does seem like a pretty short period of time to have, you know, the world ending or any of these other prophecies come true.
Chris Sharkey (01:03:32):
Yeah. Interesting. I mean, it's a interesting, uh, approach to solving it, like limiting themselves to 20 years like that.
Michael Sharkey (01:03:41):
Um, so the last thing I wanted to cover today, , which I said that was it, but there's one more thing, um, is a mi is AI making us sad drunks, um, was an article that was Pu published. And I know that all of these advancements and all of this news, uh, is definitely, I, I don't know, some days I'm really anxious about, I'm like, oh, everything's moving too fast, I can't keep up. And so this article talks about, uh, this AI hype is enough to drive you to drink, uh, this ai Hi, AI hype is enough to drive you to drink and lose sleep. Don't, don't take our word for it. These eggheads claim to have approved it. Employees working with AI software to complete task at work are more likely to feel lonely, which can drive them to insomnia and drinking according to a paper published by the American Psychological Association.
Chris Sharkey (01:04:34):
Well, I love the, the, the, the quality journalism. These eggheads Yeah. , these bloody loser researchers. It's
Michael Sharkey (01:04:41):
Kind of why I trust it more . But yeah, they said employees working, uh, with AI software to complete tasks feel lonely. They're spending way too much time on their computer, less time interacting with their colleagues. Although chatbots mimic human communication, it is not typically meaningful and users do not receive the feedback associated with talking to other people. So, yeah, I
Chris Sharkey (01:05:06):
Mean, that's fair enough.
Michael Sharkey (01:05:08):
I wonder if this just feels like, you know, in the future, the more we work with ai, the more depressed we become because we feel like we don't have a purpose. You know, it's just doing the work and we're kind of like massaging it to do the right thing or monitoring it.
Chris Sharkey (01:05:23):
I imagine there's parallels to when they first invented the, you know, the mechanical loom and things like that where people are like, oh, well I'm not doing the sewing or whatever it is anymore. Now the machine does it, therefore my life is meaningless. I could definitely see how that could happen in certain professions for sure. But I mean, that's been happening for years, hasn't it, with the automation of machines and, and you know, like, uh, Mr what's his name in Charlie in the Chocolate factory where the toothpaste lid pudding on machine replaces his job? It's the same thing happening again. Yeah.
Michael Sharkey (01:05:53):
I, I don't think that all of these, every time there's a huge tech technological advancement, this happens. People come out and say, you know, we'll have no jobs, we'll have nothing to do. And then more jobs are created and we have plenty more to do. So I'm, I'm sure that, you know, at least the next 20 years according to that paper, we're gonna be fine and, and
Chris Sharkey (01:06:13):
Just become a real, and everyone can just sell each other real estate all day. Yeah. ,
Michael Sharkey (01:06:19):
Unless there's like a really good AI robot that's doing that as well, I mean, you could easily replace them with robots. It'll
Chris Sharkey (01:06:24):
Probably buy it all up or something like that. Alright,
Michael Sharkey (01:06:27):
That's all we have time for this week. If you like the show, please consider subscribing, leaving a comment, liking, leaving a review, all that great stuff. We'll see you next week.