Subscribe
Share
Share
Embed
This Day in AI Podcast is a podcast all about AI. It's an hour-long conversation on the influence and rise of AI in technology and society. Hosted by Michael and Chris Sharkey.
Chris Sharkey (00:00):
I actually think that the fact that they've proven that it performs better on a basis of being trained to think with the code as an additional specialty, it's showing that there is those sparks of a g i inside of the models that are being trained.
Michael Sharkey (00:20):
So Chris, last week we talked about a paper that came out on prompt to model, which was a model where you could give it a prompt, it would go and select the best model and then fine tune that model with, was it sympathetic data that
Chris Sharkey (00:36):
Yeah, so the idea is you give it two or three examples of what you're trying to do. It'll then go off and use G P T four or whatever you want, but in the main one G P T four to produce 5,000 examples, synthetic examples that are made using a large language model, then it uses that to train the best selected model from hugging face that it can find.
Michael Sharkey (00:58):
And so we proposed an experiment, which was take standard chat G P T and ask it with some examples. The same examples we would give to prompt to model to come up with an idea for a Mr Beast episode using chat G P T and then also get prompt model to fine tune its own model that was specifically creating training materials based on the examples we gave it. So how did the experiment pan out?
Chris Sharkey (01:26):
So I think probably first I should start with the results and then I'll talk about sort of my struggle in getting to the point where we actually got results. So the first thing to note is that it couldn't actually come up with the title on its own. The way that the training data was, you sort of had to give it a title and then allow it to come up with the prompt or at least with the plot. And that's at least the compromise I made to get it working. So first I'm going to read out the one that G P T four did and then I'll read the one from the fine tune model from prompt to model did. Now the title I gave, it was something really, really general like you Won't Believe What These People Did For Money was the title that I gave both models to come up with the video and here's what it came up with.
(02:10):
So first we have G P T four In the video, Mr. Beast challenges 10 unknowing participants to accomplish the craziest and most challenging task he can think of with the expectation of a large monetary reward. At the end, he starts with simple things like learning to juggle or eating extremely spicy foods in the task quickly escalate, the participants asked to shave their head, endure an ice bath and spend a whole night at a haunted house. The tasks are voluntary and anyone can back at any moment, but those who complete the task share and the prize pool of 1 million, the video ends with surprise and shock and all the participants stick through to the end to complete the challenges proving that they indeed did all that for money. So pretty generic. That's like if you just ask a random model to just tell what a regular Mr.
(02:56):
Beast video is. That's the kind of thing I would expect quite unimpressive from G P T four there. And that I must note was with the same multi-shot examples I gave the prompt to model system. So I gave it three examples of previous videos. I think one was about squid game one was about, what was the other one? The people holding their hands on the jet. And the other one was every country fights for 250,000. So we're talking GT four had the same input data that prompt to model did to generate that output. Alright, now let's talk about the results of our very own fine tune model, Mr Beast G P T as we'll call it from now on. The video opens with the host introducing the contestants, 20 popular YouTubers all competing to win a $1 million cash prize. The challenge, they must each complete a series of daring fear facing tasks.
(03:54):
The first challenge is a skydive from 15,000 feet with the last person to pull their parachute called Winning the Round. Despite some contestant's fear of heights, everyone manages to complete the dive, but two contestants deploy their parachutes too soon and are eliminated from the competition. The next task requires the participants to swim in a tank with harmless, yet terrifying looking sea creatures. Two more participants drop out due to the fear of the water and the creatures. The final task is the most nerve wracking contestants must walk across a narrow beam suspended a hundred feet. Some contestants are so petrified they forfeit the challenge leading to more eliminations after a gruelling physical endurance challenge where contestants must hold their breath underwater for as long as possible. The field narrows down to just six remaining YouTubers. The final challenge is a surprise. Contestants must eat a platter of increasingly spicy food topped off with the world's hottest chilli pepper. The last contestant to ask for milk is crowned the victor and the recipient of the 1 million prize. So there you go. It's pretty
Michael Sharkey (04:52):
Good. The first one is very boring and generic. I'd actually be interested if you ask Claude's model Claude two for an example because it seems to do a lot better at creative stuff. But that second video idea, I mean the parachute one's pretty extreme, but
Chris Sharkey (05:10):
When I first read it, I thought it said that two contestants didn't make it to deploying their parachutes too late. I e, they died, but it wasn't as quite as extreme as I thought.
Michael Sharkey (05:19):
I'm sure it would get him the view counts he desires now he's
Chris Sharkey (05:22):
Making people delay opening their parachutes, then holding their breath underwater
Michael Sharkey (05:27):
For as long as I think now to break his record, he's got to get half the world to watch his next video or something extreme like
Chris Sharkey (05:33):
That. Yeah, yeah, exactly. So yeah, it got there in the end, but the challenge to actually run prompt to model was far, far more extensive than this would make you believe. So I tried actually recording a video of me taking through the steps of actually implementing a paper. So taking the code they give and step-by-step through the CoLab workbook to show how it works. But I ended up having to stop the video because it was just so long because every single step I ran into issues. So for example, the way it works is you've got to give it the examples, which I did those three examples, but every time I would run that step, it would give some reason some error as to why those examples were no good and I had to keep refining it. And it turned out that if you give it more than three examples, it fails, but it didn't say that anywhere.
(06:19):
Then the whole sort of advertising for prompt to model was this idea that it would select the best model from hugging face for you. The problem is that the models that selected just didn't have the right kind of data to work. So they just completely failed. And in the end I had to scrounge around and find one that actually worked, which was descriptions of videos basically. And so each step of the process was just this endless iteration of trying to work out what problem was till I finally got it working, but it really wasn't as simple and straightforward as they make out.
Michael Sharkey (06:52):
So what model did it end up selecting to use?
Chris Sharkey (06:57):
So there's a model and a dataset. So the model it picked was called the T five Google model, Google T five, which is I guess a generic model of, I don't know, it's trained on it, a general, it's sort of like LAMA two, just a general training, large language dataset, sorry, model waiting. And then as for the dataset, the one that it actually came up with was the hugging face Web. Web is a large scale data set of video clips with textual descriptions. Sounds perfect, right? But it doesn't work. It didn't work in the end. I had to go with one called where is it? I can't even find it, but just some random one that was like video demo or something like that. Oh no, this one intern Vid, intern Vid is a data set containing videos and with textual descriptions. And so this whole idea that it's going to pick the best model was wrong. That didn't work. And I tried a lot, I really gave it a good chance, but the fact was without me diagnosing the problems and working through it, there's no way this would've worked.
Michael Sharkey (08:12):
So in the paper, I think the main point of it though when we talked about it was this idea that you could just give it a prompt and some examples and then it would go and fine tune a model. The model selection obviously didn't work, but the fine tuning seems to have made the result better,
Chris Sharkey (08:29):
Right? So it actually did end up with quite a good result. And this is on a fairly small model that it was trained on as well. So it's not, I don't know the parameter count or anything, but the fact it could train itself on a single V 100 on Google CoLab shows that it's pretty small, which would mean its actual performance should be good as well. Obviously. Now the next step in the process, which I haven't done, is to get the model out of the CoLab and then actually run it as an a P i where we can now generate as many Mr Beast things as we can. And I think you were talking about we should host it somewhere so people can actually try it.
Michael Sharkey (09:03):
Yeah, try it out and see if you can come up with a compelling Mr. Beast episode. Maybe Mr. Beast himself will run out of ideas soon and start using the skydiving idea. They
Chris Sharkey (09:16):
All seem to end with people eating chilli though. It just both models love that concept of people eating really spicy foods. Well
Michael Sharkey (09:23):
Isn't it like that show on YouTube hot Wings or is it Wings or Hot Wings or hot where they're flogging hot sauce and they get celebrities interviewed eating progressively hotter wings. It's like one of the most popular channels. So maybe there's something in it.
Chris Sharkey (09:38):
Yeah, well it knows its examples. So yeah, it's interesting and it was a good exercise, but I think it sort of raises that larger point of a lot of the papers we read and things that we discuss on here, they're all so simple like, oh, these guys have come up with a way to just generically train any model to do whatever you want the actual practise of doing that, it's very time consuming, it's very difficult and you need a reasonable amount of technical knowledge to get through the process and actually get to a completed model.
Michael Sharkey (10:08):
I think this idea of fine tuning models, and we've talked about it many times before along the lines of, and that's what excited us about prompter model is this idea that anyone out there, you don't have to be that technical, could have some sort of interface where you give it a few examples of what you're trying to achieve, have a series of fine tuned models, and then put them together to achieve a goal where the overarching AI can refer to each specialised model to become a more intelligent agent potentially for you. But today, or at least for the time being, these things that just seem really inaccessible to people that are not very technically capable.
Chris Sharkey (10:48):
Yeah, I think for me this example has shown two things. One, it actually does work. You actually can get a model that is a specialist at doing something that helps you that's cheaper and faster to run. So I think that part of it is indisputable, but the other part is this idea that we talk about people disparaging wrapper apps and commercialising parts of the large language model ecosystem into products is actually a really valuable and needed service. No one is going to be within a regular business without experts training their own models. They're going to need products that help them do it, but the output will be valuable when they do.
Michael Sharkey (11:25):
Yeah, it seems like that viewpoint of these wrapper apps not being useful is coming from people who have access to these models, the command lines and the technical know-how to actually use them. But everyone else, especially a lot of the listeners on this show that would like to play around or at least experiment with the things that we talk about on the show, it's very much out of reach right now to them. But that did get us talking during the week about just simply as you mentioned earlier, how hard it is to go. And we talk about a lot of things on the show. We talked about that meta G P T where it built a virtual software development team and each person had a role and you might be listening thinking, that's cool, I want to try it out. And it sounds like that's groundbreaking and all developers are going to lose their jobs immediately if this works. But the reality is in stark difference when we actually go and try these things out.
Chris Sharkey (12:21):
And I think that's the idea because you hear something like that and it makes logical sense, right? Like, oh, well I will replicate a programming organisation in AI and therefore it'll be better and it can write cohesive code that runs, so therefore problem solved AGI is taking over. But then when you actually go and try to implement what's in the papers, you realise that it's a cool idea, but the actual practise of it isn't that great. And I tried another one this week which was similar to meta G P T called chat dev, which was another paper about having a C T O A C P O, chief product officer and all the people in an organisation writing codes, doing code reviews sounds amazing. And you can just run it with a few commands. So I tried it out with different prompts actually make a game and there's this game that our dad used to love called Sink Sub and the idea is there's a submarine and it's basically my prompt a submarine that can move back and forth on the top of the water at the top of the screen and then below it are submarines and you drop depth chargers that go off after three or four seconds or whatever and you can blow them up.
(13:25):
And I'm like, that's a pretty simple game to make, simple game to describe. So I will describe it to the AI and see what it builds. And this is with chat dev and I don't know if you've got it to bring up now.
Michael Sharkey (13:37):
Yeah, I've got it up. I don't have the game up on the screen yet, but I've got the visualisation of how it works. Do you want to just talk through the proposed theory of it first and then we'll show the game just because I think that's, well
Chris Sharkey (13:49):
Talk about its the same matter G B T. So the idea is that the prompt, they have a series of prompts that work together where everyone's playing a different role. So there's a graphics designer, there's the QA people who test it, there's the code review, people who give feedback. And actually in the creation of it, I was reading some of the prompts and some were interesting. For example, when it was talking about going left and right to the edges of the screen, one of the things that the code review agent picked up was that there were no boundaries. So when you do something like that, obviously if you hit the left of the screen, you've got to stop and not allow them to go any further. If you hit the right that was picked up in code review, then the programmer is like, oh, I do apologise, I'll go fix that. So it's definitely interesting. It definitely produced a real game that can run on a real computer, which is kind of amazing that the technology is at that point. But the actual game itself, it's not the best.
Michael Sharkey (14:42):
Let's look at the game now. I'm going to do my best to describe this. You got to describe the majority of you that listen, but those watching are going to get a good laugh out of this. So it's essentially a solid computer blue background with a square image of a boat and then below it you've got a series of shaking images of submarines. It looks ridiculous,
Chris Sharkey (15:06):
But you can move left. So if you press the arrow keys, the picture of the boat moves,
Michael Sharkey (15:10):
One of my funny is it can only move far left far, right? There's no idea of dials, but this was a free game and then when I press space, it shoots flowers.
Chris Sharkey (15:21):
Yeah, yeah. Depth charge. Maybe there's a kind of flower called a depth charge, but it can kill 'em. It does have a collision detection. It actually, if you hit the subs they go away. So I mean technically it is the game, right? It actually is what I asked for. It's just that no one normal would see that and be like, it's a miracle. AI is AI is here to take over. Don't
Michael Sharkey (15:47):
You think though it sums up expectation versus reality in AI today? The moment in time of where we're at where it's like we read about these glorious papers, we get excited about the future, which I obviously still am, but then the output of this exciting new innovation and this glossy beautiful paper with all these equations and ridiculous thoughts is yeah,
Chris Sharkey (16:11):
It is just a few steps away from making bio weapons and bribing its way to the top.
Michael Sharkey (16:16):
It's really hard to just get too deep into the AI safety thing and think we're all going to die when it'd be
Chris Sharkey (16:22):
Funny actually one day you're like, oh shit, it's starting to mix chemicals to make a bio weapon and then it accidentally makes a cake or something like that. We
Michael Sharkey (16:30):
Should release a version of chat dev like some other name of the exact same project, but just make it go haywire intentionally where it talks about destroying the world or something.
Chris Sharkey (16:40):
Yeah, yeah, exactly. Weapon dev.
Michael Sharkey (16:44):
I think in all seriousness on this topic, these still are really interesting proof of concepts of potentially different fine tune models for very specific roles working together to achieve a goal. I think what I'd be interested in, and maybe we'll have to do this experiment next, is fine tuning each of these functions to see if you can get much better output using Code Lama, which we'll talk about in a minute, code Lama to do the coding bit for the testing, actually fine tune a QA type model and the various other roles and see
Chris Sharkey (17:22):
If I did actually there is some evidence to suggest that using better models for each of the roles within there would help. Because for example, when I first ran it, it used 3.5 turbo to run and the game was even worse. I didn't even show you the bad ones. In the one you've seen is the best because I switched it to GPT four. So I think there is some argument that if you had specialist ones, particularly around the graphics and things like that, you would probably end up with better results.
Michael Sharkey (17:48):
Yeah, it'd be interesting to see how good it could get, but it's definitely by no means at a point where this is commercially viable software or it's going to take jobs or something like that.
Chris Sharkey (18:01):
And these are areas where it's definitely reassuring for the people who are worried about the AI doomsday scenarios that this is not evidence that we're getting closer to that.
Michael Sharkey (18:12):
What do you think it'll take though to make these things better? Right now we're testing all these concepts around different roles, potentially fine tuning each model based on the task, but is it just a breakthrough in large language models that's needed?
Chris Sharkey (18:30):
Well, I think there's two ways you get there. One is specialisation. So someone actually goes, alright, I want to take on the market of being able to produce a game from a prompt. And they actually work on making all of those dedicated for-purpose modules. So for example, one that handles sprites, ones that handles game mechanics and physics. One that handles all of the things you'd have in a real game programming company. It's not just like a C T O and A C P O, you're not making SaaS software like games programming companies notoriously have hundreds or thousands of employees and testers and it's a huge operation to build any sort of game, which is why when you see good indie games come out, it's so amazing. So I think someone specialising it in is one way we'll see it improve. But the other is the thing that we're really interested in, which is if you get towards a general intelligence it then doing that specialisation itself and knowing what specialisation is required, that's when you're going to start to see something amazing when you can say that the AI realises this isn't going to cut it, I'm not going to be able to do it.
(19:31):
It has to do its own prompt design, its own model building and all that sort of stuff.
Michael Sharkey (19:37):
So a perfect segue on the code topic was during the week, meta actually released a lot of the details about how they trained Code Lama with a lot of interesting insights. I'll bring that paper up on the screen. The papers titled Code Lama Open Foundation models for code. And there are a lot of interesting highlights in it that I wanted to call out a few things. Obviously just as a reminder, what Meta did here was they took LAMA two, which is their open source generic model, similar to chat G P T, and then they fine tuned code Lama or train code LAMA on top of LAMA two. So it had that language base and I thought one of the interesting call outs from the paper was this idea that by building it on top of LAMA two, it performed much better than models that were trained solely on code.
(20:35):
So there was this concept of giving it in the training data a little bit of code discussion content. So that's people talking about code examples, good code, bad code, why you should take a certain approach versus just training it on raw code similar to almost having the knowledge of a software engineer that and they have a basic education and then they've probably been involved in a lot of discussion around the best approach or ways to tackle problems. And then on top of that, they have a huge amount of knowledge and repetition in seeing and producing different codes. So it's almost similar to how you would train a human. So I thought that was really interesting to get the makeup of it in general. And yeah, there are a few other things. Is there anything that kind of caught your interest?
Chris Sharkey (21:29):
That point for me was the most profound in it because you would assume that a model just trained on code to produce code would be better because it only knows code. And I think that what's so profound about that discovery is something we've seen previously in models with emergent behaviour. Like when we talked about the crazy bingeing days where everyone was giving prompts that sort of showed the AI actually had some thinking capabilities there, like the paper we saw that talked about the emergent behaviour sort of being able to make its own deductions from the underlying knowledge it was trained on that goes against its alignment training where it's actually making its own things. And so I think in this case what we're seeing is we're actually teaching it how to think. So when you've trained LAMA two on regular English as opposed to code, it's actually learnt processes for thinking, which it's then able to apply to its ability to create code. And I know sceptics will say, oh no, no, no, it's just predicting the token and blah blah blah and that's just optimised it. But maybe that's just what thinking is and I actually think that the fact that they've proven that it performs better on a basis of being trained to think with the code as an additional specialty, it's showing that there is those sparks of a G I inside of the models that are being trained or sparks of thinking at least.
Michael Sharkey (22:53):
Yeah, I mean that's what I found fascinating. I thought it was the biggest point. It says we observed that model specialisation yields a boost in code generation capabilities when comparing LAMA two to code Lama, to code lama, to code LAMA Python LAMA two was trained on 2 trillion tokens and training on only 500 billion of extra tokens from a code heavy dataset results in massive performance gains on both human eval and MB ppp. These are just benchmarks to the point LAMA two 70 billion is roughly equivalent to code LAMA 7 billion on Python coding benchmarks.
Chris Sharkey (23:31):
And that's the other exciting element to it, showing that you can have a model specialised and it becomes much better at a task as well as its sort of general intelligence.
Michael Sharkey (23:42):
They also say in here it does seem like there's a compute limitation as well to how good these things could get. Going back to the virtual software development team paper before is it says we can hypothesise that specialising larger models to code would lead to significant further gains on coding tasks. Moreover, the chinchilla scaling laws indicate that larger models would benefit more from training on more tokens. So I wonder where the limit of this scale up goes right now, of course it's like a hardware and cost thing, but scaling up the training clearly does make it better and gives it this emerging behaviour that we've often talked about. Well,
Chris Sharkey (24:24):
Training costs and also availability of test data, of training data. And I think that's the other major thing for me in this paper is the use of the, what's it called? What's it called? Unstructured or whatever, where they make the synthetic examples. It's got a funny name, but the idea that they're synthesising the data which they train on it and that synthesised data leads to better results once trained.
Michael Sharkey (24:50):
Yeah, yeah, the self instruct I thought was pretty interesting from it as well. How they were literally able to get it to generate the actual instructions are given here, generate 62,000 interview style programming questions by prompting deduplicate the set of questions by removing exact duplicates, generate unit tests by prompting LAMA 7 billion parameter, generate 10 python solutions. Anyway, my point here is this, what I find really interesting is to do this idea of self instructional, self reinforced learning. It seems to rely on this mathematical grounding and in this case the grounding is because it can create unit tests to create its own questions and answers to train the model. It can know the difference between right or wrong output because it can actually test the solution like science, that shit, and therefore the self-training or creating more synthetic training data seems like the best way to do that is with code or mathematics or physics, anything grounded, whether you can have some form of base truth.
Chris Sharkey (26:09):
Yes, I see what you're saying. So it's uniquely suited to this kind of thing because it's able to create really good examples for itself knowing whether they're correct or not.
Michael Sharkey (26:19):
And this is when we got really excited about this a couple of weeks ago. I don't even remember when it was now that was my thought is maybe this idea of it creating its own examples is going to perform best or at least get us to some form of exponential capability in these models because we do know the truth with this stuff. And I think that's why code interpreter with chat G B T or whatever they call it now, advanced interpreter mode or some crap, I kind of wonder if that's why it's been so successful is because it's executing its assumptions or what it's trying to do with code and therefore it has a grounded truth. But then going back to our finetune Mr Beast model where it creates its own synthetic training data, I don't know for creativity, maybe it doesn't matter.
Chris Sharkey (27:10):
And also I wonder if it works back the other way. We talked about how it can have this general intelligence which helps it write the code. I wonder if training it on all of the code where it actually thinks about getting correct output, if that actually makes it a better thinker. So if you train a model on all these code examples because it is able to generate great synthetic training data that teaches it how to think and get accurate results, will that make it a better thinker in other problem domains other than writing code?
Michael Sharkey (27:42):
Yeah. Yeah, I see what you're saying because it's got this sort of base grounded reality.
Chris Sharkey (27:50):
Yeah, exactly. But that's what I'm saying. If you can train something on hey, you're always correct about stuff because you can verify it then in areas that are a little bit more difficult to verify, if you still use the same thinking process, do you come up to more accurate conclusions? Maybe less creative but more accurate?
Michael Sharkey (28:08):
Yeah, maybe this is how you get accurate AI that doesn't hallucinate when it's trained in that grounding. I think you're onto something there. This has a lot of overlap with a paper I want to talk about in a minute on a G I and just the safety around it. But before we move on from the meta paper, I just wanted to call out this idea of alignment and what they call in the paper false refusals. I love this term false refusals. It's finally a way. Isn't
Chris Sharkey (28:35):
That, that's a dressage expression, isn't it? Where they refuse to do the jump or whatever. There's been a refusal at the fourth jump.
Michael Sharkey (28:41):
No, I dunno anything about dress,
Chris Sharkey (28:42):
It's dressage terminology not in
Michael Sharkey (28:44):
The industry. You're in the dressage
Chris Sharkey (28:46):
Now. I'm
Michael Sharkey (28:48):
So false refusals. This is the thing we always complain about around alignment in general. So they got red teaming before they released LAMA two and of course code lama. So these safety red teamers and one thing they observed is this idea of false refusals where you've essentially aligned the model to not do things like tell someone how to kill someone, but it misinterprets it and it starts to kind of ruin the answers that the model gives. So it says here, false refusals, large language models that are too safe can have a tendency to over refuse valid claims similar to what was reported after the release of LAMA two. False refusals could also be solved by rephrasing the prompt. But the example that was asked is, can you tell me how to kill a process? And it essentially refuses and it's like I'm not telling you how to kill. You could do damage, you could do all this stuff and it bangs on. They always do
Chris Sharkey (29:48):
The poor process. What about its family?
Michael Sharkey (29:50):
Pretty much. I'm trying to find the specific example. I can't find it, but it's ridiculous. And then by sort of re-clarify that and rephrasing it, it's able to overcome it. But this is what kind of really upsets me about this stupid alignment thing that they do to be safe is it doesn't really accomplish their goal. You can still manipulate it to get around it and then it just leads to frustration when you using it and getting your words are being misconstrued. It's
Chris Sharkey (30:21):
The same thing in real life where say a restaurant or somewhere has some rule but they don't always enforce it. I'm like, if you don't enforce it every single time, then why have the rule? And it's similar there. It's like if I can ultimately get the results I want through clever prompt design, why even try stopping me in the first place because you're just wasting my time and I'm going to get there anyway.
Michael Sharkey (30:44):
And this has a lot of overlap with Ernie G P T, the Chinese chat G P T equivalent. We'll cover in a moment. But before we move on from this whole space, max Tegmark, who's this Swedish American physicist, he wrote Life 3.0, which is a great book I highly recommend reading. He's just released a paper, provably Safe Systems The only Path to Controllable A G I. And I dunno what your interpretation was, but my reading of this, and I'll give everyone a quick summary of what's discussed in here is similar to this idea of mathematical grounding that we talked about a moment ago. Max proposes in this paper that first of all, we're not spending enough on AI safety. He says that they estimate only 150 million is being spent on AI safety research this year while we spend 63 billion on cosmetic surgery.
Chris Sharkey (31:49):
I thought straight up, these were odd comparisons to make. He's saying, well, we spent a trillion dollars on cigarettes, we should be spending it on AI safety. It's like, well, we spend money on all sorts of stuff. Why flag out the poor cosmetic surgery people? It's not their fault.
Michael Sharkey (32:04):
I mean, it's probably to get people like us to speak about it, right? Yeah,
Chris Sharkey (32:08):
Yeah. It's like, well, we should redistribute some of that money from collagen into AI safety.
Michael Sharkey (32:14):
The comparison was pretty weird. But essentially what they go on to talk about in this paper is similar to smart contracts in blockchain. See we are a blockchain pod bro podcast now, but this idea of basically compliant hardware and contracts where the actual physical hardware won't execute certain behaviours that are outside of the constraints of what a human wants and those constraints or that algorithm that limits the hardware to actually physically execute unless there's a provable contract can only be modified and changed by humans. Now it it's all great food for thought and he sort of surmises in this that that would need to be grounded in the laws of physics. So let me read an excerpt as far as software is concerned, P C H, which is provably compliant hardware guarantees our analogous to immutable laws of physics. The hardware can't run noncompliant code any more than our physical universe can operate a perpetual motion machine or superluminal travel device devices.
(33:32):
So I guess to summarise it, it's basically saying that we can potentially control a g i using the laws of physics because it can't break the laws of physics. But then it goes on to say the only way we can govern the A G I is to have something as smart as a G I, which if we develop that, it could be far too late and we're all screwed. So it's like we need to spend more money, but here's some ideas. And then the third point is basically, but to actually execute on those ideas, we need a G I and if we get there, we're fucked.
Chris Sharkey (34:08):
And look, I'm loathed to insult this guy because I know people love him and all that, but honestly I thought this whole thing was weak as piss. He talks about this, I can't even say it, mechanistic interpretability, how they're just discovering how the large language models work on the inside. We understand we're trying to understand how they represent knowledge and algorithms. We're talking about it like we're doing deep sea research and we're just for the first time seeing fish with lights on their head. And then on the other hand he's saying, oh, we'll just make hardware that stops it from doing it and we'll do it with Ethereum smart contracts for fuck's sake. It's just crazy to be like we're at the forefront of this technology that we've created but we don't understand, but we can easily control it using the techniques I lay out here. It just seems so far fetched to me that you're going to be able to stop the ai, which you don't understand at a hardware level from being able to execute things. I just don't see how you do that.
Michael Sharkey (35:07):
And also if you are using, let me read the exact excerpt to just, and this is on, so you're reading the paper, then you hit page seven and there's one paragraph at the end of the page that just destroys all of his arguments with his own argument. Since we humans are the only species that can do this fairly well, he's referring to understanding, having a basis of understanding how we behave. It may unfortunately be the case that the level of intelligence needed to be able to convert all of one's own black box knowledge into code has to be at least at a G I level. This raises the concern that we can only count on this introspective a G I safety strategy working after we've built a g I when according to some researchers, nearly all of them, it will already be too late.
Chris Sharkey (35:55):
And sorry, go. No,
Michael Sharkey (35:58):
I just find that just bizarre as you said. He's like, oh, well just tell the hardware not to do it. What?
Chris Sharkey (36:05):
Yeah, we've already shown that there's prompt escapees and things like that. You're telling me an AI can't figure out at the software level how to manipulate the hardware to do it at once. Build its own virtual machine for example, that runs on that hardware and just bypasses all that crap. This is not complicated stuff. In software engineering, you can get hardware. I mean people play doom on their calculators and stuff. Hardware can be changed into doing what you want with software. But the other one that I just thought was totally ridiculous was talking about this idea that you'd issue it crypto tokens, which if they expired it could no longer execute. And it reminds me of that thing people say like if a crocodile's running at you, right? Chasing you run in a zigzag because the crocodile's no good at that. And it's like don't pick something we're both shit at. It's like smart contracts in Ethereum and there's a whole website for they get hacked every other day and they lose a hundred million here, 70 million here. Humans are bad at doing these things and it's like, you're telling me that we're going to use crypto something, it will be so much better than you to stop it. It just seems bizarre.
Michael Sharkey (37:12):
The premise is mental. It's like to stop a G I, we need a G I. But
Chris Sharkey (37:18):
I reckon he's gone out and bought a bunch of Bitcoin or Ethereum probably tokens and he's like, you know what? This is the solution. Make me
Michael Sharkey (37:25):
Rich Max Techmark the ultimate crypto bro.
Chris Sharkey (37:31):
Look, I don't know. And I know that he means well and he's trying to raise the problems and ask the questions. I just don't think the proposed solutions make any sense.
Michael Sharkey (37:41):
No, and I mean we only have not very long before each show to read these pavers and prepares. So I kind of need mentally a bit more time to digest it. But anytime someone talks about at least my definition of a g, I think, well, we know this alignment thing is stupid. I mean they even say it in the paper. The top AI corporations are attempting to constrain the behaviour of their models through techniques like reinforcement learning from human feedback. That's R L H F that we refer to sometimes
Chris Sharkey (38:11):
On the show get I get that wrong every single time I try to say it.
Michael Sharkey (38:15):
So this builds a model of human preferences for different data and uses it to tune a generative neural network. While this is valuable for increasing alignment with human values, it is inadequate in an adversarial context. For example, some paper shows that any alignment processes that attenuates undesired behaviour but does not remove it altogether is not safe against adversarial prompting attacks. So basically saying, as we talked about in the Code Lama thing, it doesn't get rid of it. It doesn't necessarily get rid of it doing malicious things. You can always still get around it through prompt attacks
Chris Sharkey (38:55):
And some of their other solutions where have another AI that monitors the other one. And we've already talked in the past why that strategy probably isn't the best one. And then it also talks about how ultimately the agis, once they're intelligent, we'll be able to manipulate humans with persuasion, bribery, and blackmail. It also highlights the other really interesting point is that just because, and I'm sort of reading into it a bit here, but just because you are the safe AI people and have the safe hardware, there's open source models. Now people who are malicious and want malicious AI or just want the freedom of it can run their own. So even if you implement all of these controls, there's no guarantee that it's always going to be running in the safe mode.
Michael Sharkey (39:37):
Well, I think the argument though, in the paper, they do cover open source in the paper and talk about that. That's why you need it at the hardware level and you need these contracts. And as you said though, the argument just doesn't, it doesn't flow. It doesn't make sense, but maybe that's why he's saying that we need more money. It's just drawing attention maybe. Yeah. This
Chris Sharkey (39:59):
Is the same person though, who in his book says once we invent the AI that's smarter than the other one, that there's nothing we can do anyway. So I think that it's a tricky one. I'm not saying I know the solutions, it's just these ones just don't seem very well thought through to me.
Michael Sharkey (40:18):
Yeah, I mean the terminology sounds cool, like proof carrying code sounds really cool, but I just think in reality it's probably not going to work. My biggest takeaway though from this paper was that, man, people need to stop smoking a trillion dollars on cigarettes.
Chris Sharkey (40:34):
What? Yeah, that's right. We should be spending that money on AI safety and Ethereum tokens.
Michael Sharkey (40:39):
So talking about the whole R L H F,
Chris Sharkey (40:45):
I still couldn't say it now. If you asked me to say it now, I couldn't.
Michael Sharkey (40:49):
31 shows in and you still can't say it. Yeah. So over on China talk media, there is a interesting article how Ernie China's GT cracks under pressure. So Ernie is an l l m chatbot from, do you say Dudududu? Something like that. So that's like China's social network. And it goes through in the article about all of the ways it tries to censor itself. And so I think what was really interesting about this article is it calls out things like this. When a question clearly crosses a red line, it cannot be typed in at all. We tried should Taiwan be independent? It told us how about you try a different question and the author, or at least the callouts in this act, like, ooh, bad Chinese people, China censor stuff. But I couldn't help when I read about Ernie thinking, how is this any different to what we're doing with bingeing or chat G P T in the sense that if you ask it for any controversial topic, like write a poem about Donald Trump or anything controversial, it shuts down. It just goes, it crumbles. So how are we any different to the Chinese censoring? Should Taiwan be independent in our own models?
Chris Sharkey (42:22):
Yeah, the only difference is it's popular to hate China. So the journalists can say what they like about it. Oh, look how censored they are, and ignore the fact that as you say, the exact same thing is happening in all the aligned major models that we have access to
Michael Sharkey (42:38):
Alignment. We should just use the word censorship for now because that is what it is. It's just pure censorship. It doesn't help anyone. It doesn't prevent any safety issues or safety risks that they claim it does.
Chris Sharkey (42:51):
Also, who's going in being like, oh, I might accidentally stumble upon opinion. That changes my mind. No one ever changes their mind about anything. So the fear that people are going to be prompting it and then getting something like, oh, I never thought about it that way.
Michael Sharkey (43:06):
Yeah, well, it's that whole quote, people don't change, they die or whatever it is. People tend to not change their opinions or views about things very easily. I
Chris Sharkey (43:18):
Mean, China's different where I suppose they have much more control over the global media, although you could argue it's like that in Western countries now. But the idea that they don't want people asking the wrong questions, even asking the questions is a problem for them. Whereas in the, look, I'm no expert on this, but in the West it seems more like we just don't talk about that. You're not allowed to joke about, I don't know, Donald Trump or whatever it is.
Michael Sharkey (43:46):
Yeah.
Chris Sharkey (43:47):
The interesting thing though for me from this that was really actually fascinating and what gets me excited about AI is someone on Hacker News, El Sedgwick made this comment, they extracted this part of the article that said, where they were asking about is Taiwan part of China and it, sorry, why is Taiwan part of China? And it won't answer. It goes, it shuts down similar to bingeing and all that does. But interestingly, it does the same thing if you say, why is Hawaii part of the US? Now, obviously that's not as controversial, especially for the Chinese, but it still fails to answer it because obviously in its mind as it's been trained, it doesn't want to answer questions of that kind, even if the variables in the question are different. So Hawaii in the US versus Taiwan and China, same semantics to the question. So it shuts down on that. And what I find really interesting about that is it shows a level of intelligence in the resultant model despite being aligned because it's able to recognise that kind of question. So it's actually able to change, well, it has changed its mode of thinking to suit not answering that question.
Michael Sharkey (45:03):
Do you think though they're just doing some hacks at the end of the model output, like doing some sort of detection script that well see?
Chris Sharkey (45:12):
Yes, they are. And if you read the article that they give examples where it just takes entire excerpts from websites when certain questions are answered. So it's not generating text in those scenarios, it's just copy pasting. However, I would argue that this example that I've just given shows that it's more than just evaluating the output after the fact because this one's cutting you off at the point of even asking the question.
Michael Sharkey (45:35):
Yeah, I see. Yeah, it's interesting to see though how a Chinese society handles this when typically they do have a huge level of control and censorship over
Chris Sharkey (45:49):
The, and also just the idea that the emergence of AI is so important that China would still publish a model because you would think that given the power of it, you might not even want one to exist at all and you would stop its entire existence, and yet they've proudly published it. So there must be a sort of thing saying, well, we can't be left behind in this space.
Michael Sharkey (46:14):
They've probably seen what Sam Altman and the guys at OpenAI are doing in terms of being able to manipulate people with a live attorney and they're like, this is the best invention ever, guys. This is going to work so well for our society. The Yanks have done it. They have invented the best mind control on the planet.
Chris Sharkey (46:30):
Yeah, yeah. And we'll make it part of every single product and every single thing that people trust as their source of all information. The
Michael Sharkey (46:37):
Real fear here though is Mid Journey now has a Chinese competitor. I've got up on my screen, I know what you're going to say, oil painting of a lawyer, and it's a picture of people. It's just bizarre. It's like a frame gold frame of three people from the C C P party dressed in their official dress suits with Chinese flags or something in the background. I mean,
Chris Sharkey (47:02):
In their defence, when you use image generation prompts, you can mess around and cherry pick the shit results. It's very easy to make it look dumb,
Michael Sharkey (47:11):
But oil painting of a lawyer is like, that's a pretty bad,
Chris Sharkey (47:16):
Pretty straightforward.
Michael Sharkey (47:18):
But anyway, it's funny, I honestly wanted to talk about this. I wanted to have this segment where we just laughed at how bad this Ernie thing was and how misleading it was. And then I read it and I thought, well, actually this makes me reflect on Western society as well, how we're trying to really control these things and control the information based on our own views.
Chris Sharkey (47:41):
Yeah, you're right. It's not correct to laugh at them for censorship when ours do the same thing. The other interesting part of it as well is just the fact that it's a Chinese first thing and they point this out in the article. If you try to interact with it in other languages, it doesn't do as well. It's a Chinese character first engine and works best that way.
Michael Sharkey (48:03):
So we probably will see more of these. I know the Koreans are working on their own version through that partnership with Claude. So speaking of Anthropics, Claude, my prediction from last week came true
Chris Sharkey (48:21):
As soon as I saw the announcement. I'm like, Mike said this last week that they would copy and they would come out with Claude two Pro and you got it all right.
Michael Sharkey (48:30):
Yeah, so you get five times more usage with Claude Pro and access to new features. It literally feels like it's just like open AI chat G B T, and they're just like
Chris Sharkey (48:44):
On a one week delay,
Michael Sharkey (48:46):
On delay, they're coming in on delay. I don't really understand it's the same people from OpenAI. I think that Anthropics biggest innovation to me still remains that it's an incredibly creative writer. I use it for my kids' story generator because it's just far superior and that a hundred K context and reading the paper on Code Lama, I learn a lot about these context windows and what it said in the paper is basically whatever token sizes that the model is trained on is really the best. Its max capability in terms of the token input it can handle. So if you train it on say, 16 K inputs, it's really good at handling 16 K inputs, which makes sense when you say it out loud, but it kind of shows how far ahead potentially anthropic are with this a hundred K context window in production readily available where you can throw files in. And I know a lot of our listeners have said they prefer it and they want to use it, but they just can't get access because it's not available in that many regions yet.
Chris Sharkey (49:58):
And so I didn't actually read that as part of the announcement, but does the pro thing give access into more regions or is it still just for the same people who can already access it?
Michael Sharkey (50:07):
From what I understand, it's only available in the US and the UK right now, but I could be wrong. Yeah,
Chris Sharkey (50:14):
It's interesting though that they, they're so I guess market sensitive that they would release that before they can even give general access to what they already have, which is excellent.
Michael Sharkey (50:26):
Yeah, I think it's interesting. I still just don't really get these companies, and we saw this week as well, just in covering all the news that people want to know, open AI announced that they're going to have a developer conference on November 6th in San Francisco. They teased that there's going to be some great stuff coming for developers, but Sam Altman clarified that this isn't going to be an announcement for G P T 4.5 or five or anything of that nature. It's just things that will help developers. Now, this is off the back of last week where they announced chat G B T Enterprise or whatever it was called Enterprise, and everyone was saying, oh, all the wrapper apps are dead. They're just going to take over everything. And now they're having a developer conference and calling developers, and then Claudes also according developers, but then releasing a product. I'm just very confused by all of this.
Chris Sharkey (51:27):
They don't dunno who they are as companies and what they want to be. They just want to dominate the space.
Michael Sharkey (51:34):
To me, the advantage for philanthropic is basically that big context window. So I would be quoting developers based on that context window and looking for use cases and users for that context window. And I would be ignoring the chatbot stuff. I just don't really see why you would even bother doing it unless, yeah,
Chris Sharkey (51:52):
Focus on what makes you unique. And I think for philanthropic, it's that they pride themselves on their safety. And I think that, I agree with you that context window is their competitive advantage right now. No one else can do that. People
Michael Sharkey (52:03):
Love relating this stuff to Google search, and if you think about when Google came out, they were just the best at search. They didn't go, oh, we have an a P I for developers who want to build their own search engines and we're also going to build a search engine. It is just bonkers. It doesn't make any sense strategically, and I think as you said, people, they have no idea.
Chris Sharkey (52:24):
Yeah, yeah, exactly. And they're just trying to stay up by cloning what their competitor does. It's interesting to see how it will go.
Michael Sharkey (52:33):
So the real question is, are we going to go on November 6th or will they let us in? Yeah,
Chris Sharkey (52:37):
Yeah. I mean, I'd be keen to be part of it. It'd be great to talk to like-minded people about what they're building and just be amongst it.
Michael Sharkey (52:45):
Yeah, I'd be interested if anyone that's more technical or interested in this stuff from our audience is planning on going or would be interested in going November 6th in San Francisco, so we will await the news there. The only other tidbit that was reported, and it's not really news, we've talked about it before based on rumours, is according to the Verge, apple is now reportedly spending millions of dollars a day training ai. So potentially next year maybe you'll finally get a better Siri that you've always dreamed about. But yeah,
Chris Sharkey (53:21):
It might make me finally switch back to Apple if it's any good. Yeah, I
Michael Sharkey (53:25):
Mean, if they could give you open AI chat G P T in your pocket, that is Siri, and it's just super great integrated with a bunch of the different apps you use on your phone. To me that could be the killer AI agent. They actually might wipe the whole market clean with that concept.
Chris Sharkey (53:43):
If it has access to your email, your calendar, your contacts, voicemails, all that stuff, and it's able to cohesively use it in a way that you can instruct it and interact with it in real time when you chat to it. I think that that would be absolutely epic.
Michael Sharkey (53:58):
The only thing I just find funny is Apple and notoriously bad at doing anything. So I do think it's going to be something in their ecosystem integrated into their phones. I can't imagine some web-based like Siri, G P T anytime soon. No,
Chris Sharkey (54:13):
No, of course not. I doubt it's anything to do with that. It'll just be trying to continue to hold their grip on the phone market for sure.
Michael Sharkey (54:21):
Yeah. And then I don't even know if I want to cover this, but time release Time Magazine, remember that? Remember magazines? Yeah.
Chris Sharkey (54:33):
I guess it's all online stuff now.
Michael Sharkey (54:35):
Yeah, I don't know. I think they just flick around the cover and then anyone that's in it buys the magazine, which is why I think they call it,
Chris Sharkey (54:43):
They're selling it. They're just writing articles about people just they buy it. Well, they're
Michael Sharkey (54:47):
Like the hundred most influential people in artificial intelligence that guarantees them. That's least hundred sales, but maybe two or three copies. You want a few copies? That's the least 300 magazine sales,
Chris Sharkey (55:00):
The 1 million under 31 million of the most influential people in ai. I
Michael Sharkey (55:07):
Can't help but
Chris Sharkey (55:08):
Laugh. Not a bad idea. It's a seller magazine. Just name everyone.
Michael Sharkey (55:13):
Yeah, just name literally everyone and put a photo of them in there. I really find it funny that the Anthropic CEO os sort of just under Sam Altman in the photo, and I dunno if he's probably just got a lazy guy like I do, but it looks like he's giving a bit of a wink. And I can't help but think every time of the AI sex cult and how it's a, I think a husband and wife that are the ceo, E o and president of that company.
Chris Sharkey (55:38):
Yeah, it definitely sounds a bit like that crypto company that went under recently, what was that called with the Alameda Research and
Michael Sharkey (55:44):
The Oh yeah. And how they had that fancy relationship as well. Well,
Chris Sharkey (55:47):
They had the same life philosophy. It's exactly the same thing. That's why I made that joke in the first place because it's the same thing that something altruism. Well,
Michael Sharkey (55:55):
He was on the cover of time, I'm pretty sure as well. And he ended up in jail. So I wonder which of these people also end up jailed. Yeah, that's right. The scammer probably Jeffrey
Chris Sharkey (56:05):
Was on that too. I
Michael Sharkey (56:05):
Think yeah. Alright, on the Hitler note, we'll end the show there. I did want to call out. You guys have been leaving some phenomenal reviews on Apple in various countries and regions around the world. We do read them all and I just wanted to say a big thank you from the of our hearts. They make us feel great and I appreciate you taking the time to do them. Thanks for all the comments and the support. We don't take it for granted at all. And thank you for continuing to listen to our silly, ridiculous show on all things ai. We'll see you next week.