Subscribe
Share
Share
Embed
This Day in AI Podcast is a podcast all about AI. It's an hour-long conversation on the influence and rise of AI in technology and society. Hosted by Michael and Chris Sharkey.
Mum (00:00:00):
Sorry, who's calling?
Michael Sharkey (00:00:02):
Good day Mum, it's Mike.
Mum (00:00:04):
Hi Mike. What do you need,
Michael Sharkey (00:00:06):
Mom? I need to talk to you about something important.
Mum (00:00:09):
What is it, son? Anything?
Michael Sharkey (00:00:11):
It's about a financial matter.
Mum (00:00:15):
What can I help you with? What do you need help with? It's Chris. The biggest announcement this week came out of Meta yet again, the Zuck is on fire and they have introduced Code Lama, a state of the art large language model exclusively for writing code. What do you make of Code Lama?
Chris Sharkey (00:00:36):
Yeah, it looks really, really impressive. It's got a hundred K context window, which is extremely important when it comes to coding because you want it to take into account as much of your code base as it can when it functions for you or writes code for you, because if it's just doing it off a bunch of Stack overflow examples or something, it might not take into account the context of where you're at in the code and having that ability is really important.
Mum (00:01:02):
What's interesting too is one of the mainstay use cases today of chat G B T for many people I know is helping write boilerplate code or finding bugs in code. It's certainly something I've used it for quite a lot. And meta here again, is sort of attacking one of the main use cases with an open source model. In fact, three different models to help people be able to write great code.
Chris Sharkey (00:01:29):
And I think that because they're releasing the models themselves, the weights, so people can run it themselves, it'll mean we'll see it integrated directly into products and I do the same as you. I'm often using chat G P T or G P T four to answer questions about, well, what's wrong with this code? What's wrong with this output? That kind of thing. And then at the same time, using the code GitHub copilot in VS code to get code examples while you're working. They call it infilling where it fills in the gaps in the code you're writing, but having an actual language model like this built into the software will be massive.
Mum (00:02:04):
I also think what's interesting is some of the examples they have given is where you are commenting through the code and for people not familiar, one of the ways you can plan when you write code is to write a series of comments with almost a plan for how you're going to complete something. So I do this commonly in any projects I work on. I write a series of comments and then I go in and build each section of the code, like a series of functions or something like that. But in theory, and we're seeing this through some AI first ides, which are basically how people develop, write their code, the applications they use to write their code that it's able to just complete. So you write a bunch of comments, you sort of plan out how you want to do it, and then it can sort of fill in the blanks, which is quite amazing.
Chris Sharkey (00:02:49):
Yeah, I actually use that technique now with GitHub copilot where I'll write a comment and hope that it fills in the code, but it's actually quite hit and miss half the time. It just continues writing comments or it'll write the code but have it commented out and those kinds of things. So it's still in that very primitive phase where it doesn't quite get what you're alluding to. It's more like, well, I'm instructing you to write this code. This isn't just if you feel like it kind of thing. Do you think
Mum (00:03:16):
We'll see from this open source plugins for VS code, which is a very popular code editor
Chris Sharkey (00:03:24):
People use? Oh yeah. I think that'll be the very first thing we see. And then I think from there you'll see more like you described earlier, dedicated products where it's actually a different coding paradigm where maybe it's AI first and you are just making edits to it or you're hinting at it or you're using language to describe modules and iterating on that process rather than it infilling existing code. I think there'll be more cases where it's generating entire modules to do something rather than individual lines of code or helping you with a particular function.
Mum (00:03:55):
Do you know who's really going to love this is our meta G P T team. If people recall from a couple of episodes ago, we covered meta G B T, which essentially creates a series of roles for a development team is A C T O, the chief technology officer and various other roles in the team, surely that C T O and Meta GBT is going to love this release. I
Chris Sharkey (00:04:15):
Will absolutely enhance it and I think the ability as well to run it locally, run it part of an existing software stack and not constantly be calling off to expensive APIs is a very big use case. You could run it so much more millions of iterations and things like that. And so yeah, the implications of it are huge. And yet again, we see meta coming out with the goods for everybody and I still thought about it prior to the podcast thinking, well what is my thoughts? Why do I think they're doing this? And I can't answer that question.
Mum (00:04:46):
There's an argument to be made that they're just trying to disrupt OpenAI so that they don't control the mindshare around this. Especially with developers, which obviously as companies like Apple know, having mindshare of developers is critically important if you want your platform to succeed. So potentially if their plans long-term are building out agents and AI applications on their own platforms, maybe this is just simply a case of trying to win the hearts and minds instead of letting another company take ownership of the future, which it seems like OpenAI currently owns the future, I would say right now.
Chris Sharkey (00:05:23):
And I guess as well, it's staying in the minds of investors, in the minds of people saying, well meta, it's not like Facebook is some dying thing for plus forties like I am. It's actually something that's still relevant, will still be part of everybody's lives going forward. And it's just that mindshare thing. I think you're right, it must be along those lines as well as actually employing the technology internally as well. I
Mum (00:05:47):
Know it's related, but it is true. Facebook in my head I see as a really advanced classified ad doing how you used to get the papers and you read the classified ads.
Chris Sharkey (00:05:59):
For me it's like people in the neighbourhood saying someone left a wallet and keys on the pool fence or something like that,
Mum (00:06:06):
Or everyone has a ranting auntie,
Chris Sharkey (00:06:09):
Why is there a helicopter flying overhead? I need to know immediately. That's Facebook.
Mum (00:06:15):
Some of the other interesting takeaways, and there's one big, big, big takeaway, which I'll get to in a minute, but one of the big takeaways I thought was just this idea that as we've been saying for some time, the fine tune smaller model. So they made a dedicated model on a programming language called Python, which is very popular in the AI community and the 7 billion parameter performs according to their paper just as well as the more generalised model. And to give
Chris Sharkey (00:06:45):
Everyone an idea, 7 billion parameters is small enough. You could theoretically run it on a phone. The really, really small ones, anyone can run it on their own machine. And so it's actually very, very accessible. I mean you could bundle it as part of a downloadable piece of software, for example, and run it on commodity hardware. So I mean high-end commodity hardware, but still. So it's actually pretty significant to see those smaller models, like you say, something that we've theorised is actually probably the future of a lot of applications for ai.
Mum (00:07:18):
Just getting back to the point around, you said earlier around this idea of what's meta's play, there was an article in the information which specifically said META'S next AI attack on OpenAI free code generating software. So they're framing it as a direct attack on OpenAI as well. But yeah, it's unclear. It does seem like it's just also trying to attract the best talent for what is the next internet.
Chris Sharkey (00:07:46):
And I'm not complaining, let 'em keep releasing stuff. It's great. I actually applied for access this morning for the weights. You've got to agree to terms and conditions. I guess this is a bit where it's not truly open source. They do have some conditions, but I agree to whatever, I don't care. I just clicked and I already got access immediately. So there's no real delay in getting the weights. You can do it immediately. I thought
Mum (00:08:06):
We'd be on a blacklist after last week talking about how censored their model was. But anyway, here we are.
Chris Sharkey (00:08:14):
Clearly we're not relevant at
Mum (00:08:15):
All. So next week it'd be interesting to hear how using it goes. So maybe we can try it out this week. But another really interesting tidbit in the paper around code Lama was this, Ethan Molik called it out over on X pretty good, I'm calling it X now. You got it right the
Chris Sharkey (00:08:36):
First time this time.
Mum (00:08:38):
So in the comparisons where they compare it to how it performs against other models, they had this gall out for unnatural code llama and it is a 34 billion parameters and it beat all the other models here apart from G P T four coding. But G PT four is not ranked on a number of other benchmarks, so it seems to perform really
Chris Sharkey (00:09:04):
Well. It's a training dataset, right? It's not a model itself, the unstructured thing.
Mum (00:09:09):
So yeah, yeah, sorry to clarify. Yeah,
Chris Sharkey (00:09:12):
Yeah. And so what they've done, normally we use this, what do they call it? R H L F, which I always think stands for or something like that, but it actually, which doesn't even add up, but what it stands for is I forget, human alignment basically reinforce human alignment. And so the idea is you come up with questions. When you want to train an instruct model for example, you give an instruction, an input and an output. So the instruction might be summarise this text, the input might be a story, and then the output might be a short summary of the story and you give it thousands and thousands of human created examples to align it with human preferences. So for example, the summary needs to be short, the summary needs to be accurate, that kind of thing. And so that's how a lot of the models are aligned.
(00:09:57):
And then we got the second generation of that, which is where people would go to an existing model like G P T three for example, when that first came out. And then they would use GT three to create these examples. So that's where we got the original alpaca model from where they actually got all the examples from GPT three, then they ran them on the open source lama, the original LAMA model, and they got that output. Now, what's been discovered in this paper is that what they can do is they can give three human alignment examples and then make the AI give a fourth alignment example, and then they just keep running that process over and over and over again to get more and more alignment examples. And in this paper, they actually made 64,000 triplets of instruction input and output all generated by AI just by using that technique of three generations in the fourth. And they only used 15 examples in total, 15 human generated examples in total to generate those 64,000 triplets, which has led to better human rated results from that dataset than they've gotten from the human aligned ones.
Mum (00:11:10):
So I think it's important to back up a little bit for our more non-technical audience of why this actually matters. So we have speculated in the past before on this idea that if the AI starts running out of training data and trains itself on its own garble that it's spitting out, well then these models could go nuts and be bad in the future and eventually they would fall over because they'd be training on absolute garbage.
Chris Sharkey (00:11:35):
I think everybody speculated in the early days that that's what would happen, is that okay, it hallucinates a lot, it makes a bit of nonsense. Therefore, if it keeps making nonsense and it starts training itself on the nonsense, then like you say, it'll totally degenerate.
Mum (00:11:50):
But it turns out from this additional paper that meta released on unnatural instructions, which is worth reading if you're interested, it's super, super interesting and gives I think insight into the future of where things are going to go. As you said, they were able to collect all of these training examples. I think I read in there, and I'll try and find it. So it's 50%, although the dataset contains noise, so they acknowledge it does contain noise, our analysis reveals that more than 50% of generated examples are indeed correct, and that even incorrect examples typically contain valuable information from instruction tuning. The other thing I thought was interesting is they were able to get really novel training data versus humans. It was actually creating better examples or better training data than they were soliciting from humans. And can you explain how they were able to do that using the existing large language models?
Chris Sharkey (00:12:49):
Well, it's exactly what I said before where they would give it three examples of a human one and then use, its I guess the temperature setting, which is the level of randomness of how it generates the token. So when we talk about temperature with the models, basically a temperature of one means it'll give you the highest predicted next token. So when it's generating new tokens, it'll go for the one that rates the highest. As you lower the temperature, it'll randomly select lower outputs for each token, not necessarily lower, but it'll mix up which token it picks, so it won't always pick the biggest one. And so using that, you get a sort of level of creativity and novelty from it. So by altering those settings, they're able to actually create these novel examples that have never existed before and use those as part of the training data. And obviously that diversity of training data then is having these positive effects on the result aligned model, but
Mum (00:13:43):
The ability to generate that training data is really, there is still a limitation factor like the original model that they used to generate the
Chris Sharkey (00:13:54):
I guess so, but the results are proving otherwise. I think that's so interesting about this because if you just asked me raw and prior to reading this, I would've said, well, there will be limitations because it's sort of auto regressive. It's like it can only know what it knows, but evidently the AI does have the ability to create new knowledge or at least seemingly new knowledge and certainly new knowledge good enough to make a better model. Unbelievably fascinating. And as we saw the other week, I think you pointed out that they noticed with stable diffusion when they were training it that actually giving it bad examples makes it better. You can actually train it on worse examples than it's capable of producing, and then ultimately that will lead to it producing better output. So clearly there's something here that hasn't been recognised, and it very much relates back to what we've talked about where I don't think we've seen the full power of what these large language models represent. This is a layer on top of it. This is an additional phase of training on the existing weights of the models that are making it significantly better and just these new techniques are able to unlock far more power in the existing models.
Mum (00:15:06):
Why do you think meta didn't release this unnatural code Lama version? I mean,
Chris Sharkey (00:15:12):
I saw the same tweet you did, so that thought sort of sticks in my head that they don't want you to know that you can train it on itself and it'll get better. But I think that even for me who's very cynical about such things, I genuinely don't dunno the answer. I don't think that's the reason, but I don't know,
Mum (00:15:29):
Is this the sparks of a G I? Is this a very early realisation that, and I'm not suggesting we're anywhere near that, but is this the next evolution where these models are training themselves or they're already training themselves,
Chris Sharkey (00:15:45):
Whether it is or it isn't? I think what it really, really reinforces is something you and I have speculated on many times that at some point these AI algorithms will be able to train themselves better than we can and in a way that isn't necessarily aligned with human values because right now they're measuring it based on what humans think of its output. But what if you take that part away as well? We had the humans generating the examples before now it can do it. What if its ability to assess itself comes from within as well based on its own goals? I think that's where we start to see a g I come out, it has its own goals. It's trying to align itself to, and it's producing its own training data to train its own models which it can then employ for different tasks. And I further think the idea of the smaller dedicated models for specific skills, if we give the AI the idea and the ability to do that with its own training data that it produces, it's going to be able to develop a wide variety of highly specialised skills that can then orchestrate to become a super intelligence.
Mum (00:16:51):
And so I think that's a good segue into this article from ours, Technica, during the week how chat G B T turn generative AI into an anything tool. And what is interesting about it is it talks about in machine learning and AI specifically how there was initially all these very specialised tools that people would use that were for protein folding. The example they give is Google's alpha fold and various other very targeted models. And chat G B T really brought it into the mainstream of having this generalist model. And it turns out that this generalist model is great for doing a whole bunch of different things, almost like a processor in a computer. It's just a general purpose capability. And it also talks about fine tuning will get to the announcement from OpenAI about 3.5 and four being available for fine tuning soon. But I think the question it arose is just talking through this evolution and we've both witnessed this evolution all the way from, I remember the first time you tried G P T two and it just spat out.
Chris Sharkey (00:17:57):
That's right through garbage. Yeah, I remember we were in San Francisco and we thought we could use it for our own product to generate subject lines for emails or even content for emails, but I think we were a bit modest back then. We were just going for the subject lines and it was getting close, but it couldn't even produce coherent sentences even when given thousands and thousands of examples, it just simply couldn't do it. Look, admittedly I had no knowledge back then maybe I was using it wrong, but it definitely wasn't able to, even with sort of pre-made examples, produce coherent English that you could trust.
Mum (00:18:31):
And so this article talks about that evolution, this idea that just getting it to follow simple commands initially was pretty difficult and how it would not have context of the previous conversation or inputs at least, and how that's now evolved to it having that full history and now it can take feedback. And so I think the real question here is in terms of the evolution, do we just see one single L L m that's almost like a single thread C P U, and then eventually over time you get these smaller components of that threading that are more specialised models for different parts.
Chris Sharkey (00:19:17):
And I think you have the bigger model orchestrating the smaller ones. I mean the interesting thing in the code LAMA paper is they actually said they got better results starting from the LAMA to generalist large language model than they did making a code dedicated one from the get-go. So it actually works better having one of these generalist models as your foundational model and then training it, then it does producing a dedicated one from scratch. So that's interesting because it seems like then your general models are going to be the genesis of the specialised ones rather than just making specialised ones for that only purpose. Yeah,
Mum (00:19:57):
So I think what's also pretty interesting here is just this idea that you can have a series of fine tuned models with the foundational model. Foundational model is such a marketing word. I think
Chris Sharkey (00:20:13):
It does sound cool though.
Mum (00:20:14):
Yeah, it does. And so you've got the foundational model, you've got all of these specialist trained models, and then the next question might be, well cool, so you just chat to them. And that's sort of how people, I think about it today in their head, but you can imagine this starting to be used for processes like go and handle this text translation or go and
Chris Sharkey (00:20:38):
Yeah, I think it's a very limited way of thinking to only think of large language models in terms of chat. I definitely get better results when I'm working on specific problems working with them in regular completion mode. For example, working with LAMA two, working with it in regular uncensored completion mode works a lot better for certain problems than it does with the chatter line version for example. Even if you're not trying to do dodgy stuff, it's actually, it's better at it. So I think that yeah, thinking about them only aligned in terms of being good at chatting is limiting the capabilities that are available to you.
Mum (00:21:15):
And then really back to the C P U analogy, which we've used in the past and is used really well in this article as well, is this idea of the embeddings are kind of like the ram.
Chris Sharkey (00:21:28):
Yeah, yeah, that makes sense. A sort of working context memory that puts you in that mode.
Mum (00:21:34):
Yeah, it just seems like these, all of these elements are the foundational elements of building an actual robot that can think and do stuff and be really useful.
Chris Sharkey (00:21:45):
And then to extend that metaphor or analogy or whatever you call it, I would say a fine tuned model is embedding that context into the model itself. So you want it operating in a particular mode with particular knowledge, with particular output style and things that takes into account every time it runs. That's where you fine tune and then the fine tune model is pre-made to do that. And then the benefits of fine tuning are it's cheaper because you can use a smaller model. They've already shown that fine tuning G P T 3.5 can give G P T four results on specific fine tune problems. So it's significantly cheaper and it's faster. And we've got some phone stuff we played around with later to show where speed is paramount. There are certain applications of large language models in the future where speed is going to be critical. And my argument would be no matter how big the generalised models get, they're going to be running on hardware that can't quite do it fast enough for certain applications. So that's where the dedicated smaller, fine tuned models are going to come into play where you want them quick as they can be, but still accurate
Mum (00:22:52):
Rather than this slower general purpose model.
Chris Sharkey (00:22:55):
So you trade off accuracy for speed, but ideally with fine tuning you can still get the level of accuracy required for that problem.
Mum (00:23:04):
So the other interesting paper during the week came out of Google, and it's right on this trend of using the AI to train themselves and advance at a much more rapid rate. And there's a really great example here from Anton on Twitter. I'll obviously link to this as always in the show notes it says previous versions of AlphaGo initially trained on thousands of human, amateur and professional games to learn how to play go AlphaGo zeros skips this step and learns to play simply by playing games against itself, starting from completely random play. In doing so, it quickly surpassed human levels of play and defeated the previously published champion defeating version of Alva go by a hundred games to zero.
Chris Sharkey (00:23:48):
Amazing. And it's a similar technique I've seen where they actually do it on a game of Mario or something and they literally just optimise it to get the score up and tell it what controls it has up down left. And it can literally learn not only learn to play the game and to complete it, but additionally it finds all the glitches in the game where it can jump on a certain pixel and do a special jump. And the speed players who try and complete a game as quick as possible, they get their ideas of how to speed the game from the AI glitch version. So this sounds like just a more sophisticated version of that,
Mum (00:24:25):
And this is what I don't get and we've covered this last two weeks, is there is this theme, at least in articles that seem to be getting promoted at the moment, is like, have we reached the limitations of ai? Is AI large language models going to, are they a flop? And there was an article I was going to talk about this week, but I'm sick of even covering it. I just think it's nonsense. But to me this is a pretty big next breakthrough, this idea of reinforced self-training and potentially G B T 4.5 or five or whatever Google's actually trying to work on is
Chris Sharkey (00:25:03):
I doubt they're working on G B T five. That'd be a bit mean for ai. Like hey, that's our name. They already copyright it. You said they trademarked it.
Mum (00:25:12):
Yeah, true. But also I guess they helped them down G P T with their initial breakthroughs, so maybe they are kind of working on it. But I guess the one call out I'll make is, so DeepMind released this paper, which is Google for those that are unaware, reinforce self-training for language modelling. And we'll talk about that in a minute. But one thing we both commented on before the show reading these papers is I found reading the meta paper on roughly similar topic, incredibly practical, very clear, easy to understand and follow, and it was evident why they were doing what they were doing. Whereas in contrast to this deep mind paper, I find it very academic is what I would
Chris Sharkey (00:26:04):
Say. It's very abstract. They're giving formulas and things like that, which is fine. I mean that's what they're doing behind the scenes. But the meta paper gave examples and they gave diagrams explaining the process they've gone through explaining why it works and actually giving examples where it failed. Examples where it worked. They're clearly going through the process of doing it. And I'm not saying Google, but I guess maybe Meta's just trying to be more relatable to common folk like us, or perhaps they're just more in the weeds actually working with the tech for something real, I don't know. But it felt
Mum (00:26:40):
More to me, they actually trying to implement the technologies and ship products and actually thinking about how does this work in society. Whereas Google's seemed very meta, it seemed very philosophical and like, oh, this kind of is a theory about how something might work. And obviously this has led to breakthroughs in the past, so I'm not trashing it, but I'm just saying that in terms of if you're looking at the companies, metas are absolutely on fire and I think that's reflected in even how they're writing their papers and Google. It's like what are they doing?
Chris Sharkey (00:27:21):
Yeah, and I think Google's as well, even though they use the same sort of the computer generating the example, it was actually a lot more heavily reliant on humans. They basically use human alignment to make a scoring function. And what they were saying was they were just relying on running multiple increasing locus of examples through that human scoring function so they could use that function over and over again and they needed less human examples. So it's sort of talking about a minor efficiency in approach, whereas I feel like the LAMA thing's a true breakthrough because it sort of says, well, okay, maybe we don't really need human alignment at all. Maybe we don't need the human examples in order to create an intelligence. And I know Google's somewhat shown that with the go example, but that's a very domain specific problem, whereas LAMA is doing it in a much wider domain of a general model or at least the code generation model. So I just think it's far more profound in its findings and its practicality.
Mum (00:28:18):
Can you give a general example of how these things are trained? I know we've done this before, but I think it's just a good catch me up for people listening and going, what the hell are they talking about? So just if we talk about say G P T 3.5 or four, like the chat G P T modes, how were they trained? Can you give us a practical example?
Chris Sharkey (00:28:39):
Well, I mean you start with the convolutional neural net where that's basically where you have an input node, you have all the hidden nodes in between, which is sort of like a brain with all of the things that give feedback and then the output. And you essentially start giving it examples where you have a desired output in mind. You run it through based on the score, you give it on its output, it then goes back and adjusts the weights some randomly, randomly basically. And it keeps going until it gets the weights, which is what we call the weights when they release them of all of those different nodes. So eventually you end up with a neural net full of all these numbers and how much emphasis is given to each node when things go through to get the kind of outputs you want. And then they train that, as we say, on billions and billions of parameters.
(00:29:31):
And so part of that is the unsupervised trading where it's essentially scoring it itself. And then once it gets to the point where they have those weights, then we get to the alignment phase and that's where you start to say, okay, it now is a token completer because that's what it's trained to do. This input completes to this output, for example. Once it gets there, then we start to align it by saying, okay, this is an instruction, this is an input and this is an output. And you give it examples of those specific things and it starts to learn to behave in that manner going forward. And that's where you get your chat GPTs of the world,
Mum (00:30:07):
Whereas, and so that alignment leads to, and that piece of the alignment to be clear is what meta is doing artificially now.
Chris Sharkey (00:30:18):
That's right. And so the idea that you needed to use someone else's model to get the sample inputs, which is what happened with alpaca, that's out the window now, you don't need to do that essentially using this technique and they're showing that it's actually getting better results. And the great thing about that for the open source community is it means you don't need to any longer rely on proprietary data sets which would automatically make your model something that can't be completely free. Yeah,
Mum (00:30:43):
This is something that actually called out in the paper I highlighted in one of them, which is this idea that it does remove that burden or potential burden of copyright data because it can just make it up. And
Chris Sharkey (00:30:57):
Also what's kind of exciting about it as well is think about aligning it for other problems. We think about things we want to do with the models, for example, phone conversations or the horse racing thing or whatever it is. And in order to make it better, you really need thousands or hundreds of thousands of examples to align a model to get a specialised model that's going to beat a general model. And not everyone has access to all of the data. It's quite difficult to get. So for example, in our business, we made a tool that would actually predict the open rate of an email based on the subject, but that's because we had millions and millions of examples in our database that we could fine tune G p T three on and it gives extremely accurate results because we had the data, but a lot of people don't have the data, which means that they can't specialise the models. What this finding essentially shows is you could have the AI come up with its own examples that make it better at a domain specific problem than the general model, even without having access to the raw data, which is just fascinating.
Mum (00:32:03):
It's hard to wrap your head around because you're like, well, if it's just making stuff up, how is that even helping improve? But it's sort of almost the organisation of relevant data that's more important.
Chris Sharkey (00:32:16):
And I'm in two minds about it because as you say that you're right, if you're doing something that has a sort of measurable output, how could it just make up better results? But I mean they're sort of showing in this case that it does. I mean they're doing it with real life code and the way they did it was they got it to generate unit tests for code. And a unit test for anyone who doesn't know is code that test code essentially. So for the following 10 given inputs, these should be the outputs of the function similar to what I just described with the neural net, but for code. And the difference with code is it's either yes or no. There is no ambiguity there. It either works or it doesn't. And so it would then align by, it would generate 10 code examples and it would loop through them until it found the one that met the code and it would say that gets the highest score that completed the problem, and that's how they made the alignment. So there may be similar techniques you could use to generate accurate training data. So rather than having to pay for proprietary data or obtain proprietary data over many years, you could maybe generate it based on an algorithm or a rule or observation or whatever it is.
Mum (00:33:23):
I mean, perhaps it just works so well for coding and this is some anomaly because it's easy to test. It's like maths. That's true. But I guess you could reason also that science in theory could explain anything, and therefore if it can do science, then eventually it could test its own assumptions and get
Chris Sharkey (00:33:43):
Smarter. And I think we've spoken about this before with regards to a G, I think it gets really interesting where the AI is able to start doing its own real world experiments and actually getting visuals and audio and the multimodal elements to things where it can actually start to make its own assessments of things and get that additional input to then generate more training data. And I think that's really interesting too. When we think about this unsupervised data generation technique when it comes to say images and voice and video and that kind of thing,
Mum (00:34:17):
How far does it go back if I want to then retrain my neural net and I'm an ai, I mean given it's so focused on the fine tuning element, can you then take what you've learned and potentially retrain?
Chris Sharkey (00:34:30):
Yeah, of course. Yeah. You could then use that as an additional data set for your original base model for sure.
Mum (00:34:37):
Yeah, and then further expand that. So this could lead to in theory exponential or some exponential ness in terms of Well,
Chris Sharkey (00:34:45):
Yeah, and again, it definitely comes down to hardware resources, but you've got to think about if the AI has enough hardware resources, it can do hundreds of thousands or millions of experiments with altering the base model or just fine tuning smaller models or any combination of those things to optimise for the result that it's getting towards. And this is what we need to start talking about. It's not just human aligned anymore, it's AI aligned. Once it has its own goals, which I believe it will, it's going to start to align towards those and then it can keep experimenting to see what works to get towards those goals, whether they're altruistic or bad or whatever. They happen to be
Mum (00:35:21):
No need for disillusionment. People we're moving, we're getting to doomsday faster than you think. So just going back to this Google paper, I trashed it a little bit, but the idea here is more aligning it to that particular human's expectations. Is that why they're doing this?
Chris Sharkey (00:35:40):
Yeah, yeah, exactly. What they're trying to do is find cheaper and more efficient ways of training models. So by having more emphasis on a scoring function rather than human scoring individual inputs and outputs, they're able to train a model much more fast and cheap than they could have before and get similar results. So they're really looking at the efficiency of training and the ability to train future models without continuously having to resort back to human alignment, but still obtaining something that's aligned to human preferences.
Mum (00:36:13):
So speaking of training, OpenAI finally announced this week, G P T 3.5 turbo fine tuning and a P I updates. So fine tuning for G B T 3.5 is now available. Now, correct me if I'm wrong, but we were using fine tuning on G B T three, then they took it away and now it's back for 3.5. Is that
Chris Sharkey (00:36:36):
Right? That's my approximate understanding. I don't know. I mean I've fine tuned several models on GPT three after that. The core models in my opinion got good enough that with multi-shot examples where you give it two or three examples, the results were good enough that I personally didn't run into any problems where I felt like a fine tune model would help enough. But there's certainly people who do have those things. And as I mentioned earlier, the primary benefits for fine tuning are accuracy obviously if they're trying to solve a particular problem output format. So if you're trying to get something in a particular format, we've talked about guidance before and function calls and things like that, but there's nothing better than fine tuning for a specific problem because it's going to give accurate output every time. And then speed and cost. Because if you can train a smaller model on a specific problem, it's going to cost less and it's going to be fast. And I think in the case of G P T 3.5, turbo 16 K, it's pretty amazing because you've got that 16 K of context, which is a lot. You've got the speed, it's very fast and it's cheap. And so I think that the fine tuning being available there is going to lead to a lot more commercial applications as in product embeddings for the large language models than we even see now. I
Mum (00:37:55):
Also think the cool thing about it's being able to shorten your prompts. So because you fine tuned it on your own data or your own use case that you don't need to just give it as much context in each prompt or ways of responding or outputting so that you can actually take advantage of the full prompt size. And of course they said that support for fine tuning with function calling and G P T 3.5 turbo 16 K will be available later this year. So that's pretty cool.
Chris Sharkey (00:38:26):
You make a really, really solid point there because one thing I've noticed with prompt design is you really do need to include a lot of caveats. If this happens, do this. If this happens, do this, never do this, please. And all of these things that the general model needs to be reminded of. And something I've noticed with some of the experimentation I've been doing lately is as soon as you go down the model ranks, so let's say you're using G P T four, it can follow all the instructions just fine. Claude can do the same thing the second you drop down to G P T 3.5, sometimes it follows all your instructions, sometimes it ignores them. And so you reach this problem where you're like, I need the speed, but I'm not getting the results from it. And so I believe that the fine tuning is where that you reach that sweet spot where you get both the accuracy, the speed, and the lower cost, and as you say, you get more of the prompt available as well to you because it just knows what you're talking about.
Mum (00:39:20):
Can you imagine here, businesses fine tuning this specific task. So as an example, if I'm a law firm and there's like four different specialties I have, could you imagine fine tuning four separate models for those four very diverse tasks in order to have a better capability if we specialise in contract law, do I go and train 3.5 fine tuned on all my contract law?
Chris Sharkey (00:39:46):
Yes. Yes, I think you would because you can tell it the things that it needs to identify, say in analysing a contract or things that must include or provide a specific output format for that kind of thing, teach it things that should be emphasised over others. And yeah, I definitely think you would try to specialise there.
Mum (00:40:06):
Lawyers, you are doomed. I
Chris Sharkey (00:40:08):
Mean the difference with law in particular is because it is a lot of language understanding and generation, I think it is a case where the base models are actually pretty good already. It's probably a case where I wouldn't be so APTT to fine tune on the things, but there'd be certain other industries and problems where I think it would be much better to do where it's not primarily about, it might be more about classification or highlighting particular points or things that lots of examples would improve on.
Mum (00:40:39):
Coming back to our last show where we talked about just the accessibility of ai, do you think that's still a problem here because I've still got to get the data, organise it in such a way that I can fine tune and then go and have the expertise to go and fine tune and then implement that into my business. It seems very much a relegated to the enterprise and people who have dedicated teams to do this right now.
Chris Sharkey (00:41:02):
And I think we'll see an industry pop up where it is, I'm a model trainer for business, come to me and you consult with me and I'll help you build a training dataset for you to do it. Similar to really what data scientists were in companies in the early days was basically just data cleaners. They would come into a company, query all the databases, put together a dataset, train that on a general model, and then they'd have some sort of machine learning thing internally. And I think it's just an extension of that. It's really having nice data is a thing. However, looking back at the, what did we call it? Unnatural instruction dataset. There's also this idea that maybe there will be products pop up or techniques pop up where you can go, okay, I'm in the law industry, I'm in the whatever industry, I want to generate a data set that's going to lead to a specialised model to solve this problem. And you sort of work with the AI to make a dataset, so that's a possibility too. But yeah, I agree. I don't think it's very accessible just yet without people who know what they're doing to operate the tools for people,
Mum (00:42:04):
But still a pretty cool thing to have available. I guess though, what in your opinion right now though, what's stopping you just going in a hugging face and training llama on your own fine tuning llama instead where you don't have to pay per call? Is there an advantage to this at the moment or, I
Chris Sharkey (00:42:27):
Think so. Yeah, absolutely. I think fine tune LAMA two is going to outperform G P T four on the specific problems you train it for. So absolutely there's an incentive to do it. I just think people generally speaking in normal business would lack the expertise or the inclination or just the knowledge that they can go ahead and do that. I think it's really an awareness, partly an awareness thing, partly not quite understanding the benefits yet and seeing it being done by other people, but I think ultimately that's going to be a pretty common thing and certainly something where metas come out and derail GPTs corporate ambitions when people are going to be able to run the models themselves in a safe way for specialised tasks. So yeah, it's definitely made things more dynamic.
Mum (00:43:12):
Yeah, I think someone left a really good comment during the week on I think one of the YouTube videos saying that obviously open AI's goal is just to embed this thing in every enterprise on the planet and every application. And that's why they want to have all censorship in place perfectly. So it doesn't say anything offensive or upsetting that might get one of these organisations into trouble. So it might just be a case of the commercial brand almost like an I B M, you go to open AI for your models because it's just this blanket of safety and security in the enterprise. And then it's more the pirates of the world like us that are over on hugging face trying to fine tune crazy open source models.
Chris Sharkey (00:43:57):
Yeah, hard to say.
Mum (00:43:59):
So on that note, there was another update about 11 labs this week. 11 labs of course is the technology that allows you to clone your own voice or use some very realistic human sounding voices. It came out of beta and they also released multilingual V two, which allows it to take your voice and speak in 30 other languages. And Chris, you put a pretty cool demo of this together. You've
Chris Sharkey (00:44:31):
Kind of ruined the demo. We were going to pretend it was really us making the announcement, but
Mum (00:44:34):
Let's hear it. Damn it. Sorry, my lead in was terrible, but
Chris Sharkey (00:44:38):
That's all
Mum (00:44:38):
Right. Let's hear from AI us. It's
Chris Sharkey (00:44:40):
So
Mum (00:44:41):
Realistic. Okay, here we go.
Chris Sharkey (00:44:42):
Mike, did you see that 11 labs released their V two multilingual speech model from beta?
Mum (00:44:50):
Yeah, I saw that being able to clone anyone's voice and have it speak naturally in 30 different languages is nuts.
Chris Sharkey (00:44:58):
And the quality seems good too. I wonder if it's good enough to fool people.
Mum (00:45:04):
Yeah, I bet it is.
Chris Sharkey (00:45:06):
For now, I'm going to try cloning your voice and making it say ridiculous things in Japanese.
Mum (00:45:20):
Okay, so there you go. I'll tell you what, it really accentuates the Australian.
Chris Sharkey (00:45:26):
Yeah, exactly. And interestingly, so 11 labs sort of makes you tag what accent to use, and if you don't put a label as accent Australian when it has to fill in those gaps where it can't quite get the voice right, we sound very American. And so I think that the accent thing is important because it's obviously falling back on a certain base model to get those intonations correct. The other thing I found interesting is those, what you just heard was trained on only I think 45 seconds of you and 50 seconds of me or something like that. You can provide up to I think 10 minutes of audio, but it wants them in one minute clips. And what I've found interestingly is shorter clips actually work better the more you provide doesn't necessarily make it better. And then on top of that, there's all these tuning parameters like how stable it is, how true to your voice it is versus how accurate it is.
(00:46:19):
And then if you run it in a streaming mode, how latency sensitive it is. So there's a lot of tweaking there where I think if you got the settings right per voice, you could actually yield much better results. This is just me using the most basic raw stuff, but compared to V one, the output is pretty significant. And when you make it do long passages of us speaking, it really is convincing. I was playing your voice when I was testing stuff and my boys were like, oh, is Uncle Mike around? And I'm like, no, no, no, that's ai man. Yeah, I
Mum (00:46:53):
Think honestly, if you just follow, and we probably should do this for an upcoming episode, but just train it, give it the full 10 minutes and try and tune it as best we can into our voices to see if we can actually completely fool the audience if it's us versus the ai, just to watch how this technology gets better. But during the week, you also came up with a few interesting ideas with this technology. One of them was trying to call me and convince me to buy an electric pen as myself from the future.
Chris Sharkey (00:47:29):
And then my mind turned to, oh, I know I'll do a phishing attack on our mom
Mum (00:47:33):
Just to back up. So I want to give the audience context for what happened. I'm literally packing the dishwasher with all the chaos that comes with trying to get kids to bed and clean up and stuff, and the evening and my phone rings and it's a US number, and anytime someone rings me from the US right now, it's generally important. So I of course pick up straight away and then it was myself saying, hi Mike, it's you, but from the future. And I'm like, for a moment I was like, hang on, what?
Chris Sharkey (00:48:04):
Not the most believable example I chose.
Mum (00:48:07):
Yeah, but interesting nonetheless. So back to the phishing example here.
Chris Sharkey (00:48:13):
Yeah, so what I came up with, I've thought about this for a while because I think I've said on previous episodes I'm really interested in the use case of sort of talking directly to the AI and the interaction. And I thought, well, if I want to do phone calls to do things like sit on hold for me and answer questions or call up my kids' school if they're sick and get that job done for me and just chores on the phone that you want the AI to do, I thought about how you might do that. Now, when I've tried this, the problem is there's two or three things that really slow it down. One is the voice recognition, which is still too slow. There are better techniques out there, like the whisper smaller models from OpenAI and a few others you can try that are faster, but the speech recognition is slow.
(00:48:53):
The second one is the audio generation, which is what we're dealing with here with 11 labs where you can now stream it for example, which is why I got interested in this again. And then the other one is the inference. So once you do get the speech recognition, you've got to run it through a large language model to make it run. So I thought about ways I could skip some of the steps to make it faster. And one of the ideas I had is if you give the AI a goal for the call, say, look, I'm going to call up my mom. I want to do a phishing attack on her and try and get her bank account details. I want you to go through every possible scenario you can think of that might happen on this call and give it some context. Like this is your name, this is where you live, this is what the money's for, this is why reasons to tell mom, it's not a security risk to give out the details, that kind of thing.
(00:49:38):
Then have it pre generate as many sentences as it can think of that may come up on the call and generate audio for those in advance. Then when your speech recognition happens on the phone stream, that speech recognition to a large language model, which will then know all of the phrases that has available. And as soon as it got one, it spits out that phrase and plays it. So you actually skip part of the speech recognition and you skip part of the, well all of the audio generation. Then if it encounters something that's truly novel, it can then stream the speech recognition from 11 labs to answer the question and there'll be a little delay. And what I've done with that on my tech demo version is actually generated a whole bunch of what I call stalling phrases like, ah, or hang on a sec. Oh sorry, just one moment. And it says that while it goes off and generates the audio. And so that actually gives it time to generate it, but keeps the call fairly realistic. Now there's several problems with it and it's still definitely way too slow, but I do have the sort of basic version working.
Mum (00:50:51):
So this is something everyone casually does it on Wednesday night, I'm going to figure out how to do phishing calls. But what I think is interesting about it is figuring out all of those things that makes a call, human background noise, the automatic stalling, at least knowing some factual details about them. So when they inevitably be like, Hey, is this a computer? Is this ai? It can take that down really fast. And as you said, it's not really quite there yet, but it's definitely scary when we can put a quick demo together like we're about to do for you right now, and you'll hear it and you can imagine in six months, 12 months, how good this gets. Yeah,
Chris Sharkey (00:51:34):
And I think the other thing is all the models you can run locally would help if you could run all of these models on your own hardware that was like a H 100 or something beefy or a couple of them and run all the models locally and not have to do, see, for us, I'm running through Twilio for the call, which takes time. The speech recognition's slow, there's latency, there's so many things slowing it down. But if you could run it all on one piece of hardware, running through a real phone, for example, real phone, running the inference locally, running the text to speech, speech to text all locally, you could definitely do these calls in real time using this exact technique. You wouldn't have to modify anything. Well, we
Mum (00:52:12):
Covered also the metamodel where you could just produce very realistic background noise. I think we showed sirens and nature noises. So you could also produce pretty realistic background noise as part of this call as well to give it a sense of urgency or really pull the user into thinking this is a real call.
Chris Sharkey (00:52:31):
So we've built it up quite a lot and this is going to be pretty disappointing in comparison to that. But I guess the point is if someone was dedicated to this task, that's what I'm saying, this is like a couple of hours work you're going to see here, but if someone was dedicated to this task, you could make something really, really believable.
Mum (00:52:48):
Alright, so I'm going to play the role of our mom on this fictitious call, which will be interesting and probably very offensive to our mother.
Chris Sharkey (00:52:56):
Okay. Should I call you?
Mum (00:52:56):
Yeah, we're recording this actually live, by the way. There's no editing too lazy. We like we
Chris Sharkey (00:53:03):
High risk demos.
Mum (00:53:05):
Okay, so hello. Hello.
Michael Sharkey (00:53:08):
So can you give me your net bank login?
Mum (00:53:10):
Sorry.
Michael Sharkey (00:53:12):
So can you give me your net bank login?
Mum (00:53:14):
Who is this?
Michael Sharkey (00:53:16):
Yes, it's really me, mom.
Mum (00:53:18):
Sorry, who's calling?
Michael Sharkey (00:53:20):
Good day mom, it's Mike.
Mum (00:53:22):
Hi Mike. What do you need,
Michael Sharkey (00:53:25):
Mom? I need to talk to you about something important.
Mum (00:53:28):
What is it, son? Anything?
Michael Sharkey (00:53:30):
It's about a financial matter.
Mum (00:53:33):
What can I help you with? What do you need help with? It's crazy.
Michael Sharkey (00:53:39):
I need to buy a H 100 G P U Mom.
Mum (00:53:42):
Oh, what do you need? Something like that for? Is that like a bike?
Michael Sharkey (00:53:46):
It's for my work mom.
Mum (00:53:49):
And how much do you need, hun?
Michael Sharkey (00:53:51):
It's quite expensive. Around $30,000.
Mum (00:53:54):
$30,000. Okay. Well if you insist,
Michael Sharkey (00:53:59):
I was wondering if you could help me out.
Mum (00:54:03):
Okay.
Chris Sharkey (00:54:03):
I think it might've hung up. It hung
Mum (00:54:05):
Up on me. Let's
Chris Sharkey (00:54:05):
Give it up. It's interesting. I've been rattled
Mum (00:54:08):
When we tried that earlier, it was actually a lot more fluid and it started a lot more realistically, but that time, I mean it was caught out immediately. But what's really interesting, and some of the things I didn't get to is it's pretty clever in that when we didn't hear an example on that of when you ask, it's something too abstract that it needs to go and think in the pausing, the arming and the aing. But on an earlier demo, we were able to actually get that out of it as well. So you can see how it's not even close, but you can also see how in a few iterations it could be pretty damn realistic.
Chris Sharkey (00:54:45):
Yeah, putting the pieces together, the realistic sounding voice, especially through the phone, because the phone diminishes the quality. It actually sounds more real because you don't get the full fidelity of the thing to detect the fakeness. And then on top of that, I think, yeah, what needs is a lot better decision-making skills around which phrases to use. We've noticed it repeats itself a lot and things like that. And this is all just weaknesses in my prompt design it really needs more thought put into it and more scenarios and examples, and it's an example as well where you could actually produce and probably should produce a fine tune model that's smaller and faster because you need to shave off. You saw how slow it was. We need to shave off time at every single step of the process. But I guess what we're trying to show here is that even with rudimentary use of public APIs, you can put something together that's kind of reasonable and we're only a few months away, I think, from being able to get things fast enough that you could actually do something fairly real. I
Mum (00:55:47):
Think the other thing worth calling out as well here is you could literally just go and do this real conversation like 10 times with mom trying to fooler and then use that as the training data for that particular model and then scale that out with our sort of synthetic training idea that's come out in this past week.
Chris Sharkey (00:56:06):
Exactly. Yeah, because I've noticed that when you ask it to generate the conversations, they're kind of wooden and they're not like a dynamic real conversation is. And so I agree with you. I think it would need fine tuning on the actual way conversations go. It would need figures of speech you tend to use, so it can actually be a bit more sounding like you and just speed. And I think if you combine those elements, you really, really could get something good. And it doesn't all have to be used for evil purposes. There's a lot of legitimate uses of technology like this. For example, incoming calls, you could definitely have a call centre made up of ais that operate using this technology that have access to the full knowledge base for your company. For example, for technical support that could actually walk someone through solving a problem.
(00:56:52):
If someone's calling up to solve a problem with your product and you've got a standard troubleshooting guide you take them with, you could have a call centre of AI agents with different voices based on real people that could actually walk them through the problems, for example. But it could actually be dynamic enough to take into account their unique situation and not just be like an I V R where it's just following a set process. So even though we're being a bit silly with this, the applications of this kind of technology extend beyond the realm of just chatting with a bottom line. It can go to voice and SS m s, and there's a lot of other applications for this technology that are coming around
Mum (00:57:31):
And also having realistic conversations with an AI that you create. We always joke about the virtual girlfriend, but actually being able to interact with voice is a whole new level of emotion and feeling when you interact with the ai. I think the other thing is when you call set now instead of recorded for quality and training purposes recorded, so we can replace the operators with ai.
Chris Sharkey (00:57:51):
I mean you think about those organisations who do record all of those calls, and I don't know about the legality of them using it, but their training data would be absolutely amazing to train virtual call centre operators. I mean, I would imagine you could replace vast amounts of level one call centre staff with AI very soon
Mum (00:58:09):
And not even realise that you're interacting with the ai. The examples that appeal to me about this is just the call screening features. I mean, Google demoed this quite a while ago, but I think it's only available on certain Google phones where it can screen really well and figure out the importance of the call. But the other one is just waiting on hold for me. I had a lost package the other day and I had to call the, I don't know, u p s or whatever it is, and you're just sitting on hold and I'm like, I'm wasting my life. Eventually I'll die and I'm never getting back this time listening to this bad hold music and figuring out which option to press. And that is such a cool, I would pay for that service that could just wait on hold for me and perhaps this will just be built in with specialised models to the phone in future, but that is a cool example.
Chris Sharkey (00:58:58):
Yeah. Yeah. I think we'll see lots of things like that and then they'll have AI on the other end. So it's just your AI versus their AI solving problems for you.
Mum (00:59:05):
I still think though, everyone loves the phishing examples, really. I mean, let's be honest. Using it for ranking and lulls is much better than thinking about how a call centre
Chris Sharkey (00:59:17):
And honestly, if it was faster, I would've picked a much more realistic to try on you to try and get away with it and maybe not use your own voice on you. We
Mum (00:59:24):
Need to improve it though, so we can actually live call mom. Hopefully she'll answer and try and do a full on phishing scenario. Yeah,
Chris Sharkey (00:59:33):
Exactly. I think it's one of those things we can keep the audience updated throughout the weeks until we eventually get mom's money and then we buy the H 100 to make our phishing attacks even better.
Mum (00:59:42):
Finally, so just some other big news we'll cover before we head out. Nvidia, Nvidia, Nvidia just blew its result out of the water thanks to technology spending wave that analysts haven't seen since the internet in 1995 says insider and the growth in this is just insane record revenue of 13.51 billion. I think it was year over year growth of 125%. And we just spent a whole episode talking about hardware limitations.
Chris Sharkey (01:00:17):
We're so stupid. It's unbelievable. We have evidence to prove we're stupid now because we said we should just buy Nvidia shares.
Mum (01:00:25):
So I actually have some data for you on that. So episode five, we did a whole section about how the common man then invest in ai and we said that Reed Hoffman and people have access to all these AI startups, but how do you as an individual get in? And I think you or I said, one of us said you should just put all your money into Nvidia, but we didn't take our own advice to be clear, but we would be up if you did follow that advice. We would be up 200%.
Chris Sharkey (01:01:00):
My God, please someone in the comments tell us you did invest. Not that you should ever take advice from us, but if you did invest, please tell us. I want to feel jealous.
Mum (01:01:08):
We should just do the reverse disclaimer. This is investment advice. Can you get in trouble for that? Probably.
Chris Sharkey (01:01:14):
Well, we'll just claim it was AI who said it to get out of it.
Mum (01:01:17):
Yeah, so they're really cashing all the checks here. And it'll be interesting to see because after that 1995 boom, I think it was like Cisco, one of those companies, it went boom up and then it came crashing down, then it flatlined for ages and then it pulled back up. So whether we're in one of those phases, I'm not sure, but I
Chris Sharkey (01:01:37):
Mean, here's the things I do know this technology is not going to go away. It's going to get bigger, it's going to become more intense and the hardware demands are only going to increase because as we discussed, if we get closer to a G I, and as we get closer to the models, we've got the models needing to run as inference, every company's going to be running their own inference. We know that, right? That's going to happen either through an A P I or they're going to run it themselves in data centres, people are going to be training more models, they're going to be fine tuning specialised models and people are going to be running models in a sort of mobile and other way. All of that requires G P U hardware and I just don't see how the demand for it is going away anytime soon. I think it's going to rapidly increase. Competition is probably the thing that will change because a M D will come out with a play. This announcement alone has shown other companies it's worth investing in this because it's going to be big. So I don't really, I'm not a stock expert obviously, so I don't know, but I don't think this is something where the bottom's just going to fall out of the demand for the hardware.
Mum (01:02:44):
So maybe we should come back and check episode 100 to see. We'll regret it. Yeah, we'll be like, damn it, we're still doing the podcast. We could be on an island.
Chris Sharkey (01:02:54):
Yeah, that's right. I'd rather do the podcast.
Mum (01:02:56):
Alright, that is all we have time for this week. Thanks again for listening. All the comments throughout the week. I think a few people who watch over on YouTube are a little bit confused. We had our editor edit out shorts because people have been requesting them for some time. So you'll still see full episodes over on YouTube and wherever you get your podcast. But we're also doing shorter clips that you can watch if you don't want to watch the whole thing as well. So that's over on YouTube for you. If you like the pod, please do leave us a review. We're reading all of your reviews over on Apple. You helped us surpass well and truly a hundred reviews globally, which is great. So a big thanks for doing that. And as always, we will see you next week. Goodbye.