{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"80,000 Hours Podcast","title":"#141 – Richard Ngo on large language models, OpenAI, and striving to make the future go well","html":"","width":"100%","height":180,"duration":9859,"description":"Large language models like GPT-3, and now ChatGPT, are neural networks trained on a large fraction of all text available on the internet to do one thing: predict the next word in a passage. This simple technique has led to something extraordinary — black boxes able to write TV scripts, explain jokes, produce satirical poetry, answer common factual questions, argue sensibly for political positions, and more. Every month their capabilities grow. \n\nBut do they really 'understand' what they're saying, or do they just give the illusion of understanding? \n\nToday's guest, Richard Ngo, thinks that in the most important sense they understand many things. Richard is a researcher at OpenAI — the company that created ChatGPT — who works to foresee where AI advances are going and develop strategies that will keep these models from 'acting out' as they become more powerful, are deployed and ultimately given power in society. \n\nLinks to learn more, summary and full transcript. \n\nOne way to think about 'understanding' is as a subjective experience. Whether it feels like something to be a large language model is an important question, but one we currently have no way to answer. \n\nHowever, as Richard explains, another way to think about 'understanding' is as a functional matter. If you really understand an idea you're able to use it to reason and draw inferences in new situations. And that kind of understanding is observable and testable. \n\nRichard argues that language models are developing sophisticated representations of the world which can be manipulated to draw sensible conclusions — maybe not so different from what happens in the human mind. And experiments have found that, as models get more parameters and are trained on more data, these types of capabilities consistently improve. \n\nWe might feel reluctant to say a computer understands something the way that we do. But if it walks like a duck and it quacks like a duck, we should consider that maybe we have a duck, or at...","thumbnail_url":"https://img.transistorcdn.com/VO1STE7hN95RRg9QdLo4soV2VhhbR9PF5ZZlRhDYcwE/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQxNDAyLzE2ODM1/NDQ1NDAtYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}