{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Thinking Machines: AI & Philosophy","title":"Pre-training LLMs: One Model To Rule Them All? with Talfan Evans, DeepMind","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/51cc5d0d\"></iframe>","width":"100%","height":180,"duration":2256,"description":"Talfan Evans is a research engineer at DeepMind, where he focuses on data curation and foundational research for pre-training LLMs and multimodal models like Gemini. I ask Talfan: Will one model rule them all?What does \"high quality data\" actually mean in the context of LLM training?Is language model pre-training becoming commoditized?Are companies like Google and OpenAI keeping their AI secrets to themselves?Does the startup or open source community stand a chance next to the giants?Also check out Talfan's latest paper at DeepMind, Bad Students Make Good Teachers.","thumbnail_url":"https://img.transistorcdn.com/S6OjXZjcpOAZ6jDX4fc4XvtZpoBLPUwb1-xRPS1F5K0/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQ1MTk5LzE3MDg3/MDIxODItYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}