Pretrained

Pierce and Richard are back for the second listener mailbag. They break down what reward hacking really is and why models so often learn the wrong lesson, explain practical fine-tuning (from pre-training to prompting), unpack why LLMs use tokens instead of words, how context length is a hardware versus mathematic limitation, and much more.

What is Pretrained?

Everyone's talking about AI but most of it is hype, jargon, or someone trying to sell you something. Pierce Freeman and Richard Diehl Martinez are two Stanford friends who actually work in this space with nothing to sell you but good vibes. We'll make you smarter - and you might actually enjoy it.