{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Pop Goes the Stack","title":"Training vs Inference: Are they the same?","html":"","width":"100%","height":180,"duration":1289,"description":"Training and inference get lumped together in casual AI conversations, but they behave differently enough that the distinction matters for cost, architecture, and security. In this episode of Pop Goes the Stack, Lori MacVittie, Joel Moses, Ken Arora, and Kevin Baughman (who leads F5’s AI Center of Excellence) unpack what’s truly different, what’s the same, and where people get misled. Joel makes the “math is the same” case: both phases run similar computations, but training must retain intermediate activations for backpropagation, while inference can discard them. Ken and Kevin pull the conversation back to practical differences: training is about baking knowledge into the model, while inference is about using a frozen model and shaping behavior with context, retrieval, and few-shot examples. The weights don’t change during inference; the input does, which is why it can feel like “learning” without actually being permanent. That distinction becomes a security and governance lever. If you don’t want sensitive or proprietary data baked into a model, you avoid training on it and instead keep it in a controlled knowledge base (RAG or similar) that can be updated, removed, or scoped per tenant. Meanwhile, training pipelines emphasize massive data ingestion and throughput, and inference emphasizes responsiveness, session context, and efficient serving at scale. The practical takeaway is to stop treating “AI workloads” as one thing. Training and inference require different pipeline designs, different tradeoffs in memory and bandwidth, and different approaches to data control. Pick your phase, understand the constraints, and build for it intentionally.","thumbnail_url":"https://img.transistorcdn.com/EOH5giVF50GDCoaIBECLMap8fBWcZH3C5tsFwM0Tn9s/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80MGQ2/ZDBjM2JjMmMyZDg0/MGY5ZTEyYTViOTgy/N2RiYS5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}