{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"LLM.co","title":"KV Cache Optimization for Self-Hosted LLMs Under Load","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/dcec8847\"></iframe>","width":"100%","height":180,"duration":311,"description":"When self-hosted LLMs buckle under concurrent users, the KV cache is almost always the hidden bottleneck. This episode breaks down the optimization levers — from PagedAttention and prefix caching to KV quantization and model architecture — that let you serve far more requests without reaching for bigger hardware.","thumbnail_url":"https://img.transistorcdn.com/Xh4c5SJy9DrgNKLAiE3osoWLXnIfy23Af9qWDM3xsOc/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS8xZTc1/ZDRkMDY4NzRiMTU3/OTcyNTRmMDY5OTNi/NTcxMy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}