DeepSeek's New LLM Architectures

Pierce and Richard break down DeepSeek's latest model architecture moves in Manifold-Constrained Hyper Connections and Engram memory. Are these conceptually sound? Will they hop the pond over to US frontier labs?

What is Pretrained?

10 years after studying at Stanford, two friends have somehow become AI experts. One builds startups, the other studies at Cambridge - together they break down LLMs and machine learning with zero BS and maximum banter.