{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Manifold","title":"AI on your phone? Tim Dettmers on quantization of neural networks — #41","html":"","width":"100%","height":180,"duration":4023,"description":"Tim Dettmers develops computationally efficient methods for deep learning. He is a leader in quantization: coarse graining of large neural networks to increase speed and reduce hardware requirements.\nTim developed 4-and 8-bit quantizations enabling training and inference with large language models on affordable GPUs and CPUs - i.e., as commonly found in home gaming rigs.\nTim and Steve discuss: Tim's background and current research program, large language models, quantization and performance, democratization of AI technology, the open source Cambrian explosion in AI, and the future of AI.\n0:00 Introduction and Tim’s background\n18:02 Tim's interest in the efficiency and accessibility of large language models\n38:05 Inference, speed, and the potential for using consumer GPUs for running large language models\n45:55 Model training and the benefits of quantization with QLoRA\n57:14 The future of AI and large language models in the next 3-5 years and beyond\n\nTim's site: https://timdettmers.com/\nTim on GitHub: https://github.com/TimDettmers\n\nMusic used with permission from Blade Runner Blues Livestream improvisation by State Azure.\n--\nSteve Hsu is Professor of Theoretical Physics and of Computational Mathematics, Science, and Engineering at Michigan State University. Previously, he was Senior Vice President for Research and Innovation at MSU and Director of the Institute of Theoretical Science at the University of Oregon. Hsu is a startup founder (SuperFocus.ai, SafeWeb, Genomic Prediction) and advisor to venture capital and other investment firms. He was educated at Caltech and Berkeley, was a Harvard Junior Fellow, and has held faculty positions at Yale, the University of Oregon, and MSU.\nPlease send any questions or suggestions to manifold1podcast@gmail.com or Steve on Twitter @hsu_steve.","thumbnail_url":"https://img.transistorcdn.com/3dN5jXpdPU-2tksLGunno-_3QSF26xY_bpkov6O7drs/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzI3Njg3LzE2NDI3/MjI3MjUtYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}