{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Machine Learning Tech Brief By HackerNoon","title":"I Built a Tiny GPT That Speaks Sanskrit in a Weekend — Here’s What Broke","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/19d8e9ee\"></iframe>","width":"100%","height":180,"duration":439,"description":"\n        This story was originally published on HackerNoon at: https://hackernoon.com/i-built-a-tiny-gpt-that-speaks-sanskrit-in-a-weekend-heres-what-broke.\nThe off-the-shelf models are bad at Sanskrit largely because of tokenization and data scarcity\nCheck more stories related to machine-learning at: https://hackernoon.com/c/machine-learning.\n            You can also check exclusive content about #transformers, #sanskrit, #from-scratch-transformer, #gpt, #build-your-own-gpt, #tokenization, #llms, #hackernoon-top-story,  and more.\nThis story was written by: @amitshukla. Learn more about this writer by checking @amitshukla's about page,\n            and for more stories, please visit hackernoon.com.\nI gave myself a weekend and a constraint: build a GPT small enough to understand completely, but on a language that would actually fight back — Sanskrit. I have a pile of Devanagari text and an NVIDIA DGX Spark sitting on my desk, so why not?","thumbnail_url":"https://img.transistorcdn.com/KyA01h2FD2insgk-wX_xzV6vbJnTNl2BvPYVL-XaI9A/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQxMjcyLzE2ODM1/ODI0ODgtYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}