{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Programming Tech Brief By HackerNoon","title":"Orca 2: Enhancing Reasoning in Smaller Language Models - Technical Details","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/30c4bb96\"></iframe>","width":"100%","height":180,"duration":528,"description":"\n        This story was originally published on HackerNoon at: https://hackernoon.com/orca-2-enhancing-reasoning-in-smaller-language-models-technical-details.\nOrca 2 enhances small language models' reasoning by teaching diverse strategies for tasks, outperforming models up to 10x larger in complex benchmarks.\nCheck more stories related to programming at: https://hackernoon.com/c/programming.\n            You can also check exclusive content about #language-models, #orca-2, #reasoning-techniques, #machine-learning, #small-models, #imitation-learning, #ai-benchmarks, #model-training,  and more.\nThis story was written by: @textmodels. Learn more about this writer by checking @textmodels's about page,\n            and for more stories, please visit hackernoon.com.\nThe Orca 2 dataset has four main sources:FLAN: Our main source of prompts for synthetic data generation is the FLAN-v2 Collection 33, which consists of five sub-collections. Following Orca 1 42, we consider tasks from only CoT, NiV2, T0, Flan 2021 and Dialogue. Some of the tasks are associated with an associated answer. For the Cautious Reasoning dataset we selected ~602 zero-shot user queries from the split of 1448 high quality tasks out of 1913.","thumbnail_url":"https://img.transistorcdn.com/KhCapPSRkLGL2Xw8888yuChkNRWthaKapLYTvNdu4W4/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQxMTY2LzE2ODM1/ODIzMzAtYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}