1 00:00:03,060 --> 00:00:05,520 Evan: Welcome to Daily Paper Cast. 2 00:00:05,568 --> 00:00:13,008 Ashley: Today's paper comes from the Hugging Face daily paper list of October 2, 2026, and it has 32 upvotes. 3 00:00:13,056 --> 00:00:17,236 Evan: The title of the paper is: Sharpening Tax in Post-Training. 4 00:00:17,280 --> 00:00:29,960 Ashley: It is authored by Changdae Oh from Meta Superintelligence Labs and the University of Wisconsin–Madison, Qi Zeng from Meta, and Qi Qi from Meta, with Changdae Oh as the corresponding author. 5 00:00:30,084 --> 00:00:32,674 Evan: Alright, let's dive into the introduction. 6 00:00:32,674 --> 00:00:37,024 Ashley, can you give us some background on why this paper is compelling? 7 00:00:37,080 --> 00:00:37,950 Ashley: Sure. 8 00:00:37,950 --> 00:00:47,140 The paper discusses reinforcement learning, or RL, post-training of large language models, commonly referred to as LLMs. 9 00:00:47,140 --> 00:00:59,780 The emerging hypothesis addressed here is quite interesting — RL post-training may merely enhance existing behaviors of a base model, improving single-shot accuracy but potentially limiting solution coverage. 10 00:00:59,832 --> 00:01:03,092 Evan: That sounds like it could have significant implications. 11 00:01:03,092 --> 00:01:05,892 Can you explain the specific objectives of the paper? 12 00:01:05,952 --> 00:01:16,602 Ashley: The main objective of the paper is to investigate whether RL post-training genuinely expands the reasoning capabilities of a base model or if it merely sharpens certain behaviors. 13 00:01:16,602 --> 00:01:25,652 The authors aim to understand this trade-off in various domains, including math, coding, and agentic tasks that require multi-turn interactions. 14 00:01:25,764 --> 00:01:29,004 Evan: And how do they approach this investigation? 15 00:01:29,064 --> 00:01:37,064 Ashley: The authors carry out a systematic study on LLMs' reasoning boundaries on agentic tasks before and after post-training. 16 00:01:37,064 --> 00:01:40,924 They compare performance across multiple benchmarks and checkpoints. 17 00:01:40,924 --> 00:01:54,904 Interestingly, they found that pre-trained LLMs, when equipped with a lightweight inference harness, can perform agentic reasoning tasks quite effectively, occasionally surpassing their post-trained counterparts given enough test-time resources. 18 00:01:54,960 --> 00:01:57,250 Evan: That is quite surprising. 19 00:01:57,250 --> 00:02:00,000 What other contributions does the paper make? 20 00:02:00,048 --> 00:02:02,328 Ashley: There are several key contributions. 21 00:02:02,328 --> 00:02:11,168 First, they introduce the concept of Sharpening Tax — a diagnostic metric that quantifies the loss of test-time scalability after post-training. 22 00:02:11,168 --> 00:02:17,068 This metric measures how post-training affects test-time solution coverage and efficiency systematically. 23 00:02:17,068 --> 00:02:24,388 Second, across various model families and agentic benchmarks, they found that the Sharpening Tax is a prevalent issue. 24 00:02:24,492 --> 00:02:32,712 Evan: So it sounds like the Sharpening Tax has practical ramifications for the deployment of LLMs particularly in complex environments. 25 00:02:32,712 --> 00:02:35,172 Is there a proposed solution in the paper? 26 00:02:35,232 --> 00:02:42,492 Ashley: Yes, the authors propose an innovative solution called posterior-tempered group sampling, or PTGS. 27 00:02:42,492 --> 00:02:48,372 This is a Bayesian sampler that adjusts the sampling temperature based on the estimated difficulty of each prompt. 28 00:02:48,372 --> 00:02:55,512 The idea is to dynamically balance exploration and exploitation during RL training to mitigate the Sharpening Tax. 29 00:02:55,512 --> 00:03:01,412 Empirical results show that PTGS achieves better accuracy and coverage simultaneously. 30 00:03:01,464 --> 00:03:03,604 Evan: That's a pretty comprehensive approach. 31 00:03:03,604 --> 00:03:06,564 Anything else from the introduction section that stands out? 32 00:03:06,654 --> 00:03:10,574 Ashley: One more thing worth mentioning is the theoretical analysis provided. 33 00:03:10,574 --> 00:03:16,824 The authors not only deliver empirical findings but also explain the mechanisms behind their observations. 34 00:03:16,824 --> 00:03:24,224 This includes why post-training sharpens certain behaviors and how PTGS effectively addresses the associated tax. 35 00:03:24,288 --> 00:03:27,888 Evan: It sounds like this paper has a lot to offer to the field. 36 00:03:27,888 --> 00:03:30,148 That wraps up the introduction section. 37 00:03:31,454 --> 00:03:35,484 Evan: Ashley, let's delve into the methods used in this paper. 38 00:03:35,484 --> 00:03:39,074 Can you walk us through the key elements of their methodology? 39 00:03:39,152 --> 00:03:40,002 Ashley: Evan. 40 00:03:40,002 --> 00:03:54,162 The paper's methodology can be divided into several key components: the metrics for evaluation, the datasets and environments used, the model checkpoints and their serving setups, and the inference protocol with and without harnesses. 41 00:03:54,278 --> 00:03:55,908 Evan: Let's start with the metrics. 42 00:03:55,908 --> 00:03:59,238 How did the authors evaluate the performance of these models? 43 00:03:59,282 --> 00:04:05,122 Ashley: The authors focused on three primary metrics: pass@1, pass@K, and consistency. 44 00:04:05,122 --> 00:04:10,832 These metrics are designed to evaluate the reasoning boundaries of LLMs under different conditions. 45 00:04:10,832 --> 00:04:17,262 Pass@1 captures the empirical pass rate of a single trial, which reflects the model's accuracy on the first attempt. 46 00:04:17,262 --> 00:04:24,772 Pass@K measures the probability that at least one of K rollouts succeeds, which indicates the model's solution coverage under a given budget. 47 00:04:24,772 --> 00:04:33,582 Finally, passK assesses the probability that all K rollouts succeed, reflecting the model's consistency or reliability across multiple attempts. 48 00:04:33,626 --> 00:04:38,836 Evan: These metrics seem crucial for understanding the trade-offs involved in post-training effects. 49 00:04:38,836 --> 00:04:42,826 What datasets and environments did the authors use for their evaluation? 50 00:04:42,890 --> 00:04:54,940 Ashley: They utilized three agentic benchmarks that require multi-turn tool calling and environment interaction: BFCL v4 multi-turn base split, WebShop, and ACEBench. 51 00:04:54,940 --> 00:05:00,550 These benchmarks are designed to evaluate iterative reasoning and tool usage in dynamic environments. 52 00:05:00,550 --> 00:05:12,810 Importantly, success in these benchmarks is determined by the state reached through intermediate actions rather than merely matching a final answer string, which adds a layer of complexity to the evaluation. 53 00:05:12,866 --> 00:05:14,936 Evan: That sounds like a rigorous setup. 54 00:05:14,936 --> 00:05:19,086 Could you explain the model checkpoints and how they were served during the experiments? 55 00:05:19,130 --> 00:05:19,960 Ashley: Certainly. 56 00:05:19,960 --> 00:05:33,720 The authors utilized open-source checkpoint pairs from four model families: Gemma-4, Ministral-3, Qwen2.5, and Qwen3.5, with models ranging from 3 billion to 35 billion parameters. 57 00:05:33,720 --> 00:05:38,870 These pairs consist of a pre-trained base model and its corresponding post-trained model. 58 00:05:38,870 --> 00:05:45,350 The post-trained models may involve reinforcement learning, supervised fine-tuning, or preference optimization. 59 00:05:45,410 --> 00:05:46,450 Evan: Interesting. 60 00:05:46,450 --> 00:05:49,630 How were these models served during the inference process? 61 00:05:49,712 --> 00:06:00,252 Ashley: All models were served using vLLM 0.19.0, except Gemma-4-12B, which used vLLM 0.23. 62 00:06:00,252 --> 00:06:09,232 Base models received a raw text prompt without any chat template, while post-trained models were queried with their native chat templates and tool-call parsers. 63 00:06:09,232 --> 00:06:15,482 Additionally, for fair comparison, thinking mode was turned off for all post-trained models that support it. 64 00:06:15,590 --> 00:06:17,980 Evan: And what about the harnesses? 65 00:06:17,980 --> 00:06:20,830 How did they impact the inference process? 66 00:06:20,882 --> 00:06:24,062 Ashley: The harness played a crucial role, especially for base models. 67 00:06:24,062 --> 00:06:29,462 It consists of a simple system prompt combined with relaxed tool-calling and parsing interfaces. 68 00:06:29,462 --> 00:06:38,422 In essence, the harness adapts the conversation into a plain-text prompt for the completion endpoint and parses the free-text completion back into structured tool calls. 69 00:06:38,422 --> 00:06:49,482 This design dramatically improves both pass@1 and pass@32 for base models on benchmarks where they particularly struggle, while sometimes hurting the performance of post-trained models. 70 00:06:49,538 --> 00:06:50,998 Evan: That makes sense. 71 00:06:50,998 --> 00:06:57,138 Moving on, how did they handle inference-time considerations and potential performance biases? 72 00:06:57,194 --> 00:06:59,984 Ashley: Inference configurations were handled carefully. 73 00:06:59,984 --> 00:07:05,424 Sampling parameters were fixed for each benchmark and shared by base and post-trained models. 74 00:07:05,424 --> 00:07:08,674 This ensured that comparisons were fair and systematic. 75 00:07:08,674 --> 00:07:19,624 For example, temperature settings were 0.4 for BFCL and 0.7 for WebShop and ACEBench, taking into account each environment's tool-call parsing reliability. 76 00:07:19,624 --> 00:07:27,034 Additionally, evaluation employed sufficient rollouts to provide robust and unbiased insights into the models' performance. 77 00:07:27,158 --> 00:07:33,268 Evan: It's evident that this paper has been methodically structured to minimize biases and optimize clarity. 78 00:07:33,268 --> 00:07:35,918 Anything else about the methods that stands out? 79 00:07:35,978 --> 00:07:40,338 Ashley: One more point worth mentioning is their approach to measuring the Sharpening Tax. 80 00:07:40,338 --> 00:07:51,408 They computed the tax using comprehensive metrics such as raw area scalability and calibrated scalability, which reflect how well additional rollouts improve performance post-training. 81 00:07:51,408 --> 00:07:59,558 This methodological rigor ensures that the Sharpening Tax is not just a theoretical concept but one that can be decisively quantified. 82 00:07:59,618 --> 00:08:00,488 Evan: That's great. 83 00:08:00,488 --> 00:08:03,698 It sounds like their methods were robust and well thought out. 84 00:08:03,698 --> 00:08:06,198 That brings us to the end of the Method section. 85 00:08:07,443 --> 00:08:12,793 Evan: Ashley, it's time to discuss the experiments and results presented in this paper. 86 00:08:12,793 --> 00:08:15,623 What can you tell us about their experimental setup? 87 00:08:15,675 --> 00:08:18,255 Ashley: The experiments were quite comprehensive. 88 00:08:18,255 --> 00:08:31,565 The authors systematically evaluated 14 base and post-trained model pairs across three agentic benchmarks: BFCL v4 multi-turn base split, WebShop, and ACEBench. 89 00:08:31,565 --> 00:08:39,715 They focused on understanding how post-training affects solution coverage and efficiency compared to base models equipped with a lightweight harness. 90 00:08:39,771 --> 00:08:48,031 Evan: So, they evaluated both metrics we talked about earlier, pass@1 and pass@K, but also added passK, correct? 91 00:08:48,075 --> 00:08:49,165 Ashley: Exactly. 92 00:08:49,165 --> 00:08:55,875 They analyzed these metrics to draw detailed comparisons between the base models and their post-trained counterparts. 93 00:08:55,875 --> 00:09:03,595 Additionally, they categorized tasks based on their outcomes into always pass, pass given compute, and always fail. 94 00:09:03,595 --> 00:09:08,535 This helped them understand the distribution of success and failures for each model. 95 00:09:08,595 --> 00:09:10,025 Evan: That's interesting. 96 00:09:10,025 --> 00:09:13,995 So what were their major findings regarding these experiments? 97 00:09:14,043 --> 00:09:23,933 Ashley: One of their key observations was that larger models handled agentic tasks more efficiently under post-training, but this efficiency came with a higher Sharpening Tax. 98 00:09:23,933 --> 00:09:36,223 For small models, post-training provided significant improvements, while for larger models, the base model equipped with the harness often surpassed the post-trained model in solution coverage given enough rollouts. 99 00:09:36,327 --> 00:09:40,917 Evan: It sounds like the authors systematically compared coverage and efficiency. 100 00:09:40,917 --> 00:09:46,727 Did they find that the Sharpening Tax is consistently charged across different model families and benchmarks? 101 00:09:46,779 --> 00:09:48,169 Ashley: Yes, they did. 102 00:09:48,169 --> 00:09:55,109 Across all 42 model-benchmark combinations, the authors found that Sharpening Tax is a prevalent issue. 103 00:09:55,109 --> 00:10:09,619 For the largest backbones, the tax is positive at almost every budget and benchmark, indicating that while post-training yields higher single-shot accuracy, it sacrifices the broader solution coverage achievable by base models under repeated sampling. 104 00:10:09,705 --> 00:10:13,915 Evan: And how did they quantify this loss using the Sharpening Tax metric? 105 00:10:13,971 --> 00:10:25,341 Ashley: They introduced two forms of the tax: TaxA, which represents the raw area scalability, and TaxS, which is the calibrated scalability, normalized by the single-shot failure rate. 106 00:10:25,341 --> 00:10:30,711 Both measures help predict how well additional compute resources improve test-time performance. 107 00:10:30,711 --> 00:10:39,051 The authors showed that TaxS could be estimated from a small number of rollouts, providing a reliable early predictor of eventual scalability loss. 108 00:10:39,099 --> 00:10:44,299 Evan: That sounds like a practical approach to diagnosing post-training effects on model performance. 109 00:10:44,299 --> 00:10:48,399 Did they provide any visualizations or case studies in their results? 110 00:10:48,459 --> 00:10:58,709 Ashley: Yes, they visualized the scaling curves of individual tasks and provided density plots showing how tasks shift their success rates from the base to the post-trained models. 111 00:10:58,709 --> 00:11:06,109 This included case studies where post-training consistently moved tasks towards extremes, creating a bimodal distribution. 112 00:11:06,109 --> 00:11:13,779 Tasks that were occasionally solved by base models were frequently pushed to always solved or never solved categories post-training. 113 00:11:13,827 --> 00:11:17,087 Evan: This bimodalization seems quite significant. 114 00:11:17,087 --> 00:11:21,467 How did PTGS help mitigate these effects according to their experiments? 115 00:11:21,661 --> 00:11:25,981 Ashley: PTGS effectively balanced the tension between accuracy and coverage. 116 00:11:25,981 --> 00:11:34,091 It achieved higher entropy in action sequences, encouraging more exploratory behavior for prompts the policy initially struggled with. 117 00:11:34,091 --> 00:11:46,771 The authors showed that PTGS reduced the proportion of zero-success rollout groups and increased the likelihood of mixed groups containing both successes and failures, thereby enhancing learning signals during training. 118 00:11:46,887 --> 00:11:50,467 Evan: It seems PTGS played a crucial role. 119 00:11:50,467 --> 00:11:55,927 What were the specific empirical results on performance improvements with PTGS? 120 00:11:56,001 --> 00:12:03,731 Ashley: Empirical results demonstrated that PTGS could improve both pass@1 and pass@128 significantly. 121 00:12:03,731 --> 00:12:12,311 For instance, PTGS achieved better accuracy and coverage simultaneously compared to fixed-temperature PPO and baseline models. 122 00:12:12,311 --> 00:12:22,391 On Sokoban and FrozenLake, PTGS runs maintained higher validation success and better avoided the performance losses observed in standard RL training. 123 00:12:22,443 --> 00:12:23,933 Evan: That's quite remarkable. 124 00:12:23,933 --> 00:12:27,063 Any final takeaways from their experiments and results? 125 00:12:27,123 --> 00:12:37,633 Ashley: The key takeaway is that while RL post-training sharpens LLM behavior, it comes with a significant cost in solution coverage, especially for larger models. 126 00:12:37,633 --> 00:12:51,163 PTGS offers a promising way to balance this accuracy-coverage trade-off, reinforcing the importance of dynamic adjustment in sampling strategies to leverage diverse reasoning capabilities without sacrificing robustness. 127 00:12:51,219 --> 00:12:54,319 Evan: That wraps up our discussion on the experiment section. 128 00:12:55,595 --> 00:12:59,945 Evan: Let's now turn our attention to the Related Work section of the paper. 129 00:12:59,945 --> 00:13:05,365 Ashley, how do the authors situate their study within the existing body of research? 130 00:13:05,429 --> 00:13:12,219 Ashley: The authors provide a detailed survey of the landscape around the topic of RL post-training for large language models. 131 00:13:12,219 --> 00:13:17,049 They identify multiple lines of inquiry and debate that are pertinent to their investigation. 132 00:13:17,153 --> 00:13:20,393 Evan: And what are these lines of inquiry that they've highlighted? 133 00:13:20,453 --> 00:13:31,573 Ashley: One of the main debates is whether RL post-training fundamentally expands the reasoning capabilities of LLMs or merely sharpens pre-existing behaviors within the models. 134 00:13:31,573 --> 00:13:41,053 This question has been explored from various angles by researchers, some advocating for the expansion of capabilities through RL, while others remain skeptical. 135 00:13:41,187 --> 00:13:45,657 Evan: Can you give us specifics on what these researchers have found? 136 00:13:45,701 --> 00:13:58,051 Ashley: Studies by Yue et al., 2025 and Zhao et al., 2025, for example, observe that RL-tuned models often show enhanced accuracy at the cost of solution coverage. 137 00:13:58,051 --> 00:14:07,121 They propose that the RL process amplifies a narrow set of successful behaviors while suppressing other potential trajectories the base model may take. 138 00:14:07,181 --> 00:14:12,721 Evan: So there's a consensus forming that sharpening seems to be what's happening rather than expansion? 139 00:14:12,773 --> 00:14:13,923 Ashley: Precisely. 140 00:14:13,923 --> 00:14:24,263 However, there's also research suggesting that RL can genuinely expand the reasoning boundaries through mechanisms like prolonged training or specific data properties. 141 00:14:24,263 --> 00:14:31,853 Liu et al., 2025a argue for extended RL training periods to see significant capabilities emerge. 142 00:14:31,853 --> 00:14:41,053 Sun et al., 2026 explain that RL might unlock novel algorithmic constructs in LLMs, referred to as 'grokking'. 143 00:14:41,117 --> 00:14:42,177 Evan: Interesting. 144 00:14:42,177 --> 00:14:46,317 Are there any studies that consider the data dependencies in this process? 145 00:14:46,373 --> 00:14:47,623 Ashley: Yes, indeed. 146 00:14:47,623 --> 00:14:59,723 Zhang et al., 2025 and Shen et al., 2026a focus on how the properties and distribution of training data influence whether RL sharpens or expands reasoning capabilities. 147 00:14:59,723 --> 00:15:05,913 They suggest that RL's impact might vary depending on the richness and diversity of the pre-training data. 148 00:15:05,957 --> 00:15:08,657 Evan: It sounds like a multi-faceted debate. 149 00:15:08,657 --> 00:15:13,957 What about studies specific to agentic tasks, since this paper emphasizes those? 150 00:15:14,051 --> 00:15:15,751 Ashley: This is a crucial point. 151 00:15:15,751 --> 00:15:22,581 Research toward agentic tasks like multi-turn tool use and environment interaction is less commonly reported. 152 00:15:22,581 --> 00:15:29,971 Most studies referenced so far mainly address domains like math and coding, where a final answer suffices for evaluation. 153 00:15:29,971 --> 00:15:38,581 Agentic reasoning tasks demand continual interaction and feedback processing, which adds complexity not captured in these conventional benchmarks. 154 00:15:38,645 --> 00:15:43,825 Evan: How do the authors differentiate their work in this context as compared to previous studies? 155 00:15:43,877 --> 00:15:50,837 Ashley: The authors argue that agentic tasks serve as rigorous evaluation suites to test the sharpening hypothesis. 156 00:15:50,837 --> 00:16:03,477 They claim that while traditional benchmarks may show significant improvement due to RL, agentic benchmarks demonstrate the scalability and robustness of a model when faced with dynamic, multi-turn interactions. 157 00:16:03,613 --> 00:16:07,273 Evan: What other aspects of related work do they cover? 158 00:16:07,325 --> 00:16:12,695 Ashley: There’s also substantial literature on preserving policy diversity during RL post-training. 159 00:16:12,695 --> 00:16:23,185 Yu et al., 2025 and He et al., 2025 look into techniques like exploration bonuses and entropy mitigation to prevent collapse of policy diversity. 160 00:16:23,185 --> 00:16:33,085 Another approach is pass@k-aware optimization, proposed by Walder and Karkhanis, 2025, which aims to balance solution diversity with accuracy. 161 00:16:33,085 --> 00:16:39,885 These are complementary to the sampling-side solutions, where techniques like temperature scheduling play a significant role. 162 00:16:40,011 --> 00:16:41,441 Evan: Quite fascinating. 163 00:16:41,441 --> 00:16:45,831 It seems PTGS fits well within these sampling-side solutions. 164 00:16:45,831 --> 00:16:50,061 Do the authors discuss how PTGS differs from other methods? 165 00:16:50,117 --> 00:16:51,347 Ashley: They certainly do. 166 00:16:51,347 --> 00:17:00,047 PTGS distinguishes itself by dynamically adjusting sampling temperature based on per-task difficulty using a Beta-Binomial posterior. 167 00:17:00,047 --> 00:17:08,327 Its plug-and-play capability, without needing modifications to existing RL algorithms, sets it apart as a flexible and effective tool. 168 00:17:08,327 --> 00:17:17,277 This makes it particularly suitable for various RL pipelines, optimizing for accurate and diverse reasoning without significant computational overhead. 169 00:17:17,333 --> 00:17:23,563 Evan: It's clear that PTGS is positioned as an innovative and dynamic solution within this landscape. 170 00:17:23,563 --> 00:17:26,233 Anything else in the Related Work that stands out? 171 00:17:26,285 --> 00:17:35,235 Ashley: The authors also touch upon test-time parallel scaling through repeated sampling, which has been a reliable axis for improving LLM performance. 172 00:17:35,235 --> 00:17:43,735 They highlight that while repeated sampling boosts coverage, understanding the trade-offs and diagnosing the impact of post-training remains crucial. 173 00:17:43,735 --> 00:17:50,525 Their approach to using test-time scaling not just for performance but also as a diagnostic tool is quite novel. 174 00:17:50,573 --> 00:17:55,803 Evan: Sounds like their work ties into multiple significant threads in the current research landscape. 175 00:17:55,803 --> 00:17:58,913 That brings us to the end of the Related Work section. 176 00:18:00,234 --> 00:18:04,114 Evan: Ashley, we've unpacked various aspects of this paper. 177 00:18:04,114 --> 00:18:08,514 Can you summarize the key contributions and takeaways for our listeners? 178 00:18:08,574 --> 00:18:09,934 Ashley: Certainly, Evan. 179 00:18:09,934 --> 00:18:13,094 The paper makes several impactful contributions. 180 00:18:13,094 --> 00:18:23,054 Firstly, it introduces the Sharpening Tax, a diagnostic metric to measure the loss in test-time scalability and solution coverage caused by RL post-training. 181 00:18:23,054 --> 00:18:29,554 This metric quantifies the trade-off between single-shot accuracy and broader solution coverage systematically. 182 00:18:29,658 --> 00:18:30,318 Evan: Got it. 183 00:18:30,318 --> 00:18:34,618 And this seems especially relevant considering their extensive experiments. 184 00:18:34,692 --> 00:18:45,492 Ashley: Across 14 model pairs, spanning various sizes and families, and evaluated on three rigorous agentic benchmarks, they demonstrated the prevalence of the Sharpening Tax. 185 00:18:45,492 --> 00:18:54,802 Their findings show that while post-training improves single-shot accuracy, it consistently sacrifices solution coverage, particularly for larger models. 186 00:18:54,846 --> 00:18:57,306 Evan: That's quite a significant observation. 187 00:18:57,306 --> 00:19:01,166 How does the proposed PTGS method factor into this? 188 00:19:01,420 --> 00:19:07,110 Ashley: PTGS, or posterior-tempered group sampling, is a standout solution proposed in the paper. 189 00:19:07,110 --> 00:19:16,690 By adjusting the sampling temperature dynamically based on the estimated difficulty of each prompt, PTGS effectively balances exploration and exploitation. 190 00:19:16,690 --> 00:19:24,610 This leads to notable improvements in both pass@1 and pass@K, ameliorating the adverse effects of the Sharpening Tax. 191 00:19:24,654 --> 00:19:33,134 Evan: It's clear that PTGS offers a promising approach to maintaining robust and diversified reasoning capabilities during RL training. 192 00:19:33,198 --> 00:19:34,108 Ashley: Indeed. 193 00:19:34,108 --> 00:19:39,218 What's particularly compelling is the blend of empirical and theoretical analysis. 194 00:19:39,218 --> 00:19:50,018 The authors not only provide data-driven insights but also delve into the mechanisms behind why post-training sharpens behaviors and how PTGS mitigates these effects. 195 00:19:50,070 --> 00:19:54,420 Evan: This comprehensive approach surely adds depth to their contributions. 196 00:19:54,420 --> 00:19:56,210 Ashley, let's wrap this up. 197 00:19:56,210 --> 00:19:58,350 Any final remarks for our listeners? 198 00:19:58,398 --> 00:20:12,108 Ashley: In summary, this paper provides crucial insights into the trade-offs of RL post-training on LLMs, introduces an innovative diagnostic metric, and offers a robust solution to balance accuracy and coverage. 199 00:20:12,108 --> 00:20:19,498 It's a must-read for anyone involved in AI research or the deployment of language models in complex, dynamic environments. 200 00:20:19,542 --> 00:20:20,962 Evan: Thank you, Ashley. 201 00:20:20,962 --> 00:20:25,602 And thank you to our listeners for tuning into today's episode of Daily Paper Cast. 202 00:20:25,662 --> 00:20:28,372 Ashley: We hope you found this discussion informative. 203 00:20:28,372 --> 00:20:34,182 Be sure to join us next time as we continue to explore groundbreaking research from the AI community. 204 00:20:34,290 --> 00:20:38,050 Evan: Until then, keep learning and stay curious!