DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data

🤗 Upvotes: 21 | cs.CL, cs.AI

Authors:
Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina, Kenneth Enevoldsen, Lukas Galke Poech

Title:
DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data

Arxiv:
http://arxiv.org/abs/2608.13517v1

Abstract:
Current large language model development relies on massive, often non-permissible datasets, creating a high barrier for researchers committed to open-source and ethically sourced data. We introduce Mimir v1, a 1-billion-parameter language model based on the Hierarchical Reasoning Model (HRM) architecture, that is trained from scratch and delivers highly competitive performance for English and sets a new state of the art for Danish using only permissible post-training data. Trained on a mixture of 161 datasets, Mimir v1 outperforms the original HRM-Text 1B and competes with larger frontier models like Qwen 3.5 4B and Gemma 4 E2B, tested across 20 benchmarks for English, Math & Code and Danish. The model is available on the Hugging Face Hub: https://huggingface.co/danish-foundation-models/DFM-Mimir

What is Daily Paper Cast?

We update every weekday to discuss highest-voted papers from Huggingface Daily Paper (https://huggingface.co/papers). Both the podcast scripts and audio are generated by AI. Feedback and suggestions are welcome! Email us: [e-mail]

Creator:
Jingwen Liang, 3D ML, https://www.linkedin.com/in/jingwen-liang/
Gengyu Wang, LLM ML, http://wanggengyu.com

Listen on:
Spotify: https://open.spotify.com/show/21nrhmdaA8qoBiH8q03NXL
Apple Podcast: https://podcasts.apple.com/us/podcast/daily-paper-cast/id1777620236

Cover Image by Kawen Kuang https://kawen.art

Evan: Welcome to Daily Paper Cast.

Ashley: Today’s paper from the Hugging Face daily paper list of August 17, 2026, has garnered 21 upvotes.

Evan: The title is 'DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data.'

Ashley: It’s authored by Peter Schneider-Kamp and Jacob Nielsen, among others, with Schneider-Kamp being the corresponding author.

They are with the University of Southern Denmark and Ordbogen A/S.

Evan: Now, let’s dive into the introduction of the paper.

Ashley: Large Language Models, or LLMs, have revolutionized artificial intelligence with rapid advancements and emergent capabilities.

However, developing these models typically involves massive datasets and extensive computational resources.

Evan: Right, and this 'monolithic recipe' seems pretty exclusionary for a lot of researchers, doesn’t it?

Ashley: Exactly.

The high resource requirements create a significant barrier for many researchers and practitioners, particularly those focused on using only permissible and openly licensed data, which is the philosophy of the Danish Foundation Models project.

Evan: So how does this paper address these constraints?

Ashley: To tackle these issues, the authors employ the Hierarchical Reasoning Model, or HRM-Text framework, focusing on post-training data during the initial training phase.

This method helps in creating a viable base model without needing the vast pre-training data corpus typically required.

Evan: And what is the core contribution of this paper?

Ashley: The primary contribution is the introduction of Mimir v1, a 1-billion parameter hierarchical reasoning model trained from scratch.

This model is optimized for tasks in both Danish and English and uses only permissible post-training data.

Evan: That’s impressive.

What did they use for their training data?

Ashley: They curated a mix from 161 different datasets totaling approximately 70.5 billion tokens per epoch, all of which aligns with their permissibility standards.

They also used synthetic 'transplant datasets' to replace non-permissible data.

Evan: Synthetic alternatives?

How effective are they compared to the original datasets?

Ashley: Interestingly, the paper reports that these synthetic alternatives can achieve comparable or even superior performance without compromising data rights.

It underscores the relevance of the HRM approach for low-resource linguistic domains.

Evan: So, the model outperforms the original HRM-Text 1B and even competes with larger models like Qwen 3.5 and Gemma 4.

Is that right?

Ashley: Yes, precisely.

Mimir v1 was tested across 20 benchmarks including English, Math & Code, and specifically Danish tasks.

It delivers highly competitive performance while adhering to its ethical data use philosophy.

Evan: What’s the significance of setting a new state of the art for Danish using permissible data?

Ashley: This is quite significant because it demonstrates that you can achieve high performance and set new benchmarks even within the constraints of ethically sourced data.

It opens up possibilities for more inclusive AI research and application.

Evan: It really seems like it could democratize access to high-performing models.

That's the end of the Introduction section, folks.

We’ll move on to the methods section next.

Evan: Alright, Ashley, let’s dive into the methods used to build and evaluate Mimir v1.

Ashley: Sure, Evan.

The authors started by focusing on data.

They curated a diverse mix of 161 datasets.

These datasets cover different objectives, including English and Danish instruction, knowledge, mathematics, and agentic-style post-training data.

Evan: And how did they classify these datasets?

Ashley: The datasets were categorized into eight functional categories based on their content and intended use.

The largest categories were Danish instruction and knowledge, English instruction, and selected datasets from the Sapient mixed collection.

Evan: What’s the token distribution like across these categories?

Ashley: Of course.

Danish instruction and knowledge accounted for 22.07% of the corpus, followed by English instruction at 19.26%, and the Sapient mixed category at 17.02%.

These three categories together make up over 58% of the total corpus.

Math and reasoning data further contribute 14.8%.

Evan: And how does the language distribution look?

Ashley: Most of the corpus is in English, which makes up 68.5% of the tokens.

Danish contributes 24.7%, and there is also a small percentage of bilingual Danish-English data.

Evan: I see.

What forms does the data come in?

Ashley: The data is processed into seven distinct forms, representing different stages in the processing pipeline.

These include reformatted datasets, curated and reformatted datasets, synthetic and audited data, tool-call formatted data, translated and audited data, agreement-supplied data, and derived task data.

Evan: Can you provide a bit more detail on how synthetically generated data is created and validated?

Ashley: Sure.

Synthetic data was generated using the Gemma-4 tokenizer, producing data through methods like span-filling, denoising, reordering, and continuation tasks.

All synthetic data underwent quality audits, with varying acceptance rates dependent on the data category.

Evan: That sounds comprehensive.

How is the data repetition managed in their datasets?

Ashley: The dataset includes repeated sampling for some entries to ensure sufficient coverage.

For instance, the 'lærebogen' dataset was repeated four times to increase its sample size to 8.32 billion tokens.

Evan: Let’s move on to the architecture of the model.

What makes up the core of Mimir v1?

Ashley: Mimir v1 utilizes the Hierarchical Reasoning Model Text, or HRM-Text, architecture.

It features a hidden size of 1,536, 12 attention heads per layer, a feed-forward expansion factor of 4, and hierarchical reasoning is set up with two H-cycles and three L-cycles.

Evan: And what are the specifics of the training procedure?

Ashley: The training used Fully Sharded Data Parallelism, or FSDP, with bfloat16 as the computation type and fp32 for gathering precision.

The optimizer chosen is AdamW, with specific parameters like a peak learning rate of 0.0003 and a 2,000-step linear warmup.

The model trained for 1.65 million steps on 8 NVIDIA B200 GPUs.

Evan: Wow, that’s a sophisticated setup.

What benchmarks were used to evaluate Mimir v1?

Ashley: The authors evaluated Mimir across a range of English, Math & Code, and Danish benchmarks.

For English, they included tasks like BoolQ, Winogrande, and DROP.

For Math & Code, benchmarks included GSM8K and HumanEval.

Danish benchmarks covered grammatical tasks, question-answering, and summarization.

Evan: How did Mimir v1 perform against other models?

Ashley: Mimir v1 showed dominant performance on Danish benchmarks and delivered standout results on several English and Math & Code benchmarks.

It outperformed models like HRM-Text 1B and was competitive with larger models such as Qwen 3.5 and Gemma 4 E2B.

Evan: It sounds like Mimir is setting some new standards, particularly for Danish NLP tasks using permissible data.

Ashley: Indeed, Mimir v1 is remarkable in that it maintains high performance while strictly adhering to permissible data use, which is quite a breakthrough.

Evan: That wraps up the Method section.

Stay tuned for the next part where we’ll delve into the results and discussion.

Evan: Let's move on to the experiments and results section.

Ashley: Sure thing.

The evaluation of Mimir v1 spans 20 benchmarks across three key areas: English, Math & Code, and Danish tasks.

The goal was to measure its performance against various models in similar parameter ranges.

Evan: Let's start with the English benchmarks.

What did the results show?

Ashley: On English benchmarks, Mimir v1 performed exceptionally well.

It outperformed all considered competitors on tasks like BoolQ, Winogrande, and DROP.

For example, Mimir achieved an accuracy of 87.8% on BoolQ and 73.5% on Winogrande.

Evan: How did it fare against other 1-billion-parameter models?

Ashley: Against other 1-billion parameter models like HRM-Text and Gemma 3, Mimir showed a significant edge.

For instance, its accuracy on BoolQ was notably higher compared to HRM-Text, which scored 87.5%, and Gemma 3, which scored 62.4%.

Evan: Got it.

What about the Math and Code benchmarks?

Ashley: Math and Code benchmarks included GSM8K and HumanEval.

On GSM8K, Mimir v1 achieved a commendable accuracy of 89.9%, placing it as the leader among 1-billion parameter models.

Although it was second on GSM8K overall, it still significantly outperformed HRM-Text, which scored 84.8%.

Evan: That’s impressive.

And what about the HumanEval benchmark?

Ashley: On HumanEval, Mimir scored 56.7%, which is again higher than HRM-Text's 0% — a substantial improvement.

Evan: Those are great results.

Now, let's talk about the Danish benchmarks.

This is where Mimir v1 particularly shines, right?

Ashley: Mimir v1 outperformed all competitors on a range of Danish benchmarks.

For example, it achieved a remarkable 96.1% F1 score on the DaLA task and 85.6% on the GEC task.

The performance on WikiQA with 66.8% and Nordjylland News summarization with 35.87% chrF scores further underscored its strengths.

Evan: It sounds like it's really setting the bar high for Danish language tasks.

Ashley: Indeed.

Mimir’s performance on these Danish tasks is particularly noteworthy given that it uses only permissible datasets.

This aligns with their project’s ethical data use criteria while achieving state-of-the-art results.

Evan: So, to sum up, Mimir v1 stands out not just in Danish NLP tasks but also performs strongly in English and Math & Code benchmarks, often outperforming other state-of-the-art models within its parameter range.

Ashley: Exactly.

And this comprehensive evaluation across diverse benchmarks truly highlights Mimir v1’s capabilities and the efficacy of using permissible data for high-performing models.

Evan: That’s the end of our discussion on the experiment section.

Stay tuned as we dive into the related work and conclusions in the next part.

Evan: Alright, Ashley, let’s jump into the related work section.

What comparisons do the authors make with existing research?

Ashley: The related work highlights various key areas.

First, the authors contextualize Mimir v1 within the broader development of large language models, noting the reliance of other models on massive, often non-permissible datasets.

They particularly focus on the ethical implications of such data usage.

Evan: And how does Mimir v1 differentiate itself from these existing language models?

Ashley: Mimir v1 stands out due to its adherence to using only permissible post-training data, in stark contrast to many state-of-the-art models that depend heavily on large-scale, sometimes ethically controversial datasets.

This focus aligns with their aim to promote ethically sourced AI research.

Evan: Do they mention which models they compare against?

Ashley: Yes, they perform an extensive comparison with several major models.

HRM-Text is the immediate predecessor, and they also compare against larger models such as Qwen 3.5 and Gemma 4 E2B.

These comparisons span multiple benchmarks in English, Math & Code, and Danish tasks.

Evan: How do they assess the performance of Mimir v1 relative to these models?

Ashley: The evaluation methodology comprehensively assesses Mimir v1 across diverse benchmarks.

The results underline the model's competitive performance, showing its edge not only over its 1-billion parameter peers but also challenging larger models on several tasks.

Evan: It's interesting that they include ethical considerations in their methodology.

Do they discuss the synthetic dataset generation in the context of related work?

Ashley: Yes, they do.

The use of synthetic datasets as a substitute for non-permissible data is highlighted.

This approach is compared to works like the Common Pile and Dynaword, which faced challenges in dataset permissibility and licensing.

Mimir’s synthetic data approach offers a feasible and ethical alternative to these hurdles.

Evan: That seems like a significant advantage.

Do they touch upon any related advancements in hierarchical reasoning models?

Ashley: Definitely.

The HRM, or Hierarchical Reasoning Model, used in Mimir v1 is not an entirely new concept.

The authors reference the work by Wang et al.

from 2026, which laid the groundwork for efficient pretraining beyond mere scaling.

Mimir builds on this by integrating ethical data practices.

Evan: That’s a good connection.

How do they position their work in the broader landscape of open and ethical AI development?

Ashley: They emphasize the contribution to the broader movement towards open, transparent, and ethical AI.

Their work supports the ethos of making advanced language models accessible and reliable, without relying on ethically dubious data sources.

This stands in contrast to some large, proprietary models that prioritize performance over data ethics.

Evan: So, how does this relate to other open-source initiatives?

Ashley: They tie their work to other open-source initiatives like Hugging Face’s Transformers and various AI benchmarks from the AI Security Institute.

By benchmarking Mimir v1 against these and other public models, they demonstrate that ethical AI can also achieve high performance.

Evan: It's notable how they balance ethics and performance.

Any last thoughts on this section?

Ashley: Overall, the related work section highlights Mimir v1’s alignment with current AI ethics trends while pushing the envelope in performance, especially for low-resource languages like Danish.

Evan: That concludes our discussion on the Related Work section.

Next, we'll move on to the paper's conclusions and future directions.

Evan: Alright, let’s wrap up with the key takeaways and contributions of the paper.

Ashley: The paper presents Mimir v1, a 1-billion-parameter language model leveraging the Hierarchical Reasoning Model.

Its standout feature is the exclusive use of permissible post-training data, setting a new standard for ethical AI model development.

Evan: Through an extensive evaluation across English, Math & Code, and Danish tasks, Mimir v1 outperforms previous HRM models and competes strongly with larger models like Qwen 3.5 and Gemma 4 E2B, establishing itself as a top performer.

Ashley: A crucial aspect is its use of synthetic datasets to replace non-permissible data, maintaining high performance without compromising on ethical standards.

This approach particularly shines in Danish language tasks, reinforcing the model’s applicability for low-resource languages.

Evan: In summary, Mimir v1 is a significant step towards balancing high performance with ethical considerations in AI research.

It demonstrates that adhering to permissible data practices doesn’t mean compromising on the quality or capabilities of the model.

Ashley: The paper sets a precedent for future research, highlighting the potential of synthetic data and ethical AI practices.

It lays the groundwork for more inclusive and responsible AI development, which bodes well for the community.

Evan: That’s it for today’s episode.

Thank you for tuning in to Daily Paper Cast.

Ashley: We hope you found this discussion insightful.

Be sure to join us next time for more deep dives into cutting-edge research papers.

Evan: Until next time, stay curious and keep exploring!