How do you make a 4B-parameter model good at five different things without diluting any of them? Our answer: don’t make one model learn it all at once. Train five specialists first, then distill everything they know into a single student. That’s exactly how we’re building Domyn Edge.
On October 20th, we released the first bundle of Domyn Edge, a 4B-parameter language model trained from scratch on 13 trillion tokens, end-to-end on European public infrastructure. The model was trained on Leonardo, the EuroHPC supercomputer hosted by CINECA, and it is built for agentic work at long context, trained up to 1M tokens and recommended at a 256k working context.
Domyn Edge is released in a total of three bundles and it is fully open. The first bundle covered the pre-trained base model, a checkpoint every trillion tokens of pre-training plus the endpoint of every stage, as well as the stage 1 pre-training dataset, the SFT checkpoint (known as the student), and Domyn Forge (the training code). Everything is released under the Apache 2.0 license, so the model can be inspected, audited, reproduced and trained further.
Today, with this second bundle, we ship the first step of the post-training process: the five domain teachers behind our distillation protocol, the data they were trained on, the student's own supervised mix, the pre-training datasets for the remaining five stages, and Domyn Eval, the open-source evaluation harness we measure everything with.
Why are specialist teachers useful?
Domyn Edge is post-trained with Multi-Teacher On-Policy Distillation (MOPD) [1]. We train a separate specialist for each capability domain, then distill all of them into a single student that keeps what each specialist learned. The teachers come first, and the distilled model follows in Bundle 3.
All five teachers start from the same checkpoint, the released domyn/Domyn-Edge-SFT. Each one is given a supervised pass on its domain, then trained with GRPO against verifiable rewards on that domain, using NeMo RL. Distillation then runs on the student's own rollouts, with every teacher scoring the same trajectories.
We are releasing the teachers as intermediate artifacts that make the final model reproducible, and several of them are useful on their own for a reader who needs one capability rather than a generalist.
The five teachers
Math. Competition and olympiad mathematics, evaluated on AIME 2025, AIME 2026, HMMT and MATH. The supervised pass is followed by GRPO against an answer verifier, with a penalty for correct answers returned in the wrong format.
STEM. Multiple-choice science and quantitative reasoning, evaluated on GPQA Diamond, MMLU-Pro and MGSM.
Code. Program synthesis and code reasoning, evaluated on LiveCodeBench, LiveBench Coding, BigCodeBench, HumanEval, MBPP and CruxEval. Training runs at a 64k context, which is the limit at which generations stay coherent.
Instruction following. Optimized on instruction following and tool calling, evaluated on IFEval, IFBench, IFStruct, IFExtract and single-turn only.
Agentic. Multi-turn agentic work in terminal and software-engineering environments, evaluated on tau2-bench, tau3-bench, Terminal-Bench, the multi-turn split of BFCL v4 and SWE-bench Verified.
Each teacher card reports the full training recipe: framework and version, node count and the split between generation and training, context and sequence length, learning rate and schedule, batch size and rollouts per prompt, KL coefficient, clipping, number of steps and wall time, together with the reward specification and the verifier used.
Results
Every number below is pass@1, measured with domyn-eval under one chat template and one set of sampling settings. AIME results are averaged over ten runs and reported with their standard deviation. Qwen 3.5 4B is the reference model, as it was on the student card.
Each teacher on its own domain
One table per teacher, each one on the suite that teacher was trained for. The student is the checkpoint every teacher started from, so the middle column against the first shows what the specialization added, and the middle column against the last shows where we stand against the reference model.
Math Teacher
STEM Teacher
Code Teacher
Instruction Following Teacher
Agentic Teacher
The data
Pre-training, stages 2 to 6. Bundle 1 released the stage 1 mixture, 8.4 trillion unique tokens, as domyn/Domyn-Edge-Pretraining-Mix-8.4T. This bundle releases the five mixtures that follow it, in one repository with one config per stage. Each config pairs with a checkpoint branch of domyn/Domyn-Edge-Base-checkpoints, so a reader can take the checkpoint at the start of a stage, the data that stage consumed, and the recipe from Domyn Forge, and repeat the step.
Stage 2 raises the weight of reasoning, mathematics and code at a 16k context. Stage 3 is the first context extension, to 64k. Stage 4 adds agentic tool-calling and coding trajectories at 32k. Stage 5 is the second extension, reaching a 1M-token training context. Stage 6 decays the learning rate in two steps, at 64k and then at 512k.
The student SFT mix. The supervised mixture behind domyn/Domyn-Edge-SFT is released as domyn/Domyn-Edge-SFT-Mix, with one config per supervised stage: 64k, 128k-cascade and 512k-cascade, holding 228.1B, 29.8B and 57.3B tokens. The dataset card lists every source with its link and license, the sample and token counts per source, and the filtering and decontamination applied. We generated two sets, marked as ours in the table above: a long-context question-answering set and a set of tool-calling trajectories collected against a live agent platform. One split is withheld, the branding identity data, so the per-stage token counts in the release are lower than those reported on the model card.
The teacher mixes. Each teacher has one dataset repository with an SFT config, plus an RL config wherever reinforcement learning was applied. The sft config is made of conversations, while the rl config is made of prompts with their verifiable answers or environment specifications. Each card describes the composition of the mixture and attributes every third-party source it draws on, with that source's own license.
The Apache 2.0 license covers what we produced. Upstream corpora maintain their original license terms, each of them stated in each dataset card.
Domyn Eval: The evaluation harness
Every number in this post was produced with domyn-eval, which we are releasing with this bundle under Apache 2.0. It orchestrates external evaluation suites behind one interface and runs them on SLURM clusters, serving the model under test through Domyn Swarm. The harness ships with the configurations the tables above were produced with, so the results can be reproduced.
Reproducing a published benchmark score is harder than it looks. A score is comparable with another only when they have the same sampling parameters, chat templates, the prompt sets, judge models and versions of the upstream suite. In domyn-eval, an evaluation is a configuration file, so a result can be rerun as it was produced. From that file the harness runs the suites, serves the model and collects the results.
- Many suites, one interface. Suites such as Inspect AI and lm-evaluation-harness are integrated as they are published, each pinned to a version and isolated from the others, and a new suite is added without disturbing them. Results from all of them come back in the same format, so a model is measured across suites in a single run.
- Serving and grading on the cluster. Domyn Swarm serves the model under test through vLLM, with a second endpoint for the judge model. Benchmarks that execute generated code are graded in a sandbox, isolated from the host and from each other.
- A check before any compute is spent. The configuration, the environment and the endpoint are validated before a run starts.
- One report, from a versioned profile. Every run keeps its configuration with its logs and results, and reporting collects them into one table per model, or a leaderboard across models.
- Statistics alongside the scores. For each benchmark, each report also counts how many generations were truncated, came back empty, could not be scored, or degenerated into a repeating loop.
Coming up next: Bundle 3
Bundle 3 will bring the final Domyn Edge model — our recommended choice for general use — built by distilling the five teachers into the student through MOPD. The student generates the rollouts, each teacher scores them, and the student is trained on that signal. The final result is a single generalist model that inherits what each specialist learned. This release will come with a technical report, covering data lineage, the mixtures and sampling weights of every stage, the architecture and tokenizer ablations, and what we learned along the way.
Until then, the training recipes are available in the teacher cards, and the composition of each mixture in the dataset cards. We welcome corrections and questions on any of the artifacts released here.
Resources
Bundle 2
- Teacher models: Math, STEM, Code, Instruction following, Agentic
- Teacher data mixes: Math, STEM, Code, Instruction following, Agentic
- Student SFT mix: domyn/Domyn-Edge-SFT-Mix
- Pre-training datasets by stage: domyn/Domyn-Edge-Pretraining-Mix-Stages
- Evaluation harness: domyn-eval on GitHub
Bundle 1
- Base model: domyn/Domyn-Edge-Base
- Pre-training checkpoints: domyn/Domyn-Edge-Base-checkpoints
- Stage 1 pre-training dataset: domyn/Domyn-Edge-Pretraining-Mix-8.4T
- Student model: domyn/Domyn-Edge-SFT
- Training code: Domyn Forge on GitHub
- HPC tooling: Domyn Swarm and SDAG on GitHub
- Collection: Domyn Edge on Hugging Face
Reference
[1] Ma, W., et al. (2026). MOPD: Multi-Teacher On-Policy Distillation for Capability Integration. arXiv:2606.30406.
Frequently Asked Questions
