Domyn reaches trillion-parameter scale on Europe’s public supercomputer for upcoming frontier AI model EUROPA

Europe still needs more ambition, more data centres, more capital and more time to build its AI capacity. But alongside that investment, there is an opportunity to make better use of the infrastructure already in place. Domyn’s work adds concrete evidence of what Europe can achieve with compute that is already operational and publicly funded.

As lead of the EUROPA consortium, alongside Fraunhofer IAIS and IIS, Domyn has demonstrated training of the Europa model at approximately 952 billion parameters, close to one trillion. Parameters are the values a model learns during training to recognise patterns and generate responses. The more parameters, the more capable the model, and today's frontier sits at one trillion and beyond.

Instead of relying on a newly-built cluster, the team used JUPITER, Europe’s publicly funded supercomputer, already operating in Jülich, Germany. Domyn’s engineers rebuilt the training stack around PyTorch and TorchTitan, using 384 NVIDIA GH200 GPUs for each copy of the model being trained in parallel.

Along the way, the team increased training throughput 14-fold, from 90 to approximately 1,300 tokens per second per GPU, making it possible to pre-train a trillion-parameter model in about four months. The gains show that Europe’s AI capacity depends not only on building more infrastructure, but also on unlocking the potential of the supercomputers it has already funded. This establishes a working configuration ahead of the full model-training run.

Domyn has published the configuration, engineering methods and measured results in its technical blog. The team has also contributed fixes to PyTorch and TorchTitan so other developers can benefit from the work. By sharing these methods, Domyn is helping more teams build on Europe’s investment in public supercomputing, and develop AI of their own.

The details are available in Domyn's technical blog.