Building Saga.
Saga — a Polish language model trained on the JUPITER supercomputer, with its own tokenizer and its own corpus. This page is the project record: every run and every measurement, including the failed ones.
- parameters
- 15.45B
- GH200 chips in one training run
- 128
- tokens of exposure in Fast Lane
- 263.8B
- GPU-hours of 50,000
- 48,198
Where we are
- AComplete
Reproduce and scale
Reproduction and scaling: complete.
- BComplete
Choose the tokenizer
The report selects the 64k tokenizer for Saga.
- CComplete
Corpus composition
Corpus-composition comparison: complete.
- DComplete
Train Saga 3B
The full Saga 3B training run is complete according to the report.
- EComplete
Continued pre-training and new corpus
Work in the Development project is complete: continued pre-training of the Saga 3B base, building the 15.45-billion-parameter model from it and preparing 15 data stages for Fast Lane.
- FIn progress
Growing Saga
In progress since 8 October: training the 15.45-billion-parameter model in the Fast Lane allocation. Twelve stages (T9–T20, by 2 November) lowered the loss on the unchanged Polish validation set from 2.4872 to 2.4273. After the fix to the gradient computation on 12 November the model went through 18,000 updates on the mixture with translations: against the end of T20, the loss on unseen translations fell by 0.26 and on DynaWord texts by 0.13. A second pass over the same data gained less than the condition written down before the start required and was stopped after its first segment. A chat model fine-tuned from these weights answers in the chat window as a test version.
- GPlanned
Publication
The Saga 3B weights stay in a private repository. What remains to be published is the report on the method and the recipe.
Project pages
- 01MeasurementsScale you can verifyThe model in training, EuroHPC allocations and GPU-hours used, scaling on JUPITER Booster and the September measurements.
- 02ModelsFrom Saga 3B to the 15.45-billion modelSaga 15.45B with stage and source charts, Saga 3B, the tokenizer choice, the exam built from texts outside training and the Canto 33M baseline.
- 03DataSources and measured compositionThe mixture of every Fast Lane stage and how the Saga 3B corpus was prepared: sources, deduplication, composition ablations.
- 04RegistryEvery run. Including the failed ones.Completed, running and failed jobs from the published cluster report — the registry is in Polish.
- 05MethodOpen recipe, open measurements, open mistakesPrinciples of the open record, project documentation and methodological caveats.
Chat with
Saga.
15.45-billion-parameter model · test version · access with a code.