02 — Models and tokenizer

From Saga 3B to the 15.45-billion model

The 15.45-billion-parameter model in training in the Fast Lane allocation and, below it, its predecessors: Saga 3B trained on JUPITER, the selected tokenizer and the August baseline.
Report timestamp
Model in training · Fast LaneIn progress

Saga 15.45B

Built on 29 September directly from Saga 3B: 48 layers and doubled width. Since 8 October it has been training in the Fast Lane allocation on 32 nodes (128 GH200 GPUs), 1,048,576 tokens per update. Two records of that training follow: the stages computed the earlier way, and the continuation after the fix of 12 November.

48layers
5,120layer width
40attention heads
2,048context tokens
65,536vocabulary entries

Stages T9–T24 · the earlier way of computing

8 Oct–10 Nov 2026

Loss after each stage · lower is better · a stage is 20.41 billion tokens

main line T9–T20 stage not adopted (T21, T24)
Polish validation set

The chart loads from the recorded measurements.

160 Polish windows
160 English windows

T20 · ended 02/11/2026 · Polish validation set 2.4273 · 160 Polish windows 2.3797 · 160 English windows 1.7102.

T9–T20 lowered the loss on the Polish validation set from 2.4872 to 2.4273. T21 came out worse than T20, T22 and T23 diverged and have no end-of-stage measurement, and T24 with a new mixture did not beat T20 either — so further work starts from the end of T20.

Show the stage figures
StageEndedNew Polish %Polish replay %English %Polish validation160 PL windows160 EN windowsNotes
The table needs JavaScript; the same figures are in dane/saga15.json.

Continuation after the fix · change in loss against the end of T20

12–14 Nov 2026

Windows the training never saw · 128 per source · below zero is better than the end of T20

Translations −0.258 DynaWord −0.135 FinePDFs −0.055 English −0.034 Earlier Polish text +0.010

The chart loads from the recorded measurements.

After 18,000 updates: translations −0.258 · DynaWord −0.135 · FinePDFs −0.055 · English −0.034 · earlier Polish text +0.010.

On 12 November a measurement showed that the gradient of the model's bottom blocks, computed in reduced precision, was rounding error. Since that day the eight bottom blocks have been frozen and computed in full precision, and the model went through the T24 mixture again, starting from the end of T20. The measurements in this chart are computed with the bottom blocks in full precision, so they are not directly comparable with the stage figures above. Loss measures text prediction, not conversation quality.

Show the measurement figures
UpdatesTranslationsDynaWordFinePDFsEnglishEarlier PolishPolish validation160 PL windows160 EN windowsMeasurement job
The table needs JavaScript; the same figures are in dane/saga15.json.
Earlier models and the tokenizer
Phase B · tokenizer selectionComplete

Tokenizer selected.
64k for Saga.

Bits per character · lower = better
TokenizerEncycl.WebProseMean
32k1.28041.11171.99231.4615
48k1.27711.09931.91851.4316
64k · selected1.26281.09251.93431.4299

64k selected for Saga 3B, with a mean of 1.4299 bits per character. Bits per character allow comparison across different vocabularies. This is a tokenizer result, not an assessment of the finished model.

Domain evaluation · indicative results

An exam built from texts outside training

The earlier evaluation sets shared passages with the training data, so we built a new exam solely from texts written after the data cut-off date. The 33M results remain part of the historical record.

The results are indicative and have not been independently confirmed. We do not make claims about the target model's capabilities.

Historical baselineAugust 2026 · local hardware

Canto 33M

A reference for the training process.

The recorded learning curve, evaluation and generated samples belong to the small local model. They do not describe the target Saga 3B model.

View the original curve — Polish / PL ↗
2.9867minimum validation loss
11,000completed training steps
Base modelUnconfirmed · phase D

Saga 3B

The full Saga 3B training run is complete according to the report.

The learning curve loads from the report.

195.8 billiontokens of exposure · plan completed
68 hplanned time on 64 GPUs

186,688 planned steps. This is the training budget, not the number of unique corpus tokens.

Editorial snapshot: 7 September 2026 · 19:24:40 Europe/Warsaw. Reading the current published report requires JavaScript.

Task history readout: 7 September 2026, 19:24:40 (Warsaw). Measurements and the plan retain their FAKTY source date: 7 September 2026, 17:46:39 (Warsaw).