03 — Data and recipe

Sources and measured composition

The Saga 3B corpus combines five pools of Polish text after global deduplication; composition ablations compared how mixtures affect training. The 15.45-billion model trains in Fast Lane on consecutive mixtures of 20.41 billion tokens each: new Polish text from HPLT 3.0 checked for novelty against the whole training history and for rights (from mixture T11 also filtered for templated documents), a repeat of Polish text from earlier sets, and 15% English FineWeb-Edu.Corpus recipe ↗
Report timestamp
15.45-billion model · stage mixtures in Fast LaneT9–T24

Mixture composition in each stage

Each stage is 20.41 billion tokens made of three parts: new Polish text checked for novelty against the whole training history, a replay of earlier Polish sets, and 15% English FineWeb-Edu. The new Polish text came in turn from HPLT 3.0 (T9–T11), FinePDFs (T15), MADLAD-400 with FinePDFs (T16–T17) and Polish DynaWord with FinePDFs (T23); in T24 it is 86% translations of English FineWeb-Edu educational texts made with the Bielik-11B-v2.3-Instruct model, 11% Polish DynaWord and 2.5% FinePDFs.

new Polish text Polish replay English

The mixture composition loads from the recorded data.

The figure on the right is the share of new Polish text. Dimmed stages did not enter the main line: T21 and T24 were no better than T20, and T22 and T23 diverged.

Saga 3B corpus · base model
FineWeb-2 + HPLT 2.0Polish web
FinePDFstext from PDF files
Wikipedia and literatureWikipedia, Wikisource, Wolne Lektury
Step 1Deduplicationacross sources
Step 2 · phase CComposition ablationssix training runs
Saga 3B exposure
195.8 billion tokens

Token exposure over training steps, not the size of the unique corpus: training can process the same material more than once. The full recipe is described in the method.

Editorial snapshot: 7 September 2026 · 19:24:40 Europe/Warsaw. Reading the current published report requires JavaScript.

Task history readout: 7 September 2026, 19:24:40 (Warsaw). Measurements and the plan retain their FAKTY source date: 7 September 2026, 17:46:39 (Warsaw).