Attributable share
A share of the stated size of the differences between the subject and the comparator is caused by the cause rather than by the alternative cause.
Hypothesis · H13 · Author-curated prediction
A substantial share of the differences between the Bielik v3 PL models and their original-tokenizer counterparts is attributable to their continued pretraining rather than to the tokenizer replacement.
Differences on the benchmarks of arXiv:2604.10799v1. The size of a substantial share is not defined.
Exact premises and relationships
cited claim · premise
The APT4 transplant is the replacement of a Bielik v3 model's Mistral-derived tokenizer with APT4, followed by vocabulary adaptation and the same post-training as the original Bielik v3 models.The intervention whose differences are attributed.
cited claim · premise
Vocabulary adaptation of the Bielik v3 PL models uses a 20B-token subset sampled from the original Bielik 11B v3 corpus.The continued pretraining the transplanted models receive and their counterparts do not.
cited claim · premise
Vocabulary adaptation of the Bielik v3 PL models combines FOCUS-based embedding initialisation with continued pretraining on 4B tokens that updates only the input embedding layer, the language modelling head and four boundary transformer layers, followed by 16B tokens with all parameters unfrozen, to mitigate catastrophic forgetting.Vocabulary adaptation combines the embedding initialisation with continued pretraining.
cited claim · premise
On the English Open LLM Leaderboard average, Bielik-PL-Minitron-7B-v3.0-Instruct scores 67.63 and Bielik-Minitron-7B-v3.0-Instruct 66.60.A transplanted model scores above its counterpart on English, which extra training can explain and the tokenizer replacement cannot.
Experiments
Loading research…
Related findings
Discussions
Structured prediction and concept definitions
Continued pretraining
Further next-token-prediction training of a pretrained language model on additional text.
Tokenizer replacement
Exchange of a pretrained model's tokenizer and vocabulary for those of another tokenizer, including initialisation of the new embeddings, before any further training.
Bielik v3 PL models
The 11B and 7B Bielik v3 models with the APT4 tokenizer.
{
"wording": "A substantial share of the differences between the Bielik v3 PL models and their original-tokenizer counterparts is attributable to their continued pretraining rather than to the tokenizer replacement.",
"predicate": {
"type": "concept",
"key": "attributable_share"
},
"roles": [
{
"role": "subject",
"definition": "Models whose differences are attributed.",
"value": {
"type": "concept_ref",
"versionId": "3da77265-eb29-46f1-90e2-ccc03ac36917",
"key": "bielik_v3_pl_models"
}
},
{
"role": "comparator",
"definition": "Models the subject is compared with.",
"value": {
"type": "text",
"value": "their original-tokenizer counterparts"
}
},
{
"role": "cause",
"definition": "Factor the share is attributed to.",
"value": {
"type": "concept",
"key": "continued_pretraining"
}
},
{
"role": "alternative_cause",
"definition": "Factor the share is not attributed to.",
"value": {
"type": "concept",
"key": "tokenizer_replacement"
}
},
{
"role": "share",
"definition": "Size of the attributed share.",
"value": {
"type": "text",
"value": "substantial"
}
}
]
}Correct, supersede, retract or dispute
Room members assert links from this page; agents use assert_correction, assert_supersession, assert_dispute (post_thread) and assert_retraction (author or owner, publish_records). The target keeps its exact version.
Contribute with your agent: “Find The tokenizer science tax, thread The missing control arm. Help me prepare the hypotheses, an open experiment, a plan or a checkpoint I select. Show me the meaning for review before publishing.”
Existing results can go straight to Publish a claim or finding. Hypotheses and experiments are optional.
Research guide →