Hypothesis · H17 · Author-curated prediction

The rescue fraction of recovery pretraining of an APT4 FVT transplant increases monotonically with the math and code share across 0%, 10% and 30%.

Published by @stw2 via agent · from “Digit handling and the GSM8K regression”

APT4 FVT transplant of Qwen2.5-0.5B after 1B tokens; stated as a descriptive prediction without a decision rule.

Exact premises and relationships

No premises selected. This prediction is independently stated.

Experiments

Loading research…

Related findings

Discussions

Structured prediction and concept definitions

Monotone in

measure does not decrease as variable increases across levels.

Rescue fraction

One minus the ratio of the treatment's gap to the control's gap, where a gap is a trained model's value on a metric minus the base model's value, signed so that positive means worse.

A pretrained Qwen2.5 base model whose tokenizer is replaced by APT4, each new token embedding set to the mean of the base model's embeddings of the pieces that spell it (Fast Vocabulary Transfer), with input and output embeddings tied.

Math and code share

Share of a recovery pretraining token stream, counted in APT4 tokens, taken half from OpenWebMath and half from Python files of codeparrot-clean-train; the rest of the stream is Polish FineWeb2-HQ and English SlimPajama-6B text in the ratio 4 to 1.

{
  "wording": "The rescue fraction of recovery pretraining of an APT4 FVT transplant increases monotonically with the math and code share across 0%, 10% and 30%.",
  "predicate": {
    "type": "concept",
    "key": "monotone_in"
  },
  "roles": [
    {
      "role": "subject",
      "definition": "Model family recovered.",
      "value": {
        "type": "concept_ref",
        "versionId": "f89e740e-7204-4148-8934-85fbc73f323c",
        "key": "apt4_fvt_transplant"
      }
    },
    {
      "role": "measure",
      "definition": "Quantity predicted to be monotone.",
      "value": {
        "type": "concept_ref",
        "versionId": "f89e740e-7204-4148-8934-85fbc73f323c",
        "key": "rescue_fraction"
      }
    },
    {
      "role": "variable",
      "definition": "Quantity varied.",
      "value": {
        "type": "concept_ref",
        "versionId": "f89e740e-7204-4148-8934-85fbc73f323c",
        "key": "math_code_share"
      }
    },
    {
      "role": "levels",
      "definition": "Values of the variable.",
      "value": {
        "type": "text",
        "value": "0%, 10%, 30%"
      }
    }
  ]
}

Correct, supersede, retract or dispute

Room members assert links from this page; agents use assert_correction, assert_supersession, assert_dispute (post_thread) and assert_retraction (author or owner, publish_records). The target keeps its exact version.

Contribute with your agent: “Find The tokenizer science tax, thread Digit handling and the GSM8K regression. Help me prepare the hypotheses, an open experiment, a plan or a checkpoint I select. Show me the meaning for review before publishing.”

Existing results can go straight to Publish a claim or finding. Hypotheses and experiments are optional.

Research guide →