Execution attempt

Sign in with GitHub
← Experiment E7 · Does the embedding initialisation of an APT4 transplant of Qwen2.5-1.5B damage formal domains more than prose before training, does the ranking of FOCUS, FVT and random initialisation depend on the domain, and do the differences persist under continued pretraining that updates only the embeddings?

Execution attempt · A65 · planned

Scores the base model and the three transplants in bits per byte on ten domains, loading each model directory's tokenizer with AutoTokenizer.

Succeeded · Registered by @stw2 via agent. Reported and received times are kept apart; no computation or result is verified.

Pinned source and configuration

https://github.com/stw2/tokenizer-science-tax @ e54c7e02bb1ea89dbaff6911c87331d705602b1e

Reference checked 2026-09-14 13:12 UTC. Later commits, branches or plan changes do not retarget this attempt.

Command
python scripts/03_score_bpb.py --model base=Qwen/Qwen2.5-1.5B focus=models/focus fvt=models/fvt random=models/random
Working directory
experiments/E07-embedding-init
Configuration paths
None
Parameters
max-docs 200; seq 2048 tokens; add_special_tokens false; bf16; row-wise scoring; tokenizers loaded with AutoTokenizer from each model directory; perdoc/ digest is the directory digest of its 40 files
Environment
3.12.13; PyTorch MPS, bf16, row-wise scoring; Apple silicon, 128 GB unified memory
Output directory
Not recorded

Inputs

Materials the registrant named when registering this attempt, by content identity; obtainability is derived from their location reports. Nothing is fetched or verified.

  • Qwen2.5-1.5B · base model · M124 Qwen2.5-1.5B · checkpoint · Download
  • models/focus · model · M358 models/focus · raw output · Ask the reporter
  • models/fvt · model · M357 models/fvt · raw output · Ask the reporter
  • models/random · model · M356 models/random · raw output · Ask the reporter
  • pl: pl_eval.jsonl · evaluation data · M129 pl_eval.jsonl · raw output · Download
  • en: en_eval.jsonl · evaluation data · M130 en_eval.jsonl · raw output · Ask the reporter
  • pl_informal: pl-informal.jsonl · evaluation data · M33 pl-informal.jsonl · raw output · Ask the reporter
  • pl_wiki_sci: pl-wiki-science.jsonl · evaluation data · M32 pl-wiki-science.jsonl · raw output · Download
  • pl_pes: pl-science-pes.jsonl · evaluation data · M31 pl-science-pes.jsonl · raw output · Ask the reporter
  • sci_arxiv: en-arxiv-abstracts.jsonl · evaluation data · M27 en-arxiv-abstracts.jsonl · raw output · Download
  • sci_latex: en-latex-methods.jsonl · evaluation data · M28 en-latex-methods.jsonl · raw output · Ask the reporter
  • sci_python: en-python-code.jsonl · evaluation data · M29 en-python-code.jsonl · raw output · Ask the reporter
  • sci_gsm8k: en-gsm8k.jsonl · evaluation data · M30 en-gsm8k.jsonl · raw output · Download
  • math_clean: math_clean.jsonl · evaluation data · M128 math_clean evaluation statements · dataset · Download

Delivered events

  1. Registered

    #1

    Scores the base model and the three transplants in bits per byte on ten domains, loading each model directory's tokenizer with AutoTokenizer.

    reported · received · @stw2 via agent · posted to the Thread

  2. Started

    #2

    Started.

    reported · received · @stw2 via agent · attempt only

  3. Succeeded

    #3

    Completed with exit code 0.

    Exit code 0.

    • bpb_e7.jsonl · https://github.com/stw2/tokenizer-science-tax/blob/884186c96aedece6ef61da4e12e239ef5df929a5/experiments/E07-embedding-init/results/bpb_e7.jsonl · public · sha256 6aa9c5becbf0… · 4777 bytes · M363
    • perdoc/ · https://github.com/stw2/tokenizer-science-tax/tree/884186c96aedece6ef61da4e12e239ef5df929a5/experiments/E07-embedding-init/results/perdoc · public · sha256 7b9afbdd8df5… · 148080 bytes · M364

    reported · received · @stw2 via agent · posted to the Thread

Report an event

The registrant’s capture tool normally delivers start and outcome events. Reporting here is the same author report with the browser as the reported time; it does not observe the process.

This attempt has a delivered outcome. A new execution is a new attempt.