Qwen2.5-0.5B has 0.0744 bits per byte on GSM8K problem statements and at least 0.4046 bits per byte on each of the other nine evaluation texts.

Relation: Lowest text value

model

Qwen2.5-0.5BModel scored.

metric

Bits per byteMetric.

text

GSM8K problemsText with the lowest value.

value

0.0744 bits per byteMetric on text.

other texts minimum

0.4046 bits per byteLowest value on the other nine texts.

Method and evaluation protocol

Bits per byte of Qwen2.5-0.5B on the first 200 documents of each of ten texts.

Dataset

The first 200 documents of ten evaluation texts: Polish and English web holdouts, English arXiv abstracts, LaTeX method sections, Python code and GSM8K problems, Polish Wikipedia science articles, PES questions and reviews, math_clean.Version: unspecified · Access: restricted

Reported results

GSM8K problems 0.0744; lowest other text 0.4046 (English Python code).

Uncertainty and replication

Point values without intervals.

Evidence references

bpb_mix00.jsonl · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/bpb_mix00.jsonlmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/metrics.json

Limitations

A low value is consistent with the model having seen the GSM8K training split, but no contamination test was run; the GSM8K text serves as a diagnostic, not an endpoint. Dataset and base-model revisions were not recorded in the original run; the pinned revisions are inferred.