On document-addressed cloze retrieval over Polish scientific documents, Bielik-11B-v3.0-Instruct's accuracy on prompts within its token limit is at least 0.70 at all 3 measured lengths from 18,839 to 69,328 characters, 0.857 at each.

Relation: Accuracy across lengths

metric

Within-window accuracyQuantity measured.

subject

Bielik-11B-v3.0-InstructModel measured.

task

Document-addressed cloze retrievalTask measured.

language

PolishLanguage of the documents.

threshold

0.70 accuracyAccuracy threshold compared with.

lengths below

0 lengthsMeasured lengths with accuracy below the threshold.

lengths measured

3 lengthsMeasured lengths with prompts within the token limit.

shortest length

18839 charactersShortest measured length.

longest length

69328 charactersLongest measured length within the token limit.

lowest accuracy

0.857 accuracyLowest accuracy across the lengths.

highest accuracy

0.857 accuracyHighest accuracy across the lengths.

Method and evaluation protocol

Graded generations of prompts within the model's token limit, per length.

Dataset

Polish Wikipedia science articles and Polish State Specialization Examination questions from frozen corpora, interleaved and packed to character budgets.Version: unspecified · Access: restricted

Reported results

Accuracy within the token limit by length: 18,839: 0.857 over 21 prompts; 45,214: 0.857 over 21 prompts; 69,328: 0.857 over 21 prompts. Bielik-PL-11B-v3.0-Instruct on the same task and language: 18,839: 1.000; 45,214: 0.952; 69,328: 0.905; 106,235: 0.810.

Uncertainty and replication

Point values from one greedy run; 21 prompts per length from the cloze items kept.

Evidence references

analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/c60abf85776b3d991bdbba0e47569c6d944056e7/experiments/E04-effective-context/results/analysis.jsongraded_orig.jsonl · restrictedNot redistributed: ask the Room owner, or rebuild with scripts/03_grade.py at the pinned revisionsmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/c60abf85776b3d991bdbba0e47569c6d944056e7/experiments/E04-effective-context/results/metrics.json

Limitations

7 of 10 cloze items remain after the no-context drop rule.