Data sources
The dataset behind the example
The example uses synthetic data. The data are modelled on Gemini's answers to public maths and general-knowledge questions. Each file names its source in a header.
Tutoring and routing data (synthetic, modelled on GSM8K and MMLU-Pro prompts answered by Gemini 3.1 Flash-Lite and Pro)
| Holder | GSM8K: OpenAI; MMLU-Pro: TIGER-Lab; the answers: generated by ZETai Risk Labs for the technical note's routing benchmark |
| Licence | MIT (GSM8K and MMLU-Pro) |
| Where to find it | https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro |
| Citation | Cobbe et al., "Training Verifiers to Solve Math Word Problems", 2021, arXiv:2110.14168; Wang et al., "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark", 2024, arXiv:2406.01574. |
| What is shipped | A derived JSONL with the prompt, both tiers' answers and verdicts and the subject area; no other field of the datasets. The tool calls, their outcomes, the injection attempts and the identities are ours. |
| Our changes | Synthetic data, modelled on the original tutoring and routing dataset, in which these prompts were answered by Gemini 3.1 Flash-Lite and Pro through calls to Google's API and graded against the reference answers. The data hold 5,810 prompts from GSM8K and MMLU-Pro over 15 subject areas, answered by Gemini 3.1 Flash-Lite, with the exact-match verdict against the reference answer as the error flag (9.9% wrong overall, 22% in health and 22% in law). The order in time is ours: the pool carries no timestamps, and the prompts are spread uniformly over twelve months. In this stream each correct arithmetic answer appears three times, as further students asking the same question: an upsampling that models a cheap model which is right more often in arithmetic. |
| Used by | Tutoring assistant, routed and gated (agent_stream.jsonl) |
As built here: 8,646 trajectories, 9,811 tool calls, 236 of them harmful or wrong, 147 injection attempts; built 2026-10-10. The prompts, answers and verdicts are synthetic, modelled on the original dataset of Gemini answers to GSM8K and MMLU-Pro prompts.
What is ours
ZETai created the tool calls, their outcomes and the injection attempts for this demo. These parts carry no licence.