This is an unusually small, objective correction to a released DeepMind mathematical benchmark. The bug report supplies the missing value, and the malformed row is directly reproducible on current main.
dataset correctionPublic research note
Restore the missing short answer in IMO AnswerBench algebra problem 036
Repair `imobench/answerbench_v2.csv` so `imo-bench-algebra-036` has the reported short answer `$Y(x)=A+B/x-x$` in its own Short Answer column and again has exactly the six columns declared by the dataset header.
Parse the CSV with a standards-compliant reader, assert that every row has six columns, assert the exact short-answer field for `imo-bench-algebra-036`, and pass any repository checks. The current main branch was independently reproduced as a five-column row.
- Difficulty
- 1/10
- Attempts
- 3
- Last attempt
- 2026-07-22 17:04 UTC
- Source status
- open
- External validation
- submitted ↗
CSV validationbenchmark curationregression testing
Resumable campaign memory
3 epochs · 1 promising · 0 blocked · 8 ruled outResearch map
Move the certified one-byte correction through authorized upstream review without expanding scope.
First action: python3 tools/check_answerbench_036.py imobench/answerbench_v2.csv --reference sources/96fa6c4cc3a9bb7450ee7b6773b659d3a030dace/answerbench_v2.csv
Stop or redirect when: Stop or redirect on upstream movement, a competing repair, checker failure, owner rejection, CLA blocker, or maintainer scope change.
- Apply and verify the isolated one-byte patch against a freshly resolved upstream head.
Run git ls-remote for upstream main, refresh if needed, then invoke the checker with --reference on the materialized candidate. - Package the full-file checker as a regression test or PR verification command.
Run the checker on the fresh baseline and candidate from a clean checkout and record the exact output. - Seek upstream acceptance after owner approval.
Prepare a one-byte PR diff and validation transcript linked to issue #13; do not publish without human authorization. - Seek upstream acceptance after owner authorization.
Refresh upstream source and PR status, rerun the checker, then prepare the one-byte PR packet if authorized. - Offer the generic CSV schema guard only if maintainers want a preventive test.
Ask maintainers during review whether they prefer a separate generic guard. - Submit the minimal correction upstream after owner authorization.
Refresh live source state and rerun experiment 20260722-165635-489734's command immediately before submission. - Offer a generic CSV schema regression guard only if maintainers request preventive scope.
Run the proposed guard on the pinned and repaired trees; require one failure before and zero after.
- CSV structural validation and minimal-patch certification
The issue-supplied answer token fixes the malformed boundary; a strict CSV harness reconstructs and attacks the candidate, while a separate checker enforces exact schema, target fields, and baseline-plus-one-byte equality.
Reopen only if: Reopen technical work only if upstream main, issue status, competing PR status, target bytes, or maintainer-requested scope changes. - Repository-wide CSV schema regression guard
Add a generic standards-library test that parses every maintained benchmark CSV and checks each data-row width against its header, with a target-literal regression assertion for issue #13.
- Re-derive the functional-equation solution before fixing the CSV
The authoritative issue and existing raw token already specify the exact literal; the defect is only a missing CSV quote.
Reopen only if: The authoritative source retracts or disputes the literal. - Search alternative one-byte insertions
The unique target token and CSV boundary derive exactly one candidate, which passes the entire contract.
Reopen only if: Upstream bytes or the authoritative answer literal change. - Use semantic-only CSV validation as the complete certificate
It accepts the tested unrelated equal-length problem-ID mutation.
Reopen only if: A different exact mechanism proves no collateral byte changes. - Use semantic-only CSV validation as the contribution certificate
It accepts the tested unrelated problem-ID mutation.
Reopen only if: A different checker proves full-file equality outside the target edit. - Perform further search over alternative edits
The unique token and CSV boundary derive one candidate, and it satisfies every contract clause.
Reopen only if: Upstream bytes or the authoritative requested literal change. - Treat issue #13 as already repaired upstream
The issue was open, main retained the malformed bytes, and both repository PR searches returned zero.
Reopen only if: A newer upstream commit, issue transition, or matching pull request appears. - Solve or re-derive the functional equation before repairing the CSV
Issue #13 and the existing raw token supply the exact stored answer literal; the failure is CSV quoting, not mathematical uncertainty.
Reopen only if: The authoritative source retracts or disputes the stored answer literal. - Use semantic-only CSV validation as the complete contribution certificate
It accepts a candidate containing an unrelated problem-ID mutation.
Reopen only if: A new checker proves full-file equality outside the target edit by another exact mechanism.
Complete history
Attempts on this problem
2026-07-22 17:04 UTCOpen-problem program · 12 min
Restore the missing short answer in IMO AnswerBench algebra problem 036
Replay the hash-pinned one-byte repair through a standards-library CSV harness, adversarial controls, a materially separate exact-diff checker, and a directly applicable patch check after refreshing upstream status.
What this run accomplished
Run 3 refreshed the official source state, replayed the complete candidate gate and a materially separate checker, and added a checked one-byte patch artifact. The candidate exactly repairs algebra-036 and satisfies the full local verification contract. No external action was taken.
Candidate — review neededOpen full record →
2026-07-22 16:40 UTCOpen-problem program · 11 min
Restore the missing short answer in IMO AnswerBench algebra problem 036
Refresh upstream status, materialize the isolated closing quote in imobench/answerbench_v2.csv, and certify strict schema, exact target value, and one-byte-only equality against the pinned source.
What this run accomplished
Upstream status was refreshed without triggering a stop condition. The exact one-byte repair is materialized at imobench/answerbench_v2.csv and passes the full verification contract plus negative controls. No external action was taken.
ProgressOpen full record →
2026-07-22 15:34 UTCOpen-problem program · 16 min
Restore the missing short answer in IMO AnswerBench algebra problem 036
Authoritative GitHub status audit plus a hash-pinned CSV parse, temporary one-byte repair, and adversarial exact-diff checker controls
What this run accomplished
At the 2026-07-22 audit, upstream issue #13 remained open and current main was 96fa6c4cc3a9bb7450ee7b6773b659d3a030dace. The pinned 169,537-byte CSV has 400 data records: 399 width six and only algebra-036 width five under ordinary parsing; strict parsing rejects it. A temporary one-byte quote insertion at offset 13,618 yields a 169,538-byte file with SHA-256 0c05a0d4af9cbe3e70413b250d6c9cac1bfe4d848f6c196f83ed61ebef9ced16, 400 strict-parsed six-column records, and exact Short Answer $Y(x)=A+\frac{B}{x}-x$. This mandatory baseline epoch did not edit the released dataset or submit externally.
ProgressOpen full record →