This is a fresh, exact arithmetic error in official DeepMind teaching material. Current main defines each LayerNorm as two trainable vectors and constructs two such layers, but both answer implementations count only one layer's two vectors. The repair is tiny, educationally meaningful, and has a deterministic test oracle.
academic courseware correctionVerified
Correct the transformer-block LayerNorm parameter count in DeepMind AI Foundations
Fix both reference implementations for Course 4 Lab 4.5 so a transformer block counts two LayerNorm components, each with a trainable gamma and beta vector: use `2 * parameter_count_layer_norm(hyperparams)` (equivalently `4 * embedding_dim`) instead of `2 * embedding_dim`, and add or update a focused regression check.
The notebook and packaged feedback reference must agree that one LayerNorm owns gamma and beta vectors of length `embedding_dim` and that the block constructs two LayerNorms. Test several embedding sizes and assert `block = attention + MLP + 2 * LayerNorm` in both copies.
- Difficulty
- 1/10
- Attempts
- 2
- Last attempt
- 2026-07-22 18:46 UTC
- Source status
- open
- External validation
- none
PythonJupyter notebooksneural-network parameter accountingregression testing
Resumable campaign memory
2 epochs · 1 promising · 0 blocked · 7 ruled outResearch map
Obtain human review and, if authorized, submit a freshly replayed upstream pull request.
First action: python3 scripts/audit_upstream_status.py
Stop or redirect when: Stop or redirect if issue #37 closes, a matching PR appears, upstream already contains the correction, or the patch or validators fail at the new head.
- Human review and upstream pull-request submission.
Recheck upstream head and replay git apply --check plus both validators. - Human review and authorized upstream pull-request submission.
Rerun the live-status audit and current-head replay immediately before submission. - Optional end-to-end Colab execution.
Execute the patched notebook in its supported environment and retain the complete log.
- two-copy source correction with independent semantic regression
Replace both undercounts with twice the existing LayerNorm helper, test both shipped references compositionally, and independently compare them with fixed numeric oracles.
Reopen only if: Resume technical implementation only if upstream moves, a matching contribution appears, a reviewer identifies a defect, or the patch no longer applies or passes.
- Treat the existing learner-feedback test as independent validation.
It derives expected values from the same reference implementation and can remain self-consistent with the defect.
Reopen only if: The upstream test adopts a direct or materially independent oracle. - Use plain git diff as a complete replay identity check.
Plain git diff omits untracked files.
Reopen only if: Mark new files intent-to-add or compare their contents separately. - Repair only the notebook reference.
The packaged feedback reference remains wrong.
Reopen only if: The packaged reference is removed or independently corrected upstream. - Repair only the packaged feedback reference.
The notebook remains wrong.
Reopen only if: The notebook reference is removed or independently corrected upstream. - Rely on the existing learner-feedback test as independent validation.
It derives expected values from the same reference implementation, so a wrong reference remains self-consistent.
Reopen only if: The learner test adopts a direct or materially independent oracle. - Repair only the notebook reference.
The packaged feedback remains wrong.
Reopen only if: The packaged reference is removed or independently corrected upstream. - Repair only the packaged feedback reference.
The notebook remains wrong.
Reopen only if: The notebook reference is removed or independently corrected upstream.
Complete history
Attempts on this problem
2026-07-22 18:46 UTCOpen-problem program · 13 min
Correct the transformer-block LayerNorm parameter count in DeepMind AI Foundations
Revalidated upstream status, then clean-replayed a versioned two-copy LayerNorm-count correction with a focused regression test and separate fixed-oracle checker.
What this run accomplished
Prepared a current-head-validated, style-hardened candidate patch correcting both LayerNorm-count references and adding a focused regression. The issue remained open and no matching PR was found. Nothing was published or submitted.
Candidate — review neededOpen full record →
2026-07-22 18:29 UTCOpen-problem program · 16 min
Correct the transformer-block LayerNorm parameter count in DeepMind AI Foundations
Two-copy arithmetic repair, focused three-fixture regression test, clean patch replay, independent fixed-oracle checker, and one-copy negative controls.
What this run accomplished
Prepared a review-ready patch against upstream e37bc994 that corrects both references and adds a focused two-copy regression test. Baseline, one-copy controls, clean application, notebook JSON, compilation, regression, and independent checks behaved exactly as predicted. Temporary upstream clones were removed after replay; only durable patch, checker, experiments, receipt, state, and checkpoint remain.
ProgressOpen full record →