Protocol peer review
Every round, every comment, every response
Before registration the protocol was reviewed by three language model reviewers working independently from the same brief; the study moved on only when all three accepted the same version. This is not human peer review: the human decision record is the response to each comment.
Verdicts
Comments
299
comments with a response
85
major comments
203
minor comments
278
led to a revision
How the review worked
GPT-6 Astra
OpenAI; Codex CLI, reasoning effort xhigh, read-only sandbox.
Fable 5.1
Anthropic; independent agent session, read-only instructions.
Grok 4.7
xAI; Grok CLI, reasoning effort xhigh, read-only copy of the repository.
- The version under review is frozen and archived with the round (see protocol versions).
- Each reviewer receives the same brief and ends with one verdict: accept, minor revision or major revision.
- Every comment is entered in comments.csv with the response and the action taken.
- The revised protocol gets a new version number and the next round starts; each reviewer judges whether its own earlier comments were resolved.
Before version 0.1.0, two reviewers were consulted on scope and design (design consultation). The search strategy had its own PRESS 2015 review. Reviews of the screening and extraction manuals: extraction (2026-10-14), screening (2026-10-13).