Peer review round 6: LivingGLP1 protocol 0.6.0 (reviewer fable-5.1)
What I verified on 2026-10-12 against the frozen 0.5.0 copy in round-5/reviewed-version-0.5.0/: only protocol.qmd and amendments.csv (A005) changed; concepts.yaml and the twelve other authoritative tables are byte identical, so the invariants I reported in round 5 (77 outcomes, no overlapping windows, one primary window for every outcome and population pair except O36A, O36B and O120, complete Turner and reference SD coverage, nrs_synthesis flags matching harms_nrs.csv, every node class resolving, 11 templates, 43 agents matching concepts 1.2.0) still hold, and comments.csv has the 19 round 5 rows that A005 states. R/search_strategy.R renders the four sources, and the PubMed query returns 28,673 records with the same six empty development codes as in rounds 3 to 5. I recomputed the new numbers: risks 0.50 and 0.75 with 5 percent lost from both arms give 1.93; risks 0.010 and 0.012 (hazard ratio 1.20) with 5 percent lost from the control arm only give 1.26, and the same loss from the treated arm only gives 1.14, so the statement that unequal loss has no general direction is correct. The constant hazard cumulative incidence formula matches numerical integration, and the subdistribution conversion is the Fine and Gray relation.
1. Round 5 comments
| Item | Status | Reason |
|---|---|---|
| F5-1 | Resolved (alternative) | No pooled absolute effect is derived from the recorded harms analysis; per trial proportions and within trial risk differences are displayed, and certainty uses the relative range with the scale stated. The reason (observation processes differ between trials, so no common period defines a recorded risk) is sound, and the rule is executable, which is what my comment required. |
| F5-2 | Resolved | 20 to 104 weeks restored in the frequentist analysis and the certainty section, and added to the sensitivity list. |
| F5-3 | Partly resolved | Cause specific is primary for time to event outcomes and subdistribution for binary harms; hazard type is in the network key and the connectivity rule. The response says "Agreed" but my clause that a trial reporting both contributes each to its own network was not carried; item 1 carries it. |
| F5-4 | Resolved (alternative) | Removing the exemption is sound once every input carries a hazard type; it leaves one ambiguity in "not applicable" (item 3). |
| F5-5, F5-6 | Resolved | O22C appears once; the version number reads 0.6.0. |
| F5-note | Adopted | Cumulative incidence enters as a contrast level normal likelihood, which multinma fits directly. |
2. New items in 0.6.0
No major items. Each item below completes a rule the revision introduced; none changes a model, an estimand or a primary estimate.
Minor
Two hazard type networks per binary harm, one conclusions rule (Event outcomes and harms, line 490; Competing risks, lines 514 to 518). A binary harm now has a subdistribution network (primary) and a cause specific network. Reported Cox hazard ratios for harms (competing deaths censored) are cause specific by the protocol's own definition and land in the secondary network, while counts reach the primary network only with status known for every randomized participant, so the primary network will often be the sparser one. "Conclusions about harms use this analysis when it meets the connectivity rule" does not say what happens when the primary hazard type network fails the rule and the other meets it, and nothing says whether a trial reporting both a hazard ratio and complete counts contributes to both. Replacement for the conclusions sentence: "Conclusions about a harm use the primary hazard type network when it meets the connectivity rule; otherwise the other hazard type network, labeled with its hazard type; otherwise the recorded harms analysis." Append to the competing risks paragraph: "A trial reporting inputs of both types contributes each to its own network."
Known status omits competing deaths (lines 480 to 483). Known status is "the event occurred by t or the participant was observed event free through t", so a participant who died before t without the event is, read literally, unknown, and every trial with such a death fails the every participant rule; the competing risks paragraph instead treats complete status counts as subdistribution inputs that count competing deaths as event free. Replacement, after "through t": "or died before t without the event (counted as event free, which is why complete status counts are subdistribution inputs)".
"Not applicable" is not anchored to the outcome (lines 513 to 514). Recorded per input, "not applicable (no competing event)" can be assigned to a trial in which nobody died, splitting one harm's counts into a not applicable network and a subdistribution network that never share a key. Replacement: "not applicable (outcomes with no competing event, such as O20); for every other outcome the hazard type follows the estimator even when no competing event occurred in the trial, since the two hazards then coincide".
The cause specific conversion has no source for its hazards, and the reference risk bullet still names Kaplan-Meier (lines 524 to 530; Absolute effects, lines 828 to 830). The hazards λ1 and λ2 are "required" but @sec-absolute estimates a reference risk, not two hazards, and the constant hazard assumption behind the formula is unstated; the hazard based bullet also says "reported Kaplan-Meier or cumulative incidence", which the competing risks paragraph forbids for outcomes with competing events. Replacement, after "competing hazard λ2": "both estimated as events per person year from the covering reference arms of @sec-absolute (the outcome and competing death respectively), under a constant hazard within follow-up, which is stated". In the hazard based bullet: "using reported cumulative incidence at that time when available (Aalen-Johansen for outcomes with competing events, Kaplan-Meier otherwise)".
The Kaplan-Meier ban discards a valid cause specific input (lines 518 to 520). The contrast of log(-log(1 - p)) between arms of one minus Kaplan-Meier with competing deaths censored estimates the cause specific log hazard ratio under proportional cause specific hazards (I confirmed this numerically under constant hazards); only its use as an absolute or reference risk is invalid. Replacement: "One minus a Kaplan-Meier estimate with competing deaths censored is never used as an absolute or reference risk for an outcome with competing events; a contrast of such estimates between arms is a cause specific input."
Cross references and data items (Confidence in cumulative evidence, lines 912 to 915; Data items, lines 411 to 415). The imprecision rule names only rare outcomes and reported rate ratios as using the ratio range; add "and estimates under the relative only rule of @sec-absolute, including the recorded harms analysis". The outcome level data items do not list what the new rules extract; add "survival estimator and hazard type; number with known status at t by arm; observation period".
3. Judgment on revisions made for the other reviewers
The three changes adopted from gpt-6-astra are correct, and I support them over the 0.5.0 text I accepted: the primary count rule now requires known status for every randomized participant, with t and known status defined as grok-4.7 proposed; the 95 percent rule is a sensitivity analysis with no claimed bound or direction; and the worked examples reproduce. Assigning cumulative incidence contrasts and complete status counts to subdistribution networks is right, and the conversion formulas are the standard ones. The relative only rule in @sec-absolute is numeric and reproducible, and the contribution rule now uses each trial's actual primary input with shared arm covariance preserved. The cost of the hazard type split is sparser harm networks, which item 1 addresses by ordering the fallbacks rather than by relaxing the split.
Conclusion
Every replacement from my round 5 review was carried into 0.6.0 or resolved by a route I judge sound; the invariants hold, the search claims reproduce, and the new numbers are correct. The six items above are sentence edits that complete rules whose substance the owner has already chosen (which network carries conclusions, what known status includes, where the hazards come from); none alters a model or an estimand, and I read item 1 as completing a primacy the protocol already fixes rather than as a decision still open. Applying the standard of rounds 4 and 5, I would approve the methods for registration once the administrative pre-registration conditions are completed, with the six edits made in the same pass.
VERDICT: ACCEPT