Peer review round 5: LivingGLP1 protocol 0.5.0 (reviewer fable-5.1)

What I verified on 2026-10-12 against the frozen 0.4.0 copy in round-4/reviewed-version-0.4.0/: the only files that changed are protocol.qmd, outcomes.csv (O22D, O22E and O50 mapping rules; O22E populations), nodes.csv (the iGlarLixi switching row became a template) and amendments.csv (A004); concepts.yaml and every other table are byte identical, and R/search_strategy.R still renders the four sources with the PubMed count unchanged. A script over the 0.5.0 tables finds 77 outcomes; no overlapping windows for any outcome and population; exactly one primary window for every pair except O36A, O36B and O120; every primary outcome listing its population; every log scale outcome mapped once in turner_map.csv; every continuous outcome in reference_sd.csv; nrs_synthesis flags matching harms_nrs.csv row for row; every node and agent class resolving; 11 template rows; the 43 agents and their terms identical to concepts 1.2.0; and 16 round 4 rows in comments.csv, as A004 states. The rendered 1.2.0 PubMed query runs with no field errors (six development codes return no records, as the protocol expects) and returns 28,673 records, as in rounds 3 and 4. I recomputed the attenuation example in the hazard ratio analysis: true risks 0.50 and 0.75 with 5, 10 and 20 percent lost at the start give 1.93, 1.88 and 1.79, and at risks of 2.5 and 5 percent the bias is 0.3 percent or less, so the stated numbers are correct.

1. Round 4 comments

No response declined or partly adopted any of my proposals.

2. New items in 0.5.0

No major items. The items below are gaps in text that the revision added or deleted; none changes a model, an estimand or a primary estimate.

Minor

1. Recorded harms analysis: absolute effects and imprecision (Event outcomes and harms, lines 486 to 495; Absolute effects; Confidence in cumulative evidence). The new analysis carries conclusions whenever the hazard ratio analysis fails the connectivity rule, and its absolute effects are "recorded risk differences over the median observation period", but @sec-absolute has bullets only for fixed time, hazard based and per procedure outcomes, and the imprecision and heterogeneity rules judge against "the absolute scale at the horizon", which this analysis does not have. Replacement, a fourth bullet in Absolute effects: "Recorded harms analysis: the horizon is the median observation period of the contributing trials; the reference recorded risk is the predictive median of a random effects model of the reference arms' recorded proportions on the complementary log log scale with the log observation period offset, evaluated at that period; the relative effect $d$ is applied as $p_t = 1 - (1 - p_{\mathrm{ref}})^{\exp(d)}$, and the result is labeled a recorded risk difference." In the imprecision bullet, after "for rate outcomes synthesized from reported rate ratios", add: "; for the recorded harms analysis, the per 1,000 threshold is applied to the recorded risk difference over the median observation period".

2. "Compatible follow-up" lost its definition (Frequentist network meta-analysis, line 673; Confidence in cumulative evidence, line 866). Version 0.4.0 defined it as 20 to 104 weeks in the sentence the revision deleted; 0.5.0 still restricts the frequentist odds ratio analysis of harms to "trials with compatible follow-up" and rates "odds ratios within compatible follow-up" separately, and the sensitivity list no longer names this analysis. Replacement at line 673: "restricted to trials with follow-up of 20 to 104 weeks"; at line 866: "(for example odds ratios within 20 to 104 weeks of follow-up)"; and append to the sensitivity list "frequentist odds ratios for harms within 20 to 104 weeks of follow-up".

3. Cause specific versus subdistribution hazard ratios (Competing risks, lines 500 to 503; Networks and nodes). The two "are synthesized separately", but the network key (population, population stratum, outcome, definition stratum, window, estimand) has no hazard ratio type, and neither is named primary. Replacement, appended to the competing risks paragraph: "The cause specific hazard ratio network is primary; subdistribution hazard ratios form a secondary network with its own connectivity check, and a trial reporting both contributes each to its own network."

4. "Negligible mortality" is undefined and the outcome list is incomplete (Competing risks, lines 500 to 508). The parenthetical names only log hazard outcomes, while the exemption refers to adverse events, so binary harms with 156 week horizons in cardiovascular and kidney outcome trials (O32A, O38, O39 and others) are implicitly covered but unlisted, and the exemption has no cutoff. Replacement: "For outcomes that death can preclude (O14, the O21, O22 and O23 families, O40, O50, and binary harms in trials in which 5 percent or more of randomized participants died during follow-up)" and, for the last sentence, "Binary harms in trials in which fewer than 5 percent of randomized participants died are exempt."

5. Composite definitions (lines 454 to 457). O22C appears twice with two different glosses, "doubling of serum creatinine, about a 57 percent decline" and "kidney failure, sustained doubling of serum creatinine, or kidney death"; the revision added the second without removing the first, so the illustration reads as six identifiers. Replacement: delete "O22C (doubling of serum creatinine, about a 57 percent decline), ".

6. Outcomes and time points (line 239). "77 outcomes at version 0.4.0" is stale; replace with "0.5.0".

One implementation note, not a finding: the arm level entry of reported cumulative incidence with offset log t is not a multinma likelihood when mixed with binomial arms in the same network, so it falls under the custom Stan clause; the equivalent contrast level entry (the difference of log(-log(1 - p)) between arms with a delta method standard error) is a normal contrast likelihood that multinma fits directly, and the analysis team may prefer it.

3. Judgment on revisions made for the other reviewers

The two analysis structure for harms (C4-1) is sound, and I judge it an improvement over the 0.4.0 text I had accepted: the hazard ratio analysis admits counts only when event status is known for at least 95 percent of randomized participants in every arm, which caps the attenuation at the 2.0 to 1.93 shown in the protocol's own worst case example and far less at the risks typical of harms; the recorded harms analysis keeps every trial synthesized under a label that claims no hazard interpretation; and the rule for which analysis carries conclusions is stated and executable, subject to item 1 above. The contribution rule (C4-2, G4-1) matches what all three reviewers proposed. The competing risks paragraph (C4-3) is correct in substance and needs only the primacy and cutoff sentences in items 3 and 4. The switching strategy template (C4-4, G4-5), the definition stratum in the network key (G4-3), the randomized denominators (G4-4) and the single covering reference arm rule (G4-7) are correct and executable, and the table checks above confirm that the template row and the mapping rules were entered consistently.

Conclusion

Every replacement from my round 4 review was carried into 0.5.0, the invariants I check hold, the search claims reproduce, and the administrative conditions remain tracked in the checklist. The six items above are sentence or bullet edits that restore a dropped definition or complete the rules for the two analyses the revision introduced; none alters a model or an estimand, and I apply the same standard as in round 4. I would approve the methods for registration once the administrative pre-registration conditions are completed, with the six edits made in the same pass.

VERDICT: ACCEPT