Peer review round 15: LivingGLP1 protocol 0.15.0 (reviewer fable-5.1)
What I verified against the 0.14.0 snapshot: the revision changes protocol.qmd (subtitle; the round and version counts at lines 43 and 239; the competing risk bullets at lines 620 to 622 and 641 to 671; the absolute effects paragraph at lines 997 to 998 and 1018 to 1021), amendments.csv (A014) and the README status line. The twelve other authoritative tables, concepts.yaml (1.2.0) and R/search_strategy.R are byte identical to the reviewed snapshot, so the invariants of rounds 5 to 14 hold. comments.csv carries the eight round 14 rows that A014 counts, verdicts.csv the three round 14 verdicts, protocol.html the 0.15.0 subtitle and the PR16 cross reference. Numerical checks: the single interval piecewise integral equals the closed form F(t) (0.038824 for hazards 0.04 and 0.02 over one year); log{−log(1 − F)} of a Kaplan-Meier value and log(λt) of a rate coincide under a constant hazard, so the new pooling scale is one quantity; the delta method standard error from a Kaplan-Meier interval of 0.08 to 0.125 around 0.10 is 0.12 on that scale.
1. Round 14 comments
| Item | Status | Reason |
|---|---|---|
| F14-1 | Partly resolved | Case (a), one total count per arm over exactly the horizon, is settled: lines 658 to 661 make that piecewise fit the exponential approximation with its gate and label. Case (b), single reported intervals that pass the horizon, is settled in the response ("only counts from 0 to the horizon qualify") but not in the text. The piecewise bullet (lines 589 to 594) admits a single reported count interval from 0 that ends after the horizon whenever all covering arms share it; only digitized curves are truncated at the horizon (line 593). Such a fit is a one interval constant hazard integrated to the horizon, yet the trigger at line 658 reads "from 0 to the analyzed horizon" literally, so it carries neither gate nor label, and line 666 defines d_k over an interval these arms do not report. Arms whose single intervals end at different times past the horizon are either identical "through the horizon" (piecewise, unlabeled) or not (no route), depending on how line 590 is read. The restriction also departs from line 1017, where an arm without a competing event contributes its rate "within their own follow-up" placed at the horizon as log(λt); under one constant hazard judgment a rate over two years estimates the same hazard as a rate over one, so the asymmetry is a choice rather than a reason. The residual affects which fit carries the gate and label, not F(t). Replacement text in item 1 below. |
| F14-2 | Resolved | Lines 997 to 998 cite PR17 with PR16 for the competing risk routes. |
| F14-3 | Resolved | Lines 641 to 646: with one arm u_new is not drawn and each draw is from N(y_a, V_a); with two or more, μ + Z u_new at that draw's τ_k; exponentiation is stated for the reference and the scenario draws. |
| F14-4 | Resolved | Lines 1018 to 1021 adopt the log cumulative hazard model under PR17. One word of my own text needs fixing (item 2). |
| F14-note | Partly resolved | The prior half is resolved: "to match PR17" is a thin reason for scale 1, but 0.5 and 2 are the sensitivity priors, so the choice is testable and I accept it. The horizon half (reported Aalen-Johansen values that miss the analyzed horizon) stays deferred to the pre-analysis amendment by the round 12 agreement; I restate it so it is not lost. |
2. New items in 0.15.0
No major items. None touches a relative effect model, an estimand or a relative estimate.
Minor
Single reported intervals that pass the horizon (lines 658 to 666; residual of F14-1). Two closures, either of which settles the gap. The smaller edit keeps the G14-3 restriction and adds to the piecewise bullet after "through the analyzed horizon" (line 590): "a single reported count interval that starts at 0 and ends after the analyzed horizon qualifies for neither route". The other restores the availability of 0.14.0 and matches line 1017: replace "If the identical boundaries are the single interval ... and the same pooling model applies." with "Whenever every covering arm has a single reported interval that starts at 0 and reaches or passes the analyzed horizon, the fit is the exponential approximation: from counts, each arm's constant rate over its own person time on that interval is its hazard on the single interval from 0 to the horizon; from curves, the inversion formulas above with t_s = 0; so the identical boundary requirement is met by construction and the piecewise pooling model applies. An arm that reports finer count intervals contributes the sums of its counts and person time over them when any other covering arm reports a single interval. In either form the route is used only when constant hazards are supported, as judged below, and carries the exponential approximation label." Then replace "Here d_k is the count of event k from time 0 to the analyzed horizon" with "Here d_k is the arm's count of event k on that interval". The owners should note that the second closure reverses the restriction they adopted for G14-3; I prefer it because it applies the same standard as line 1017, but both are executable.
"Reported interval" means two things at line 1020. In this section "interval" is a time interval; my adopted text meant the confidence interval. Replace "from its reported interval" with "from its reported confidence interval; a Kaplan-Meier value reported without one and without a standard error enters as the arm's rate when valid person time exists and is otherwise logged as excluded", which is consistent with lines 1015 to 1017.
No prior on the mean of the PR17 meta-analysis (priors.csv; pre-existing, made visible by the new scale). Line 831 says the complete specification is in
priors.csv, but PR15 is the mean of the competing risk routes, PR17 is τ only, and PR02 is the NMA trial baseline μ_j, which the text never ties to the separate reference arm meta-analysis at line 996. So that model has no stated prior on its mean on the logit, log cumulative hazard or log rate scale. One row fixes it: extend PR02'sapplies_toto "All binary and count outcomes; the mean of the reference arm meta-analysis of @sec-absolute on the same scales", or add PR18 with Normal(0; 5²), sensitivity Normal(0; 10²), rationale as PR02. This has survived every round unflagged and is amendment level, but it must be filled before any absolute effect is computed.
3. Judgment on revisions made for the other reviewers
Grok's three items coincide with or extend mine. G14-1 and F14-3 are resolved by the same sentences; G14-2 adds the y_a and V_a definition for counts at lines 620 to 622, which is correct and executable; G14-3 supplied the wording that resolved case (a) of F14-1 and left case (b) open, as above. GPT-6 Astra had no comments. Nothing else changed: the twelve other tables, concepts.yaml and the search renderer are byte identical to the 0.14.0 snapshot, A014's "8 comments" matches comments.csv, and the rendered HTML carries 0.15.0. The response overstates nothing except the claim that G14-3's wording settles arms whose totals cover different follow-up.
Conclusion
Three of my four round 14 items are adopted in full and the fourth for the common case, with a residual that affects only which label and gate a one interval fit carries, not its F(t). The three items above sit in one branch of the absolute effect routes or in one prior table row; item 2 corrects a word of my own text and item 3 predates this version. By the standard I applied in rounds 12 to 14, where items of this size were open at verdict time, they are amendment level, to be settled in the next revision or in the pre-analysis amendment that already carries the Aalen-Johansen horizon matter. The administrative items remain on the checklist as pre-registration conditions. I would approve the methods for registration once those conditions are completed.
VERDICT: ACCEPT