Peer review round 16: LivingGLP1 protocol 0.16.0 (reviewer fable-5.1)
What I verified against the 0.15.0 snapshot: the revision changes protocol.qmd (subtitle; the round and version counts at lines 43 and 239; the piecewise bullet at lines 591 to 595 and 623 to 625; the exponential approximation bullet at lines 661 to 675; the absolute effects passage at lines 1000 to 1004 and 1021 to 1049), priors.csv (the applies_to field of PR02), amendments.csv (A015) and the README status line. The eleven other authoritative tables, concepts.yaml (1.2.0) and R/search_strategy.R are byte identical to the reviewed snapshot, so the invariants of rounds 5 to 15 hold. comments.csv carries the seven round 15 rows that A015 counts, verdicts.csv the three round 15 verdicts, and protocol.html the 0.16.0 subtitle and the new passages. Numerical checks: under a constant hazard λ with Λ(t) = λt, the Poisson mean e^θ E/t equals λE and the complementary log log offset log(f/t) gives log{−log(1 − p)} = θ + log(f/t) exactly, so both new likelihoods are on the stated scale; 365.25/7 = 52.18 weeks per year; the standard error for the 0.08 to 0.125 example of my round 15 review is 0.121 read on the risk scale and 0.120 with transformed limits (item 3).
1. Round 15 comments
| Item | Status | Reason |
|---|---|---|
| F15-1 | Resolved | Lines 661 to 675 adopt the second closure in substance: a piecewise fit on a single interval from 0 is the exponential approximation; when the piecewise route is unavailable, every covering arm with counts and person time over one period from 0 that reaches or passes the horizon qualifies, finer intervals are summed, the gate and label apply in both cases, and d_k is now the arm's count over its own period. Lines 591 to 593 let a last count interval end after the horizon with its rates applied up to it. The response's reason (a rate over a longer period estimates the same hazard under the gate) is correct. One reading of "identical boundaries" is still open (item 4). |
| F15-2 | Resolved | Line 1028 reads "reported confidence interval or Greenwood variance" with 0 < F < 1; a Kaplan-Meier value without either enters through the arm's counts, and an arm with no qualifying input is listed with the reason (lines 1035 to 1037). The response's observation that exclusion now depends on reporting, not on the event count, is right. |
| F15-3 | Resolved | PR02's applies_to names the mean of the reference arm meta-analysis on the same scales, line 1003 cites it, and A015 records the change. The rationale column still describes the baseline as a log cumulative hazard per year, whereas the reference arm parameter is log Λ(t) at the horizon; with Normal(0; 5²) the difference of log t is immaterial, so one clause in the rationale would do. |
| F15-note | Partly resolved | Deferred to the pre-analysis amendment by the round 12 agreement, unchanged; restated so it is not lost. |
2. New items in 0.16.0
No major items. None touches a relative effect model, an estimand or a relative estimate; all four sit in the reference arm model of @sec-absolute or in one clause of the piecewise route.
Minor
Precedence between the two count likelihoods, and the definition of f_a (lines 1029 to 1034). "Its event count by a Poisson likelihood ... or its participants with an event by a binomial likelihood" does not say which is used when an arm reports both events with person time and participants with an event, and these are different quantities. "The follow-up f_a used in the outcome's own analysis" does not say whether the known status condition that governs the same complementary log log input in the network model (lines 505 to 507 and 829) applies; without it the reference arm model admits counts the network model rejects. Replace the passage from "otherwise, under a constant hazard" to "own analysis." with: "otherwise, under a constant hazard within the arm's own follow-up, its event count by a Poisson likelihood with mean e^{θ_a} E_a / t when valid person time E_a (in years, as defined in @sec-outcomes) is reported; otherwise its participants with an event by a binomial likelihood with complementary log log link and offset log(f_a / t), where f_a in years is the time at which that arm's counts qualify for the hazard ratio analysis of @sec-outcomes (status known for every randomized participant); an arm whose counts qualify for neither has no count input."
Scenario risks for arms that enter by counts (lines 1005 to 1010). "The 25th and 75th percentiles of trial reference risks" is defined in the parenthetical for outcomes with competing events but not for the arms the revision now keeps: a zero count arm has an empirical risk of 0 and no finite estimate on the log scale, and no scenario uncertainty is described for outcomes without a competing event, unlike lines 652 to 657. Insert before "for outcomes with competing events": "for outcomes without a competing event, each covering arm's own risk at the horizon: its Kaplan-Meier value when that is its input, otherwise 1 − exp(−e^{θ_a}) with θ_a estimated from the arm's own likelihood under PR02 alone; scenario uncertainty comes from drawing each arm's θ_a from that posterior independently across arms and recomputing the percentiles in each draw;". This parallels the one arm rule at line 1048.
Scale of the delta method from a reported confidence interval (lines 1027 to 1028). A Kaplan-Meier interval may be built on the risk scale or on the log{−log} scale; reading the standard error off the risk scale and applying the delta factor, or transforming the limits and halving the width, give 0.121 and 0.120 for the example above but 0.71 and 0.64 for an interval of 0.005 to 0.06 around 0.02. Replace "with variance by the delta method from its reported confidence interval or Greenwood variance" with "with standard error equal to half the width of its reported confidence interval after transforming both limits by log{−log(1 − ·)}, divided by 1.96, or from the Greenwood variance by the delta method". Transforming the limits respects their asymmetry.
Boundary matching when a last interval passes the horizon (lines 589 to 590). Now that a last count interval may end after the horizon, "boundaries identical across all covering arms through the analyzed horizon" does not say whether an arm whose last boundary is exactly the horizon matches an arm whose last boundary passes it; the answer decides whether the piecewise route is available. Replace that clause with "on reported intervals whose boundaries at or before the analyzed horizon are identical across all covering arms, a boundary after the horizon counting as the horizon".
3. Judgment on revisions made for the other reviewers
C15-1 (GPT-6 Astra) is resolved by an alternative, and the reason for declining the proposed replacement is sound: zero event reference arms are routine for the harms outcomes, and the exact binomial and Poisson likelihoods keep them without correction while meeting both requirements of the proposal. The algebra of the two likelihoods is correct (checks above). The reason given at lines 623 to 625 for keeping the working normal likelihood in the competing risk routes is thin: line 833 already combines arm level and contrast level likelihoods in one model, so a per event, per interval Poisson for count arms beside the normal for curve arms would be routine, and the consequence of the choice is that a zero count reference arm still disables both competing risk routes for exactly the outcomes the response names as routinely zero, several of which have competing deaths. It is a stated choice, not an error, and the zero count rule at lines 620 to 621 and 671 to 672 makes its effect explicit; closing the asymmetry is amendment level. G15-1 is resolved by the same passage as F15-1; removing the "met by construction" clause is correct. G15-2 is resolved (t in years, 52.18 weeks per year, rates per person year, variance input named); item 3 is the residual. Nothing else changed.
Conclusion
All three of my round 15 items are adopted and the note stays deferred as agreed. The four items above are of the same size as the three I accepted at 0.15.0: each is a precedence, definition or scale clause inside one branch of the absolute effect routes with a one sentence closure, and none changes an estimand, a relative effect model or a registration level decision. By the standard I applied in rounds 12 to 15 they are amendment level, to be settled in the next revision or in the pre-analysis amendment that already carries the Aalen-Johansen horizon matter. The administrative items remain on the checklist as pre-registration conditions. I would approve the methods for registration once those conditions are completed.
VERDICT: ACCEPT