Peer review round 12: LivingGLP1 protocol 0.12.0 (reviewer fable-5.1)
What I verified against the 0.11.0 commit: the revision changes only protocol.qmd (front matter; lines 41 and 239; Data items, lines 419 to 426; Competing risks, lines 598 to 638; Absolute effects, lines 964 to 970; 40 insertions, 23 deletions) and amendments.csv (A011). The twelve other authoritative tables, concepts.yaml (1.2.0) and R/search_strategy.R are byte identical to the reviewed 0.11.0 snapshot, so the invariants of rounds 5 to 11 hold. comments.csv carries the eight round 11 rows that A011 states, verdicts.csv the three round 11 verdicts, protocol.html the 0.12.0 subtitle and outcomes.csv the 77 outcomes the text counts. I recomputed the two facts behind the declined part of F11-2. C11-1's two histories (50 censored, then five events of each type among 50; versus ten of each type among 100, then 40 censored) both give F_1 = F_2 = 0.10, S = 0.80 and 40 at risk, so endpoint ordinates and the starting risk set do not identify the counts. The Greenwood variance of log(−log S) at 90 events among 100 is 0.01698 against 1/90 = 0.01111, while at event fractions of 5 to 10 percent the two agree to three decimals.
1. Round 11 comments
| Item | Status | Reason |
|---|---|---|
| F11-1 | Resolved | Lines 603 to 605: hazards are carried onto grid intervals "with their uncertainty and as one draw". |
| F11-2 | Partly resolved | (b) The Kaplan-Meier condition 0 < S_k(t_e) < S_k(t_s) is stated separately from the Aalen-Johansen condition at lines 613 to 615. (c) Lines 601 to 602 add the interval from the last published time to the analyzed horizon when the curve extends to it. (d) The number at risk rule is gone; lines 625 to 628 make the route unavailable only for missing inputs, nonpositive or infinite hazards, or missing joint uncertainty, so a count interval needs no number at risk. (a) My reconstruction formulas were declined in favor of C11-1's rule, judged below. |
| F11-3 | Resolved | Lines 966 to 970: the reported Aalen-Johansen value when it is the source, otherwise the incidence computed from the arm's own qualifying hazards. |
| F11-4 | Resolved | Line 4 reads 2026-10-13. |
The declined proposal. The response gives two reasons: within interval censoring leaves curve derived counts unidentified, and even a known count gives the variance of the Poisson estimator rather than of the curve inverted log hazard. Both are correct, as my recomputation shows. My d_k = n_s ΔF_k/S(t_s) is exact only without censoring inside the interval, and 1/d_k is the parametric variance of a different estimator, which diverges from the nonparametric variance when the event fraction is large. The adopted rule (lines 618 to 625) requires a reported or author supplied joint covariance of all ordinates used and propagates it through the Jacobian of the inversion; otherwise the route is unavailable. The cost is practical, not methodological: joint covariances of curve ordinates are almost never published, so the curve branch of piecewise integration will rarely qualify, and competing risk absolute effects will come from reported counts with person time, the exponential approximation, a reported Aalen-Johansen value, or the relative only rule. That is acceptable. The stricter rule errs toward unavailability rather than toward wrong pooling weights, and every fallback is already specified in the route orders at lines 640 to 654 and 984 to 992. If the team later wants the curve branch to be usable, reconstruction of individual time to event data from the curves and the numbers at risk (the method behind "validly reconstructed" at line 476) with a model based covariance would be an executable amendment; I do not make it a condition.
2. New items in 0.12.0
No major items. The three items below are clarifications of text that is otherwise exact; none changes a model, an estimand or a relative estimate.
Minor
Preserving dependence under per interval pooling (lines 622 to 630). "With this dependence preserved in pooling" sits beside "pooled per grid interval across arms with the random effects model of the reference risks", and the text does not say how a per interval model preserves one arm's covariance across its intervals and across the two events. Append after "distribution)": "Each curve arm enters as one multivariate normal likelihood on its vector of log hazards over its intervals and both events, with covariance JΣJ^T; the random effects model is fitted per grid interval and event on the components of these vectors, and integrated risks and scenario values are computed from joint posterior draws."
Data items wording (lines 419 to 426). "any reported joint covariance of the curve ordinates (requested from authors when not published), where published" contradicts itself on a literal reading: the covariance is requested when not published, yet the item applies only where published. Move "where published" to directly after "digitized reference arm curves" at line 419 and delete it from line 425.
Ordinates paired with an author supplied covariance (line 620). When Σ is author supplied, the inverted ordinates should be the authors' own estimates at those times, not digitized readings; otherwise Σ describes quantities other than the ones inverted. Insert after "is required": ", together with the authors' estimates of those ordinates when Σ is author supplied; digitized values are inverted only with a published covariance at the same times".
Two pre-existing matters I note without counting them as new errors or against the verdict: the reported Aalen-Johansen value serves as F_ref "at the analyzed horizon" (line 647) although published values rarely sit at the median follow-up of the reference arms, and the random effects meta-analysis of reference arm risks (lines 963 to 965) has no heterogeneity prior row in priors.csv (PR09 covers relative effects). Both predate 0.12.0 and can be settled by amendment.
3. Judgment on revisions made for the other reviewers
C11-1 is adopted essentially verbatim at lines 615 to 628: Poisson variance for reported counts with valid person time, no counts inferred from curve increments, the Jacobian propagated covariance, the one draw clause for subdivided intervals (shared with F11-1 and G11-1), and the fallback to another eligible route. C11-2 and G11-2 are adopted at lines 966 to 970 in wording that fits the subdistribution order at lines 645 to 654 and 989 to 992. G11-1 is partly adopted with the same decline reason as F11-2a; the remainder (separate Kaplan-Meier condition, count intervals without a number at risk, d_k defined in the exponential bullet at lines 634 to 635, the data items wording at lines 419 to 426) is carried. The response overstates nothing.
Conclusion
Three of my four round 11 items are resolved in my wording and the fourth is resolved in three of its four parts, with the declined part replaced by a stricter rule whose reason I have verified numerically and accept. The other reviewers' items are correctly carried, the authoritative tables are unchanged, and the three new items are clarifications that make the covariance rule fully executable without altering a model, an estimand or a relative estimate. The administrative items remain on the checklist as pre-registration conditions. I would approve the methods for registration once those conditions are completed.
VERDICT: ACCEPT