Peer review round 13: LivingGLP1 protocol 0.13.0 (reviewer fable-5.1)
What I verified against the 0.12.0 commit: the revision changes only protocol.qmd (front matter line 3; lines 41 and 239; Data items, lines 419 to 426; Competing risks, lines 615 to 641; 32 insertions, 21 deletions) and amendments.csv (A012). The twelve other authoritative tables, concepts.yaml (1.2.0) and R/search_strategy.R are byte identical to the reviewed 0.12.0 snapshot, so the invariants of rounds 5 to 12 hold. comments.csv carries the five round 12 rows that A012 states, verdicts.csv the three round 12 verdicts, protocol.html the 0.13.0 subtitle, and outcomes.csv the 77 outcomes the text counts. I recomputed the two facts behind G12-M1: Var(log h) = Var(h)/h² is 625 and 2,500 times Var(h) at hazards of 0.04 and 0.02 per year, and one log hazard draw of variance v copied onto ten equal grid pieces gives a cumulative hazard variance ten times that of ten independent draws. Both confirm that the 0.12.0 passage needed the correction now made.
1. Round 12 comments
| Item | Status | Reason |
|---|---|---|
| F12-1 | Resolved in substance | Lines 633 to 638 pool the arms' vectors in one multivariate random effects model and take the uncertainty of the integrated incidence from joint predictive draws, which is stronger than my per interval fit on the components. The executable detail in my wording (how between arm heterogeneity is structured) was dropped with it; see item 1 below. |
| F12-2 | Resolved | Line 420: "where published" now qualifies the curves; lines 423 to 425 request the covariance, with the authors' ordinate estimates, when it is not published. The contradiction is gone. |
| F12-3 | Resolved | Lines 624 to 626 adopt my wording: author supplied Σ is paired with the authors' own ordinate estimates, and digitized values are inverted only with a published covariance at the same times. |
| F12-note | Deferred, reason sound | Deferred to an amendment before the first analysis, as I suggested, and recorded in comments.csv so that it is not lost. Item 1 supplies the prior row that would settle its second half. |
2. New items in 0.13.0
No major items. Both items below complete the specification of the curve and count branch of piecewise integration; neither changes a model for relative effects, an estimand or a relative estimate, and the route keeps its prespecified fallbacks.
Minor
The multivariate random effects model has no stated structure (lines 633 to 636). "The arms' vectors are pooled in one multivariate random effects model of the reference risks" names neither the between arm covariance nor the link between an arm's coordinates (its reported intervals) and the grid intervals when arms report different intervals. With two to five covering arms, a free heterogeneity matrix over all grid intervals and both events is not identifiable, so an implementer must choose a structure, and the choice sets the pooled weights and the width of the predictive distribution. The per interval rule of 0.12.0 did not have this gap, and the sentence's antecedent is the curve inversion text, so it is also unclear that count arms enter the same model. Replace lines 633 to 636 ("The arms' vectors ... predictive distribution") with: "Every covering arm enters one multivariate random effects model as a multivariate normal likelihood on its vector with known covariance: JΣJ^T for curves, diagonal with entries 1/d_k for counts (independent across events and intervals). The model has one reference log hazard μ_{g,k} per grid interval g and event k, and one arm level random effect per event shared across that arm's reported intervals (a proportional deviation of its hazard function), independent across arms and events with variance τ_k² under prior PR15. An arm's true log hazard on a reported interval r is log Σ_{g ⊂ r} (|g|/|r|) exp(μ_{g,k}) plus that random effect, which is exact under the arm's constancy and equals μ_{g,k} when r is one grid interval. The reference hazard on each grid interval is the median of its marginal predictive distribution for a new arm." Add to
priors.csv: "PR15, Heterogeneity tau (reference arm risks), log_odds; log_hazard; log_rate, Random effects meta-analysis of reference arm risks and reference hazards (@sec-absolute; piecewise integration and exponential approximation), Half-normal(0; 0.5^2), Half-normal(0; 1^2), Reference arms are pooled on the logit or log scale with arm level random effects; weakly informative on the log scale, matching the PR09 sensitivity prior". A stated alternative, such as independent random effects per grid interval, would also be executable; what matters is that the structure, the link to the grid and the prior are written before analysis.Scenario values are tied to the wrong distribution (lines 636 to 638 against lines 977 to 981). The revision added "and of the scenario values" to the sentence on joint predictive draws. The scenario values are defined at lines 977 to 981 as each covering arm's own cumulative incidence at the horizon, and the predictive distribution describes a new arm, not a specific one. Replace "and the uncertainty of the integrated cumulative incidence and of the scenario values uses draws from the joint predictive distribution, so that copied intervals stay one draw" with: "and the uncertainty of the integrated cumulative incidence uses draws from the joint predictive distribution, so that copied intervals stay one draw; each covering arm's own cumulative incidence (the scenario values of @sec-absolute) is integrated from joint draws of that arm's vector under the same model, which keeps its copied intervals one draw as well."
One convention I note without counting it: the point estimate of F_ref integrates the marginal medians of the grid hazards while its interval comes from joint draws, so the reported value need not be the median of its own interval. The same holds for the univariate reference risks of @sec-absolute and predates 0.13.0; if the team prefers, "the reported reference risk is the median of the integrated draws" settles it.
3. Judgment on revisions made for the other reviewers
G12-M1 is adopted almost verbatim at lines 615 to 641: independent log hazards with variance 1/d_k for counts; Σ over every ordinate used, including the horizon ordinate; the event free survival rule; g mapping ordinates to log hazards with J its Jacobian, stated explicitly as the derivative of the log hazards and not of the hazards; copies of a reported interval as one coordinate; joint predictive draws so that copies stay one draw; and unavailability for a nonfinite covariance. My numerical checks above confirm Grok's arithmetic, and the adopted rule removes both the 625 to 2,500 fold misweighting and the factor n understatement of the integrated variance. The two additions to Grok's passage are the author supplied ordinate rule (my F12-3) and the scenario clause (item 2). GPT-6 Astra had no comments. The response overstates nothing.
Conclusion
Three of my four round 12 items are resolved and the fourth is deferred for the reason I gave. G12-M1 is correctly carried, the authoritative tables are unchanged, and the formula that mattered is now right. The two new items complete the specification of one branch of one route for competing risk absolute effects; the branch will rarely qualify, its fallbacks are prespecified at lines 651 to 665 and 995 to 1003, and neither item changes a model for relative effects, an estimand or a relative estimate. By the standard I applied in round 12, both are amendment level, to be settled in the next revision or in the same pre-analysis amendment as the F12-note. The administrative items remain on the checklist as pre-registration conditions. I would approve the methods for registration once those conditions are completed.
VERDICT: ACCEPT