Peer review round 10: LivingGLP1 protocol 0.10.0 (reviewer fable-5.1)
What I verified against the 0.9.0 commit: the revision changes only protocol.qmd (Data items, Event outcomes, Competing risks and Absolute effects), amendments.csv (A009) and the README; the twelve other authoritative tables, concepts.yaml (1.2.0) and R/search_strategy.R are byte identical to the reviewed 0.9.0 copies, so the invariants of rounds 5 to 9 hold. comments.csv carries the nine round 9 rows that A009 states, verdicts.csv the three round 9 verdicts, protocol.html the 0.10.0 subtitle, and outcomes.csv the 77 outcomes the text counts.
1. Round 9 comments
| Item | Status | Reason |
|---|---|---|
| F9-1 | Resolved | Lines 636 to 639 adopt the support source order verbatim: curves, otherwise two or more time points, otherwise the exponential approximation is not used. |
| F9-2 | Resolved | Lines 584 to 586 require the inputs "from every covering reference arm"; mixed availability makes the route unavailable rather than mixing routes. |
| F9-3 | Resolved | Lines 419 to 423 list both curves with uncertainty and risk set information; lines 600 to 603 state the dependence assumption in my wording. |
| F9-4 | Partly resolved | The grid fixed before pooling and the carry over of an arm's hazard onto grid intervals that subdivide its own are adopted (lines 590 to 592). The derivation was not: my minus log ratio of the Kaplan-Meier estimate at the interval ends, and the solution of both hazards from two Aalen-Johansen increments and the event free survival, were replaced by one formula, the increment divided by event free survival at the start of the interval and by the length (lines 592 to 595). The response says "Agreed" and gives no reason for the substitution; the substituted formula is biased and conflates the two curve types (item 1). |
| F9-5 | Resolved | Lines 943 to 954 separate outcomes without a competing event from the two hazard types, each with its own order, using grok-4.7's G9-1 sentence, which I endorse. |
2. New items in 0.10.0
No major items: no model, estimand or relative estimate changes. Item 1 is a stated formula that moves absolute reference risks in one direction, which is why the verdict is not ACCEPT; items 2 and 3 are the executability gaps left in the same paragraph. One replacement paragraph covers all three.
Minor
The per interval hazard formula is biased under the stated constancy and wrong for Kaplan-Meier curves (lines 592 to 595). Under constant hazards h_1 and h_2 on an interval of length Δ, the Aalen-Johansen increment is ΔF_k = (h_k/H) S(t_s)(1 − e^{−HΔ}) with H = h_1 + h_2, so ΔF_k/{S(t_s)Δ} equals h_k(1 − e^{−HΔ})/(HΔ), not h_k. With h_1 = 0.2 and h_2 = 0.1 per year (the C8-1 example) the factor is 0.963 on a quarter year grid, 0.929 at half a year and 0.864 at one year, and the integrated reference risk at five years is 0.510, 0.501 and 0.484 against 0.518. Both hazards shrink by the same factor, so h_1/H is right and the cumulative incidence is too low, with every absolute difference built on it. The jump identity dF_k = S(u−) dΛ_k that gpt-6-astra cited is exact only at a jump, and nothing requires the grid to be fine (item 2). Applied to a target curve that is one minus Kaplan-Meier with competing deaths censored, the formula errs the other way, because the denominator should then be the Kaplan-Meier survival rather than the event free survival: 0.210 for a true 0.2 on a half year grid, while minus the log ratio of the Kaplan-Meier estimate gives 0.200. Lines 587 to 589 ("digitized curves of the target cumulative incidence") no longer say which estimator a curve uses, and the derivation differs between the two; add to line 420 "with the estimator of each curve (Kaplan-Meier with the other event censored, or Aalen-Johansen)".
"Reported time points" and "reported intervals" are undefined for curves (lines 590 to 592), and the definition decides whether the route exists. If a digitized curve's reported time points are its steps, each arm's own intervals contain an event of one type and none of the other, the other log hazard is minus infinity, and the no omission, no constants rule of lines 598 to 600 makes the route unavailable for every curve derived input. If they are the times of the published numbers at risk, the route is usable and the count behind each increment is recoverable. The second reading is the intended one and should be written. It also makes the grid coarse (6 to 12 months in outcome trials), which is where item 1's bias is largest, so the two fixes belong together.
"Every arm interval needs a variance" (line 597) has no source for curve inputs, and the exponential route has no pooling or zero count rule (lines 604 to 607). Published pointwise bands do not give the variance of an increment. The executable source is the reconstructed event count d_k, from the increment and the number at risk at the interval start, with variance 1/d_k on the log hazard; without a number at risk the route is unavailable. The exponential bullet should say that λ_1 and λ_2 are each pooled across covering arms with the reference risk model and that a zero count makes it unavailable, as for the piecewise route. An arm level Poisson likelihood on d_k with offset log person time would remove both the variance and the zero count rule, but that is a model change and I do not require it.
Replacement for lines 590 to 600 (from "The intervals form one grid" to "this route is unavailable."):
"The intervals form one grid fixed before pooling: the union over covering arms of the ends of each arm's reported intervals, which are the intervals of its reported events and person time or, for digitized curves, the intervals between the times of its published numbers at risk; an arm's hazards are constant within each of its own reported intervals and are carried onto every grid interval that the reported interval contains. On a reported interval from t_s to t_e of length Δ, an arm's hazards are: from counts, the event count of each type over the person time at risk; from Aalen-Johansen curves of both events, with S = 1 − F_1 − F_2 (or F_2 = 1 − S − F_1 when event free survival is published instead), H = −log{S(t_e)/S(t_s)}/Δ and h_k = H ΔF_k/(ΔF_1 + ΔF_2); from a Kaplan-Meier curve of event k with the other event censored, h_k = −log{S_k(t_e)/S_k(t_s)}/Δ. These forms are exact under the stated constancy. The variance of each log hazard is 1/d_k, where d_k is the reported event count or, for curves, the count reconstructed from the increment and the number at risk at the start of the interval. An arm interval with a zero count for either event, or without a published number at risk, makes this route unavailable; no constants are added and no interval or arm is omitted. Log hazards are pooled per grid interval across arms with the random effects model of the reference risks (median of the predictive distribution)."
- Low and high risk scenarios (lines 931 to 932). State that under either route the trial reference risks whose 25th and 75th percentiles form the scenarios are each covering arm's own F_1a(t) from its own hazards.
3. Judgment on revisions made for the other reviewers
C9-1 is carried in substance (both curves from the same arm and risk set, every covering arm, one grid, no silent omission, no added constants, unavailability stated); its one defect is the interval formula of item 1, which was not in the proposed text. G9-1 is adopted verbatim and resolves F9-5 with it. G9-2's per arm interval variance is adopted without a source (item 3). G9-3 is carried: Kaplan-Meier contrasts for O20 are not applicable inputs (lines 477 to 480), and O20 routing stays consistent at lines 489 to 492, 536 to 538 and 1013 to 1017. The response overstates nothing except that F9-4 is recorded as revised when its estimator was replaced.
Conclusion
Four of my five round 9 items are resolved in my wording, and the other reviewers' items are correctly carried. The remaining defect is one paragraph: the per interval estimator understates both cause specific hazards by (1 − e^{−HΔ})/(HΔ) on any grid coarser than the curve's jumps, does not distinguish the curve estimator, and has no variance source. The replacement paragraph fixes items 1 to 3 together without changing a model, an estimand or a relative estimate. I withhold ACCEPT only because a biased formula would otherwise be registered as the primary absolute risk route for cause specific networks.
VERDICT: MINOR REVISION