Round 2: response to reviewers

Protocol version reviewed: 0.2.0. Revised version: 0.3.0.

Verdicts: GPT-6 Astra MAJOR REVISION; Fable 5.1 MINOR REVISION; Grok 4.7 MINOR REVISION. Each reviewer also rated its round 1 comments (see the reviews). Every new comment is answered below and recorded in ../comments.csv.

Summary of main changes

• Harms: clarified what the complementary log log model assumes; procedure outcomes per procedure; frequentist odds ratio analyses as sensitivity analyses only.
• Absolute effects: window target horizons for fixed time outcomes; hazard horizons capped at reference arm follow-up; a priori rare outcome flag.
• Outcomes: kidney composites O22 to O22D; O21B narrowed; O24B and O24C by construct; abstinence definitions; 76 outcomes.
• Windows: rebuilt with a selection rule; checked for overlap and a single primary window per outcome and population; heart failure, sleep apnea, MASH and pediatric windows fixed.
• Nodes: flexible node definition; reformulated oral semaglutide pairs; IcoSema, iGlarLixi and IDegLira ratios; templates flagged; retatrutide 9 mg removed.
• Nonrandomized harms: explicit specification per outcome; separate pooling by estimand, latency and contrast.
• CINeMA: incoherence rule on the direct minus indirect interval; relative ranges; contribution domains judged once.
• Search concepts 1.2.0: amycretin and zenagamtide merged; IDegLira supplementary concept fixed; Chinese candidate names; NN9487; count reconciliation logged.

GPT-6 Astra (gpt-6-astra)

C2-1 (major; Event outcomes and harms; O36A O36B). Count based cloglog model needs more than a constant hazard ratio; procedure outcomes are not hazards.

Response (partly revised). Agreed in part, with a clarification now in the protocol. Because every trial has its own baseline, the relative effect from the complementary log log model is a log hazard ratio under proportional hazards within each trial whatever the shape of the baseline hazard, provided follow-up is similar across arms (trials with markedly different exposure are flagged); the constant hazard assumption enters only the baseline model used for absolute effects, which is never extrapolated beyond observed follow-up. Reported hazard ratios are used when available, and odds ratio analyses within compatible follow-up are sensitivity analyses. O36A and O36B are now risks per procedure on the odds ratio scale with no time horizon.

C2-2 (major; Frequentist analysis). Mantel-Haenszel cannot estimate hazard ratios; zero non-events ignored.

Response (revised). Agreed. Frequentist binary analyses are on the log odds ratio scale; inverse variance only when every arm has events and non-events, otherwise Mantel-Haenszel common effect; for harms with a hazard scale primary model the frequentist odds ratio analysis is a sensitivity analysis within compatible follow-up, not a replicate.

C2-3 (major; outcomes.csv O21B O22B O24B O110B O111). New subdivisions still mix definitions.

Response (revised). Agreed. O21B is four point MACE with unstable angina only (other expanded composites described); kidney composites are O22 (50 percent or doubling), O22B (40 percent), O22C (with cardiovascular death) and O22D (with macroalbuminuria or declines below 40 percent), never pooled, with P08A and P08B added to O22B and O22C; O24B is physical functioning and O24C total HRQoL; O110B defines abstinence in the final 4 weeks; O111 is verified continuous abstinence and O111B point prevalence abstinence.

C2-4 (major; windows.csv). Overlapping O05 windows; population matching; O04B; narrow windows; kg versus percent.

Response (revised). Agreed. O05 has P01 windows (W01 to W03) and P02 windows (W27, W28); a stated rule applies windows only where outcome populations match; a check confirmed no overlaps and exactly one primary window per outcome and population; O04B is in W00 (longest follow-up); narrow windows are target plus or minus 4 weeks; the text distinguishes O01 and O02 from O10.

C2-5 (major; Absolute effects). Fixed horizons cannot be obtained from arbitrary follow-up.

Response (revised). Agreed. Fixed time outcomes use the analyzed window target and that window reference risk with no extrapolation of odds ratios; hazard outcomes use the stated horizon capped at the median follow-up of the reference arms, with no extrapolation beyond observed follow-up.

C2-6 (major; harms_nrs.csv; nonrandomized synthesis). Estimands not separated.

Response (revised). Agreed. Pooling is now separate by population, exposure and comparator contrast, estimand (initial treatment versus on treatment), latency and risk window, effect measure and design; estimates without the minimum confounders are excluded from quantitative synthesis; every harms_nrs.csv row is now explicit.

C2-7 (major; Nodes; nodes.csv). Flexible nodes underdefined; IcoSema and iGlarLixi ratios.

Response (revised). Agreed. Flexible and maximum tolerated dose nodes are defined by formulation, permitted dose set and algorithm; IcoSema 700 U and 2 mg per mL (350 U per 1 mg, verified from the manufacturer); iGlarLixi pens (100 U with 50 or 33 micrograms per mL) with pen switching per trial algorithm and a ratio specific sensitivity analysis; IDegLira 100 U and 3.6 mg per mL.

C2-8 (major; Confidence in cumulative evidence). Incoherence rule leaves cases unclassified; inestimable incoherence.

Response (revised). Agreed. Incoherence is judged from the interval of the direct minus indirect difference against plus or minus the threshold (within, outside, otherwise); inestimable incoherence is recorded and judged from transitivity and network structure with reasons.

C2-9 (minor; priors.csv). SMD s_y; O04B prior; dispersion prior.

Response (revised). Agreed. s_y equals 1 for SMD outcomes; O04B uses only reported log rate ratios with prior PR14, so no dispersion prior is needed (a count model would require an amendment).

C2-10 (minor; agents.csv; concepts.yaml). Amycretin and zenagamtide are one molecule.

Response (revised). Agreed and verified with the manufacturer release: merged under amycretin with zenagamtide and NN9487 as search terms.

C2-11 (minor; effect_modifiers.csv; Participants; meta-regression). EM02 lacks P03; heart failure pooling wording; categorical coding.

Response (revised). Agreed. EM02 includes P03 and P04 (BMI SDS); heart failure populations are never pooled in the text too; categorical covariates use prespecified indicator coding, and class coefficients need at least three trials with covariate variation.

Fable 5.1 (fable-5.1)

F2-1 (major; outcomes.csv O22). Registered composites of LEADER, REWIND and SELECT unmapped.

Response (revised). Agreed. O22D added for composites with macroalbuminuria or eGFR declines below 40 percent; doubling of serum creatinine maps to a 50 percent decline; a composite without albuminuria is used for O22 or O22B when also reported.

F2-2 (major; windows.csv W12). KCCQ window empties the P07B network.

Response (revised). Agreed. P07B primary window 12 to 36 weeks (target 26) with 36 to 64 as secondary; P07A keeps 36 to 64 (target 52) as primary with 12 to 36 as secondary.

F2-3 (major; windows.csv O05). O05 overlaps.

Response (revised). Agreed; see C2-4.

F2-4 (minor; windows.csv W00 W25). Free text and longest.

Response (revised). Agreed; see G2-5.

F2-5 (minor; windows.csv; time windows text). Window gaps (sleep apnea, MASH, pediatric) and text claims.

Response (revised). Agreed. W14 lower limit 24 weeks; MASH secondary window 12 to 48 (target 24); pediatric secondary window 20 to 40 (target 26); text corrected for O01, O02 and O10 and for windows without a secondary.

F2-6 (minor; concepts.yaml pubmed_mesh). Drug combination lines parse as text.

Response (revised). Agreed and verified (84 and 81 records as text; 0 as quoted concepts). IDegLira[nm] (57 records) replaces the first; the iGlarLixi line is removed because no supplementary concept exists.

F2-7 (minor; nodes.csv RETA_SC_9MG_QW). 9 mg cannot be verified.

Response (revised). Agreed. Row and notes removed; the node rule recreates it from a report.

F2-8 (minor; priors.csv; O36B; O120). Offset units; cutoffs; non rare outcomes with relative thresholds.

Response (revised). Agreed. Offset in years stated; one a priori rare flag drives prior and threshold; O36B (50 per 1,000 procedures) and O120 (5 per 1,000 pregnancies) have absolute thresholds.

F2-9 (minor; concepts.yaml chinese_terms). No Chinese terms for Chinese origin investigational agents.

Response (revised). Agreed. Candidate names added under candidates for verification, and both databases are also searched with English names and development codes, as the protocol now states.

F2-10 (minor; turner_map.csv O14). Wrong Turner category for incident diabetes.

Response (revised). Agreed. Remapped to infection or onset of new disease.

F2-11 (minor; effect_modifiers.csv EM02). P03 missing.

Response (revised). Agreed; see G2-7.

F2-12 (minor; populations.csv P01). Stratum delimiter.

Response (revised). Agreed; see G2-6.

F2-13 (minor; search/runs; Timeline). No logged run of the current concepts; ClinicalTrials.gov counts unreconciled.

Response (revised). Agreed. The run log now records reconciliation (PubMed: 28,454 identifiers, 28,320 articles and 134 book chapters; ClinicalTrials.gov: 2,384 and 1,038 across two queries with 667 in both, union 2,755), the code logs this automatically from now on, and the Timeline no longer claims validation: a logged harvest of all four API sources with the registered concepts version precedes the baseline search. The OpenAlex harvest awaits an API key from the owner.

F2-14 (minor; F11 follow-up). Define direct comparison for the Turner comparison type rule.

Response (revised). Agreed. A direct comparison is a distinct pair of nodes with direct evidence.

F2-15 (minor; F27 follow-up). NN9487 should be kept even with zero hits.

Response (revised). Agreed. NN9487 and NN-9487 added; the protocol states that development codes are kept regardless of current counts.

Grok 4.7 (grok-4.7)

G2-1 (major; windows.csv O05). Overlapping O05 windows.

Response (revised). Agreed; see C2-4.

G2-2 (major; nodes.csv SEMAR2). Bioequivalence notes read as pooling 1.5 mg with 14 mg.

Response (revised). Agreed. Each reformulated row names its single bioequivalent original dose and node, and the text states the three pairs.

G2-3 (major; outcomes.csv horizons). 156 week horizon in heart failure populations.

Response (revised). Agreed in substance. Rather than population specific rows, the horizon of every hazard based outcome is capped at the median follow-up of the network reference arms, which gives about 52 weeks in heart failure networks and also prevents extrapolation elsewhere (C2-5).

G2-4 (minor; priors.csv; rare outcome rule). Rare outcome switch depends on extracted data; cutoffs disagree.

Response (revised). Agreed. A rare_outcome flag in outcomes.csv, fixed before extraction, drives both the tighter prior and the relative threshold; it is never recomputed from observed risks. O35A, O36B and O120 are not flagged rare and have absolute thresholds.

G2-5 (minor; windows.csv W00 W25). Longest follow-up rows not joinable.

Response (revised). Agreed. selection_rule column; W00 lists outcome IDs explicitly; blank upper limits mean no limit; nearest target does not apply.

G2-6 (minor; populations.csv P01). Stratum delimiter.

Response (revised). Agreed; fixed.

G2-7 (minor; effect_modifiers.csv EM02). P03 missing.

Response (revised). Agreed; P03 and P04 added.

G2-8 (minor; outcomes.csv O21B O22B). Expanded composites lack heart failure and kidney populations.

Response (revised). Agreed. O21B includes P07A and P07B; O22B and O22C include P08A and P08B.

G2-9 (minor; harms_nrs.csv). As above cells; estimand pooling.

Response (revised). Agreed; see C2-6.

G2-10 (minor; CINeMA). No range for relative thresholds.

Response (revised). Agreed. Range 1/1.25 to 1.25; discrepancies as differences of log effects against log 1.25.

G2-11 (minor; outcomes.csv O120). Blank horizon.

Response (revised). Agreed. Per pregnancy (and per procedure for O36A and O36B) outcomes are exempt from horizons and stated as such.

G2-12 (minor; Absolute effects). Most frequent active node undefined.

Response (revised). Agreed. The active node that appears in the most trials.

G2-13 (minor; nodes.csv SEMA_PO_FLEX_QD). Formulation blank.

Response (revised). Agreed. Original tablet.

G2-14 (minor; nodes.csv templates). Template rows could collapse doses.

Response (revised). Agreed. is_template column; templates never enter a network.

G2-15 (minor; CINeMA). Bayesian contribution weights undefined.

Response (revised). Agreed. Contribution based domains are judged once from the frequentist shortest path matrix and apply to both frameworks; imprecision, heterogeneity and incoherence are judged per framework; the lower rating is kept.