Round 1: response to reviewers

Protocol version reviewed: 0.1.0. Revised version: 0.2.0.

All three reviewers returned MAJOR REVISION. Every comment is answered below and recorded in ../comments.csv. Comments are paraphrased; the full reviews are in this folder.

Summary of main changes

• New machine-readable tables: windows.csv (contiguous, population specific time windows), agents.csv, harms_nrs.csv, priors.csv, reference_sd.csv, turner_map.csv, registry_dose_candidates.csv.
• Outcomes restructured from 39 to 72 rows: one row per synthesis, with model scale, nonrandomized eligibility, threshold scale, absolute effect horizon and mapping rules.
• Populations: diabetes strata S0 to S4; heart failure (P07A, P07B), chronic kidney disease (P08A, P08B) and substance use (P14A to P14E) split; one 80 percent assignment rule.
• Nodes defined by rule (formulation, dosing strategy, component ratio; flexible and maximum tolerated dose nodes; reformulated oral semaglutide).
• Estimands separated (treatment policy primary; no substitution); cluster and crossover rules; duplicate extraction of outcome data.
• Complete prior specification using Turner 2015 Table IV; numeric CINeMA rules applied to both frameworks; absolute effects with declared reference node, baseline risk and horizon; rare outcome rule.
• Search: concepts version 1.1.0 with a mechanism block, new agents and codes; CENTRAL, CNKI, Wanfang and WHO INN monitoring added; Appendix A generated from the concept file; independent benchmark set.

GPT-6 Astra (gpt-6-astra)

C1 (major; Transitivity; populations.csv P07). Unresolved intransitivity still allows indirect comparisons; heart failure phenotypes split only if connected.

Response (revised). Agreed. Indirect and network estimates are not calculated for comparisons with unresolved intransitivity. HFpEF/HFmrEF (P07A) and HFrEF (P07B) are fixed separate populations. LVEF, albuminuria, fibrosis stage, AHI with PAP use, amyloid status and antipsychotic drug added as condition specific modifiers.

C2 (major; outcomes.csv O13 O24 O100). Incompatible scales share outcome rows (BMI SDS with BMI; CDR-SB with ADAS-Cog; HRQoL MD with SMD threshold).

Response (revised). Agreed. Split into O13A (BMI SDS) and O13B (percent BMI, only when SDS absent; never pooled); O100A (CDR-SB) and O100B (ADAS-Cog); O24A (SF-36 PCS, MD) and O24B (SMD across instruments only when O24A cannot be used).

C3 (major; outcomes.csv O110). Abstinence coded lower_better.

Response (revised). Agreed. One row per substance and construct (O110A heavy drinking days; O110B alcohol abstinence; O111 tobacco abstinence; O112 return to opioid use; O113 stimulant negative urine; O114 cannabis use days), each with its own measure, direction and threshold; populations split P14A to P14E.

C4 (major; outcomes.csv O22 O23 O36 O90). Clinically different definitions pooled.

Response (revised). Agreed. O22 is kidney failure, sustained eGFR decline of at least 50 percent or kidney death; broader composites (40 percent decline, cardiovascular death) are O22B and never pooled. O23 is hospitalization only and O23B adds urgent visits. O36A pulmonary aspiration and O36B retained gastric content; O90A ovulation, O90B menstrual regularity, O90C live birth.

C5 (major; Bayesian analysis). Count based cloglog models substitute for survival estimates without assumptions; no overdispersion rule for recurrent events.

Response (revised). Agreed for time to event outcomes: reported or validly reconstructed log hazard ratios are primary and count based models are sensitivity analyses only. For binary adverse events (participants with at least one event), which trials report as counts over very different durations, the cloglog model with a log follow-up offset is primary with its constant hazard ratio assumption stated, following NICE DSU TSD2, and the odds ratio model and duration restriction are sensitivity analyses (this also answers fable-5.1 F12). Recurrent events use reported rate ratios or a negative binomial model.

C6 (major; Frequentist analysis; zero events). Frequentist random effects cannot avoid continuity corrections; MH is common effect.

Response (revised). Agreed. Inverse variance random effects only when every arm has events (no correction); otherwise the frequentist estimate is Mantel-Haenszel, labeled as a common effect model, with the change in contributing trials reported. The Bayesian binomial model remains primary.

C7 (major; Estimands). Trial product results substituted into treatment policy analyses.

Response (revised). Agreed. Each result is classified by intercurrent event strategy; primary networks use treatment policy results only; hypothetical results form separate networks; no substitution. The RoB 2 effect of interest follows the causal question of each network.

C8 (major; Study designs). No executable rules for crossover and cluster trials.

Response (revised). Agreed. New section on cluster (design effect with trial ICC or 0.05, sensitivity 0.01 and 0.10) and crossover trials (paired contrast only with reported SE and addressed carryover; otherwise first period).

C9 (major; Harms synthesis). No column marks nonrandomized outcomes; pooling rules undefined.

Response (revised). Agreed. Added nrs_synthesis to outcomes.csv and harms_nrs.csv specifying population, exposure, comparators, designs, time zero and latency, estimand, minimum confounders and effect measures per harm; pooling separately by measure, design and comparator type; critical risk of bias excluded.

C10 (major; Risk of bias; meta-biases). ROB-ME is for pairwise meta-analysis; ROB-MEN is for NMA.

Response (revised). Agreed. ROB-MEN for network estimates and ROB-ME for pairwise syntheses; CINeMA reporting bias domain uses ROB-MEN.

C11 (major; Time points). Windows mix 20 and 56 week results; O10 window excludes shorter obesity trials.

Response (revised). Agreed. New windows.csv with contiguous primary and secondary windows per outcome and population, nearest target rule within each, duration as covariate. Obesity primary window is now 40 to 104 weeks (target 68), which includes the 46 week survodutide trial; binary harms use the cloglog model with follow-up offset.

C12 (major; Effect measures; thresholds). Absolute effects lack reference node, baseline risk source and horizon.

Response (revised). Agreed. Reference node rule declared before fitting; reference risk from a random effects meta-analysis of reference arms with low and high scenarios; outcome specific horizon (absolute_horizon_weeks); thresholds applied on the absolute scale at that horizon, with a relative threshold under the rare outcome rule.

C13 (major; Priors). Priors incomplete (no HR or rate heterogeneity priors) and too diffuse.

Response (revised). Agreed. Complete prior table (priors.csv) for every likelihood and model; relative effects Normal(0, 2.5^2) on log scales (1.5^2 for rare outcomes) and Normal(0, (2.5 s_y)^2) for mean differences; Turner Table IV predictive distributions (turner_map.csv) applied to log hazard and log rate ratios; prior predictive checks required. We chose 2.5 rather than 1.5 because weight loss responder odds ratios can exceed 50.

C14 (major; Nodes). No rules for flexible and maximum tolerated dose strategies; no formulation or ratio fields.

Response (revised). Agreed. nodes.csv now has formulation, dosing_strategy and component_ratio; flexible and maximum tolerated dose arms form their own nodes (for example TIRZ_SC_MTD_QW) with a highest dose sensitivity analysis; arms are never classified by achieved dose.

C15 (major; populations.csv P02W). Definition required reaching a weight loss target, which excludes STEP 4.

Response (revised). Agreed. P02W is any randomization after a GLP-1 receptor agonist containing run in, with run in criteria recorded separately and the run in regimen as an effect modifier (EM22).

C16 (major; populations.csv P01). Strata cannot classify basal bolus, premixed, mixed or replacement designs.

Response (revised). Agreed. Strata S0 to S3 (S3 prandial, premixed or basal bolus insulin) and S4 mixed background; the 80 percent rule governs assignment; replacement designs map each arm to its randomized regimen.

C17 (major; Comparators). Comparator scope narrower than agreed; surgery excluded by design.

Response (revised). Agreed in part. Any comparator, including surgery, is eligible, and clinical compatibility decides network entry. Trials without a GLP-1 receptor agonist containing arm remain ineligible; the justification and its consequences are now stated in the protocol.

C18 (major; Information sources; Appendix A). CENTRAL and Embase unresolved; Appendix A empty; benchmark not independent.

Response (revised). Agreed. CENTRAL is now searched; Embase depends on access confirmed before registration, with a stated justification if absent; Appendix A is generated from the concept file at build; the benchmark set adds regulatory pivotal trials and ClinicalTrials.gov linked publications independent of the harvest. PRESS review requires a human information specialist and is listed as a pre registration condition.

C19 (major; concepts.yaml). Endogenous hormone exclusion untested; native GLP-1 infusion excluded; missing taspoglutide aliases.

Response (revised). Partly agreed. Tested: the receptor agonist heading adds 0 records and the endogenous hormone heading adds 6,988, of which 484 are human infusion experiments; the protocol now defines scope as therapeutic drugs and states this explicitly, so native GLP-1 infusion is a scope exclusion. Taspoglutide codes R1583 and BIM51077 added.

C20 (minor; nodes.csv). Missing doses: retatrutide 1 mg, survodutide 0.6 mg, mazdutide 3 and 4.5 mg.

Response (revised). Agreed and extended: nodes are now defined by rule, the seed list includes these doses and others, and registry_dose_candidates.csv lists doses mined from registrations.

C21 (minor; concepts.yaml hrs9531). HRS9531 and HRS-7535 conflated.

Response (revised). Agreed. Separate agents and node families (GIP_GLP1 injectable; GLP1_ORAL_SM oral).

C22 (minor; Network meta-regression). Eligibility and modeling details missing.

Response (revised). Agreed. Priority order, centering and scaling, one covariate at a time, ten trials and overlap required, complete case with a 20 percent missingness limit, at most two covariates per multivariable model, and a stated prior.

C23 (minor; Administrative information; reviews). PRISMA-P items 3 to 5 incomplete; publication methods treated as protocol.

Response (revised). Agreed. Items 3 to 5 are structured with placeholders the owner must complete before registration (administrative checklist). Reviews without a prospective protocol are recorded as such.

Fable 5.1 (fable-5.1)

F1 (major; outcomes.csv O10). O10 window empties P12 and P15 networks.

Response (revised). Agreed. O10B with a 12 to 52 week window (target 24) for P12 and P15 (windows.csv W09).

F2 (major; outcomes.csv O110). O110 not executable.

Response (revised). Agreed; see gpt-6-astra C3.

F3 (major; outcomes.csv O11 O27). Several syntheses packed under one outcome ID.

Response (revised). Agreed. O11A to O11E and O27A to O27E.

F4 (major; outcomes.csv O32 O34 O36). Absolute thresholds above plausible risks of rare harms.

Response (revised). Agreed. Rare outcome rule: when the median reference risk at the horizon is below 2 per 1,000 the threshold is a hazard or odds ratio of 1.25 and absolute effects are given per 10,000.

F5 (major; Harms synthesis). No nrs column; long latency harms missing.

Response (revised). Agreed. nrs_synthesis column; added pancreatic cancer (O38), any cancer (O39), gastroparesis (O37A), obstruction or ileus (O37B), completed suicide (O35B), major congenital malformations (O120) and acute kidney injury (O121); harms_nrs.csv specifies each.

F6 (major; populations.csv P01; outcomes.csv O02). Weight management trials in diabetes lose their primary time point.

Response (revised). Agreed. P01 added to O10 and O11 with the 40 to 104 week primary window; such trials are analyzed in their background stratum.

F7 (major; populations.csv P01 strata). Prandial, premixed and basal bolus insulin trials have no stratum.

Response (revised). Agreed in substance; we split S2 (basal insulin) and S3 (prandial, premixed or basal bolus) rather than using a covariate, and added S4 for mixed backgrounds, as grok-4.7 G3 also proposed.

F8 (major; Comparators). Trials with GLP-1 as shared background produce self loops.

Response (revised). Agreed. A trial enters a network only when at least two arms map to different nodes; such trials are inventoried only.

F9 (major; Nodes). No rules for flexible dose, free combinations, randomized comparator doses.

Response (revised). Agreed for flexible dose and free combinations. For randomized comparator doses we keep one molecule level node and combine arms within the trial, so that node sets stay identical across networks.

F10 (major; populations.csv P07). HF phenotypes split only if connected.

Response (revised). Agreed; see gpt-6-astra C1. We use LVEF above 40 percent versus 40 percent or lower, matching the eligibility of the dedicated GLP-1 receptor agonist heart failure trials, with restriction to 50 percent or more as a sensitivity analysis.

F11 (major; Priors). No heterogeneity priors for HR and rate outcomes; comparison type unfixed.

Response (revised). Agreed. Turner distributions applied to log hazard and log rate ratios. The comparison type follows a rule fixed from network geometry before outcome extraction (active versus active distribution when more than half of direct comparisons are between active drugs), which grok-4.7 G8 proposed; we judged it closer to the evidence than fixing one type for all networks.

F12 (major; Effect measures O25 to O36). Odds ratios at longest follow-up pool very different durations.

Response (revised). Agreed. Binary harms use the cloglog model with a log follow-up offset as primary; the odds ratio model is a sensitivity analysis.

F13 (major; Information sources; Appendix A). Appendix A empty; Embase and CENTRAL undecided; no Chinese databases.

Response (revised). Agreed. Appendix A generated at build; CENTRAL searched; Embase decided before registration with a fallback justification; CNKI and Wanfang Data searched with Chinese terms in concepts.yaml, verified by a Chinese-reading team member.

F14 (major; nodes.csv). Node list incomplete and already behind the evidence; 9 mg retatrutide unverified.

Response (revised). Agreed. Nodes are now defined by rule so that new regimens do not need amendments; the seed list adds the named agents and doses (node families for agents whose doses registrations mask); retatrutide 9 mg is kept with a verification note because phase 3 programs use it; registry_dose_candidates.csv lists 861 dose candidates mined from registrations. A complete enumeration needs full text extraction, which cannot precede screening.

F15 (major; Data collection). Single extraction plus verification for outcome data.

Response (revised). Agreed. Outcome data are extracted independently in duplicate.

F16 (minor; Outcomes and time points). Critical outcomes wording mismatched to files.

Response (revised). Agreed; wording corrected.

F17 (minor; outcomes.csv O22 O50). Mixed kidney composites.

Response (revised). Agreed; see gpt-6-astra C4 (O22 and O22B; O50 uses the O22 definition).

F18 (minor; outcomes.csv O04). No mapping for older hypoglycemia categories.

Response (revised). Agreed. Mapping rule added in mapping_rules.

F19 (minor; populations.csv P14; nodes.csv). P14 ambiguous; P99 note repetition; dose note on placebo rows.

Response (revised). Agreed. P14A to P14E; P99 note fixed; notes on placebo, standard care, lifestyle and surgery corrected.

F20 (minor; effect_modifiers.csv). No condition specific modifiers; EM11 undefined.

Response (revised). Agreed. EM16 to EM22 added; EM11 categories defined.

F21 (minor; Selection process). Agreement statistic unnamed.

Response (revised). Agreed. Cohen kappa of at least 0.8 with percent agreement reported.

F22 (minor; Data collection hierarchy). Hierarchy could conflict.

Response (revised). Agreed. Ordered: prespecified analysis for the estimand with longest follow-up of that estimand, then regulatory review, registry, publication.

F23 (minor; Frequentist analysis). Contribution algorithm unnamed.

Response (revised). Agreed. netcontrib shortest path method, as used by CINeMA (verified as the netmeta 3.7.0 default).

F24 (minor; Network meta-regression). Subgroups cannot feed ML-NMR without IPD.

Response (revised). Agreed; wording corrected.

F25 (minor; Confidence in evidence). Only critical outcomes rated.

Response (revised). Agreed. Critical and important outcomes are rated.

F26 (minor; Rationale). Overstates approved uses.

Response (revised). Agreed. Rationale distinguishes approved from studied uses.

F27 (minor; concepts.yaml). Untested MeSH claim; missing mechanism terms and codes.

Response (revised). Agreed. Claims tested and reported with counts; mechanism block added (190 additional PubMed records on 2026-10-12); codes AM833, NNC0174-0833, SHR20004 (noiiglutide), PB-119, HM15211 (efocipegtrutide), GZR18, CT-388 and HS-20094 added, plus UBT251, zenagamtide and DR10624 found by the mechanism block. NN9487 returned no PubMed record and was not added.

Grok 4.7 (grok-4.7)

G1 (major; nodes.csv). Oral semaglutide note wrong; missing doses and agents; dose response across routes.

Response (revised). Agreed. 3, 7 and 14 mg labeled original tablet; reformulated 1.5, 4 and 9 mg tablets are separate nodes with a bioequivalence pooling sensitivity; 25 and 50 mg labeled the higher dose program; missing doses and agent node families added (cotadutide once daily in micrograms; maridebart cafraglutide every 4 weeks); dose response restricted to one molecule, route and formulation.

G2 (major; outcomes.csv O13 O22 O24 O50 O100 O110). Outcomes cannot be pooled as coded.

Response (revised). Agreed; implemented as proposed (O13A and O13B; O22 primary kidney composite and O22B; O24A with a 3 point threshold and O24B SMD; O100A and O100B; substance specific rows).

G3 (major; Participants; populations.csv P01). 80 percent rule contradicted; strata incomplete.

Response (revised). Agreed. Single assignment rule (80 percent at baseline or a baseline defined subgroup); strata S0 to S4 as proposed.

G4 (major; outcomes.csv O01 O10). Windows drop primary outcomes for P04, P05, P12, P15.

Response (revised). Agreed. Population specific windows: P04 and P05 HbA1c 12 to 40 weeks (target 26) with a 40 to 64 secondary window; P12 and P15 weight 12 to 52 weeks (target 24); P02 and P03 40 to 104 weeks (target 68). We used nearest to target within the window for every outcome rather than the trial primary time point, as gpt-6-astra C11 asked for one consistent rule.

G5 (major; Harms; thresholds). Odds ratios over unstated follow-up; recurrent events; impossible thresholds; sparse networks.

Response (revised). Agreed in part. Binary harms use the cloglog model with follow-up offset; recurrent hypoglycemia episodes have a rate ratio outcome (O04B); networks with fewer than 20 events are descriptive. For rare outcomes we adopted a relative threshold (hazard or odds ratio 1.25, as fable-5.1 F4 proposed and as CINeMA commonly uses) rather than 0.1 per 1,000, because an absolute threshold that small cannot be estimated with useful precision from trials. Gastrointestinal events and pancreatitis remain participant level outcomes because trials rarely report episode counts with exposure.

G6 (major; R/search_strategy.R; concepts.yaml). PubMed does not truncate inside quotes; HRS agents merged; cagrilintide alone too broad.

Response (partly revised). Partly agreed. Tested on 2026-10-12: "GLP-1 receptor agonist*"[tiab] returns 8,109 records, identical to the singular and plural phrases combined, and "incretin-based therap*" returns 1,260 against 1,239 for the two explicit forms, so PubMed does truncate inside quoted phrases and the renderer is correct. HRS9531 and HRS-7535 separated. Cagrilintide alone is kept: CagriSema trials randomize cagrilintide monotherapy arms and registrations name it alone, and the term retrieves about 100 records.

G7 (major; Effect measures; GRADE). Reference node and horizon undefined.

Response (revised). Agreed. Reference node rule (placebo, then standard care, then most frequent active node) and a 156 week horizon for O14, O20 to O23, O50; 52 weeks for harms and O40 (heart failure trials); stored in outcomes.csv.

G8 (major; Priors). Baseline and relative effect priors mismatched; Turner comparison type.

Response (revised). Agreed in part. Logit and cloglog baselines Normal(0, 5^2) rather than Normal(0, 2^2): a prior centered on 50 percent with SD 2 shrinks rare event baselines (risk near 1 in 1,000 lies 3.5 SD away). Relative effects Normal(0, 2.5^2) and Normal(0, 1.5^2) for rare outcomes (between your proposal and gpt-6-astra C13). The comparison type rule is adopted as proposed. Nonrandomized models use a half normal prior with scale 1 on tau.

G9 (major; Harms synthesis). No nrs column; overlap rule.

Response (partly revised). Agreed in part. nrs_synthesis and harms_nrs.csv added with the source overlap hierarchy. Mortality, MACE, heart failure, serious adverse events and discontinuation are not synthesized from nonrandomized studies because the review owner limited nonrandomized synthesis to harms; trials provide that evidence.

G10 (major; populations.csv P07 P08). HF and CKD defined as both one and two networks.

Response (partly revised). Agreed in part. Fixed separate populations: P07A (LVEF above 40 percent) and P07B (40 percent or lower); P08A (with diabetes) and P08B (without). We did not split HFmrEF from HFpEF because the dedicated GLP-1 receptor agonist containing heart failure trials enrolled participants with LVEF of at least 45 or 50 percent, so a separate 41 to 49 percent network would have no dedicated trials; restriction to 50 percent or more is a sensitivity analysis.

G11 (major; Confidence in evidence). CINeMA mixes frameworks; no numeric rules; thresholds not locked.

Response (revised). Agreed. CINeMA applied separately to Bayesian and frequentist estimates with the lower rating kept; numeric rules for every domain written into the protocol; thresholds locked before any outcome data are extracted.

G12 (major; Designs; data collection). No cluster or crossover rules; source hierarchy conflict; completers.

Response (revised). Agreed; implemented as proposed (design effect with ICC 0.05 default; crossover paired contrast only with reported SE; ordered hierarchy; non treatment policy results flagged).

G13 (major; Administrative; sources; Appendix A). Pre registration items open; no INN monitoring.

Response (revised). Agreed. Administrative checklist added; Appendix A generated; CENTRAL committed and Embase decision required before registration; quarterly check of WHO proposed INN lists added as a source.

G14 (minor; node_classes.csv). Beinaglutide half life.

Response (revised). Agreed; corrected.

G15 (minor; nodes.csv). Prandial insulin and glinides not molecule level; no OTHER_ACTIVE node.

Response (revised). Agreed. Aspart, lispro, glulisine and regular insulin; repaglinide and nateglinide; OTHER_ACTIVE rule node.

G16 (minor; Frequentist analysis). netmeta adds 0.5 to zero cells by default.

Response (revised). Agreed; see gpt-6-astra C6 (no continuity correction; Mantel-Haenszel when zero cells occur).

G17 (minor; Harms synthesis). HKSJ misnamed.

Response (revised). Agreed. REML (Sidik and Jonkman if REML fails) with the Hartung-Knapp adjustment.

G18 (minor; effect_modifiers.csv EM02). BMI in children.

Response (revised). Agreed. BMI SDS for P03.

G19 (minor; outcomes.csv O05 O10 O12). Thresholds too small.

Response (declined (provisional thresholds to be locked by clinical panel)). Not adopted at this stage. A 5 percentage point difference is the regulatory efficacy benchmark against placebo, not a minimal important difference between active regimens, and a 2 mmHg systolic difference is associated with meaningful population level cardiovascular risk change. All thresholds are provisional and will be confirmed or changed by clinical advisors and patient partners and locked before any outcome data are extracted.

G20 (minor; outcomes.csv O51 O80). WOMAC scale and eGFR slope sign.

Response (revised). Agreed. WOMAC rescaled to 0 to 100; eGFR slope defined with decline negative.

G21 (minor; populations.csv P04 P05). Missing hypoglycemia (P04) and insulin dose (P05).

Response (revised). Agreed. O04 for P04; O06 total daily insulin dose for P05.

G22 (minor; outcomes.csv O21). No MACE mapping rule.

Response (revised). Agreed. O21 is three point MACE; expanded composites are O21B and never pooled.

G23 (minor; Meta-regression; component models). Overfitting; component coding for fixed ratio products.

Response (revised). Agreed; implemented as proposed (one covariate at a time, Normal(0,1) prior on the standardized covariate, bias covariates as sensitivity analyses, 20 percent missingness limit, presence or absence coding, fixed ratio products excluded from Emax).

G24 (minor; R/search_strategy.R; selection). OpenAlex query stored unfielded; agreement statistic.

Response (revised). Agreed in part. The run log already stores the fielded OpenAlex query (search.title_and_abstract=...), and Appendix A now states the parameter. Calibration requires Cohen kappa of at least 0.8 (stricter than 0.70, as fable-5.1 F21 proposed), with percent agreement reported.