GPT-6 Astra

1. Major. Transitivity violations remain eligible for pooling. protocol.qmd, “Transitivity” permits downgrading when restriction or adjustment cannot resolve imbalance. populations.csv, P07, separates HFpEF and HFrEF only when both have connected evidence. Connectivity cannot justify clinical exchangeability. Replace: “When important intransitivity remains unresolved, we will not calculate indirect comparisons. HF phenotypes remain separate regardless of connectivity.” Add ejection fraction, albuminuria, fibrosis stage, and PAP use to condition-specific effect modifiers. Cochrane guidance

2. Major. Incompatible scales share outcome definitions. outcomes.csv, O13, combines BMI SDS and BMI using MD with one SDS threshold. O100 combines CDR-SB and ADAS-Cog despite differing constructs; its CDR-SB threshold cannot govern SMD results. O24 likewise supplies only an SMD threshold for MD analyses. Replace: Separate instrument-specific rows, units, and thresholds; permit SMD only for instruments measuring sufficiently similar constructs.

3. Major. Outcome direction is wrong for abstinence. outcomes.csv, O110, assigns lower_better to both consumption and abstinence. This reverses interpretation and rankings for abstinence. Replace: Separate substance-specific rows: consumption, continuous/MD/lower_better; abstinence, binary/OR/higher_better. Prespecify abstinence duration, verification method, and thresholds.

4. Major. Clinically different outcomes can be pooled. outcomes.csv, O22, mixes 40% and 50% eGFR decline and composites with or without cardiovascular death. O23 labels hospitalization but includes urgent visits. O36 combines pulmonary aspiration with retained gastric contents; O90 combines ovulation with regular menstruation. Replace: Give each definition its own outcome identifier. Pool only prespecified compatible definitions; analyze broader composites separately.

5. Major. Counts cannot automatically substitute for survival estimates. Bayesian analysis applies complementary log-log models with follow-up offsets whenever only counts exist, without censoring or hazard assumptions. Recurrent-event Poisson models also lack an overdispersion rule. Replace: “Use reported or validly reconstructed log-HRs. Use count-based survival approximations only under justified follow-up and hazard assumptions, separately in sensitivity analyses. For recurrent events, prefer reported estimates with appropriate variance or an overdispersion-capable model.” Effect-measure guidance

6. Major. Sparse-event instructions conflict with frequentist implementation. Frequentist analysis and zero events promise standard random-effects log-OR analysis without continuity corrections. Zero cells make ordinary study log-ORs or variances undefined. netmetabin(method="MH") supplies a common-effect model, not equivalent random-effects coverage. Replace: Specify a likelihood-based sparse-event analysis, identify MH explicitly as common-effect sensitivity analysis, and report resulting changes in included studies and connectivity. Package manual

7. Major. Estimands are knowingly mixed. “Estimands” substitutes trial-product or on-treatment results into treatment-policy analyses, while EM12 calls them nonexchangeable. Trial-product, hypothetical, while-on-treatment, and adherence effects are not interchangeable labels. Replace: “Classify each intercurrent-event strategy. Analyze incompatible estimands separately; do not substitute them into the primary treatment-policy network. Select RoB 2’s effect of interest from the actual causal question.” ICH E9(R1)

8. Major. Nonstandard trial designs lack executable analysis rules. Study designs permits crossover pooling whenever correlation is available, without addressing carryover. Cluster trials lack clustering corrections. Replace: “Use cluster-adjusted estimates; otherwise apply a documented design-effect correction with ICC sensitivity analysis. Use paired crossover estimates only when carryover and period effects are acceptably addressed; otherwise use first-period estimates or report descriptively.” Cochrane guidance

9. Major. Observational harms synthesis is undefined. Harms synthesis refers to outcomes “marked for nonrandomized evidence,” but no such column exists. Pooling “adjusted relative effects” leaves HR, OR, RR, exposure windows, and confounding requirements unresolved. Replace: Add nonrandomized_eligible; prespecify each harm’s target population, exposure, comparator, latency, estimand, minimum confounders, and effect measure. Pool compatible estimates separately by measure and design; exclude critically biased results from quantitative synthesis.

10. Major. Missing-evidence tool is inappropriate for NMA. Risk of bias and “Meta-biases” assign ROB-ME to every synthesis. ROB-ME targets pairwise meta-analysis; ROB-MEN addresses network estimates. Replace: “Use ROB-MEN for network estimates and ROB-ME for pairwise syntheses, integrating findings into certainty judgments.” Tool developers’ guidance

11. Major. Time rules create poorly comparable estimands. Time points can combine 20-week and 56-week diabetes efficacy results, while binary harms use unrestricted longest follow-up. O10’s 52–80-week window excludes shorter obesity efficacy trials altogether, including the 46-week survodutide trial. Replace: Prespecify separate short-, medium-, and long-term windows by indication; select nearest target consistently. Compare binary risks over compatible follow-up periods. Survodutide trial report

12. Major. Absolute effects and certainty thresholds lack reference risks and horizons. Effect measures uses median “control” risk across networks containing different active comparators and durations. Thresholds such as 5 deaths per 1,000 have no specified horizon. Replace: Prespecify reference treatment, population, time horizon, baseline-risk source, and uncertainty. Convert ORs or HRs to absolute effects using those inputs before applying thresholds; do not compare log-scale discrepancies directly with absolute thresholds.

13. Major. Bayesian specifications are incomplete and excessively diffuse. Priors omits heterogeneity priors for HRs and rate ratios. Log-effect N(0,10²) places approximately 95% probability on ratios spanning (3\times10^{-9}) to (3\times10^8). Replace: Supply a complete prior table for every likelihood and additional model. Consider log-effect N(0,1.5²) and log-HR/log-rate heterogeneity half-N(0,0.5²) as starting candidates, retaining them only after documented prior-predictive calibration.

14. Major. Nodes do not define flexible-dose strategies. Networks and nodes defines fixed maintenance-dose nodes, but studies randomize maximum-tolerated-dose strategies. nodes.csv also lacks a formulation field and leaves insulin-combination ratios unspecified. Replace: Add formulation, component ratio, and randomized dosing-strategy fields. Create separate flexible-dose nodes, such as tirzepatide 10/15 mg maximum tolerated dose. Never classify participants by achieved dose, which is postrandomization. SURMOUNT program design

15. Major. Withdrawal-population definition excludes intended landmark evidence. populations.csv, P02W, requires reaching a weight-loss target. STEP 4 required reaching and receiving semaglutide 2.4 mg, not a weight-loss threshold. Replace: “Participants randomized after a GLP-1-containing treatment run-in to continuation, withdrawal, or another maintenance strategy; record tolerability and response-enrichment criteria separately.” Distinguish prior run-in treatments when assessing transitivity. STEP 4

16. Major. Diabetes strata cannot classify all eligible trials. populations.csv, P01, offers only S0–S2. Basal-bolus and premixed-insulin backgrounds, mixed S0/S1 samples, and replacement versus add-on designs lack assignment rules. Replace: Add explicit insulin-regimen and mixed-background strata. Prefer randomized within-stratum estimates when available; otherwise retain mixed trials separately. Specify how withdrawn or replaced background drugs enter treatment nodes.

17. Major. Comparator rules contradict agreed scope. Comparators restricts active comparators largely to glucose- or weight-lowering drugs and categorically excludes surgery, although OTHER_ACTIVE includes procedures. Open-label design alone does not establish intransitivity. Replace: “Any active comparator is eligible; clinical compatibility determines synthesis grouping.” Also justify excluding comparator-only trials, or include eligible comparisons among prespecified nodes with corresponding searches.

18. Major. Search coverage is unresolved before registration. Information sources leaves CENTRAL and Embase contingent; Appendix A contains no reproducible strategy. Benchmark reviews come from the same harvest, limiting independent sensitivity validation. Replace: Resolve CENTRAL and Embase access or document justified alternatives; complete PRESS review; append actual rendered queries; validate against an independently assembled set spanning older drugs, emerging agents, observational harms, and registries. Cochrane search guidance

19. Major. Search exclusions undermine the all-agent inventory. concepts.yaml claims endogenous-hormone headings add no drug studies. Native GLP-1 infusion studies exist and meet the broad inventory definition. Development aliases also omit, for example, taspoglutide’s R1583/BIM51077. Replace: Include native GLP-1 administration terminology and verified historical aliases; validate rendered retrieval rather than assuming generic names occur. Infusion study, taspoglutide study

20. Minor. Seed node table omits established studied doses. nodes.csv lacks retatrutide 1 mg weekly, survodutide 0.6 mg weekly, and mazdutide 3 and 4.5 mg weekly. Replace: Add those regimens and reconcile seed nodes against trial arms before extraction. These are additions, not replacements for existing doses. Retatrutide, survodutide, mazdutide

21. Minor. Search dictionary conflates different agents. concepts.yaml, hrs9531, includes HRS-7535/HRS7535 as aliases. HRS9531 is an injectable GIP/GLP-1 dual agonist; HRS-7535 is an oral small-molecule GLP-1 agonist. Replace: Create separate agent keys and map them to GIP_GLP1 and GLP1_ORAL_SM, respectively. HRS9531 study, HRS-7535 study

22. Minor. Meta-regression eligibility is insufficiently specified. Meta-regression requires ten trials globally, regardless of covariate variation or class-interaction identifiability. Replace: Prespecify covariate priority, coding, centering, missingness handling, univariable versus multivariable models, and minimum information per coefficient. Fit interactions only with adequate overlap and identifiable parameters.

23. Minor. Registration and review-appraisal reporting remain incomplete. “Administrative information” lacks affiliations, corresponding-author mailing address, finalized reviewer roles, and support details. “Comparison with published systematic reviews” treats publication methods as a substitute for a protocol. Replace: Complete PRISMA-P items 3–5; record “No prospective protocol located” when appropriate, retaining publication methods solely as reported methods. PRISMA-P checklist

No files edited. Search-renderer assessment was static; read-only restrictions prevented R startup. VERDICT: MAJOR REVISION