Full text screening · amendment A021
Full text screening
6,782 records with an open full text were screened at full text by language model reviewers working from a fixed manual, with a re-check of flagged decisions and coordinator rulings. Every decision row is kept: corrections are appended and the latest row is the current decision. Records behind paywalls are screened when their full texts are retrieved.
Decisions and reasons
| Decision | Reason | Records |
|---|---|---|
| Randomized trial for the network meta-analysis | · | 1,073 |
| Nonrandomized study for a harms synthesis | · | 212 |
| Systematic review (inventory and cross check) | · | 1,332 |
| Listed in the inventory, not synthesized | N1 No concurrent comparison group | 144 |
| N2 Pharmacokinetic or mechanistic study without a review outcome | 41 | |
| N3 Randomized contrast does not change the GLP-1 receptor agonist containing regimen | 58 | |
| N5 Nonrandomized design not eligible for any harms synthesis | 751 | |
| N6 Comparative nonrandomized study without an eligible harms outcome | 842 | |
| N7 Retracted primary report | 2 | |
| N8 Protocol or design document without results | 194 | |
| N9 Ancillary document of an in scope study | 91 | |
| Excluded | E1 Not humans | 205 |
| E2 No therapeutic GLP-1 receptor agonist containing drug | 621 | |
| E3 Not a primary study or systematic review | 1,201 | |
| E4 Ancillary document of an out of scope record | 5 | |
| Deferred | D1 Source unreadable or not the article | 4 |
| D2 Eligibility unresolved after the full text | 6 | |
| Total | 6,782 | |
Blind agreement
A stratified random sample of 300 records was screened again by the coordinator and by GPT-6 Astra, each blind to the earlier decisions. Agreement with the final decision is not independent of the coordinator, who also set the rulings. A further seeded sample of 300 records is kept for audit by a human reviewer.
| Reviewer | Compared with | Scale | Records | Agreement | Kappa |
|---|---|---|---|---|---|
| Coordinator (blind) | First pass | Six decisions | 383 | 90.9% | 0.89 |
| Coordinator (blind) | First pass | Included or not | 383 | 91.9% | 0.84 |
| GPT-6 Astra (blind) | First pass | Six decisions | 383 | 89.3% | 0.87 |
| GPT-6 Astra (blind) | First pass | Included or not | 383 | 89.8% | 0.79 |
| Coordinator (blind) | GPT-6 Astra (blind) | Six decisions | 383 | 91.4% | 0.89 |
| Coordinator (blind) | GPT-6 Astra (blind) | Included or not | 383 | 92.7% | 0.85 |
| Coordinator (blind) | Final decision | Six decisions | 383 | 94.5% | 0.93 |
| Coordinator (blind) | Final decision | Included or not | 383 | 95.0% | 0.90 |
| GPT-6 Astra (blind) | Final decision | Six decisions | 383 | 94.8% | 0.93 |
| GPT-6 Astra (blind) | Final decision | Included or not | 383 | 95.6% | 0.91 |
Every decision
Low confidence decisions name the alternative in the note; every one was re-examined by the coordinator. The full history is in fulltext_decisions.csv.
Loading data…