Benchmark
March 2026
Agentic Clinical Trial Matching
Quantitative Benchmark vs. TrialGPT — Nature Communications 2024 Baseline
Health Universe's agentic pipeline matches the discriminative ability of TrialGPT (ROC AUC = 0.736, identical) while delivering higher precision, fewer false positives, and better clinical reasoning — the metrics that matter for real-world deployment.
0.736
ROC AUC (both arms identical)
69%
Win rate on divergent cases
+27%
PR AUC improvement
89.9%
Concordance at match threshold
0.40
Precision (vs. 0.33 baseline)
82%
Accuracy (vs. 76% baseline)
Why This Matters
TrialGPT (Jin et al., Nature Communications 2024) is the published state-of-the-art for LLM-based patient-to-trial matching. We benchmarked our agentic oncology pipeline head-to-head on the same patient cohort and trial corpus used in the original study. The result: equivalent ranking accuracy with meaningfully better precision and clinical reasoning quality.
Head-to-Head Comparison
| Metric | TrialGPT (Baseline) | Health Universe (Agentic) |
|---|---|---|
| ROC AUC | 0.736 | 0.736 |
| PR AUC | 0.30 | 0.38 (+27%) |
| Precision | 0.33 | 0.40 (+21%) |
| Accuracy | 76% | 82% (+8%) |
| Divergent Case Win Rate | 31% | 69% |
| False Positives (per 176 pairs) | 33 (19%) | 18 (10%) |
| Strong Match False Positives | 158 | 100 (−37%) |
What Our Pipeline Does Differently
TrialGPT matches raw patient text directly against trial eligibility criteria. Our agentic approach introduces a structured summarization step: an oncology-specific AI pipeline first extracts a Patient Fact Sheet — a standardized, category-organized representation of the patient's clinical profile — before matching against trials.
This means every matching decision is grounded in an auditable, structured clinical document. Clinicians can see exactly what information the system used, and correct it before results are generated. The fact sheet layer adds interpretability and auditability without degrading — and in key metrics, improving — matching accuracy.
Score Distribution (7,089 Patient-Trial Pairs)
| Match Category | TrialGPT | Health Universe | Delta |
|---|---|---|---|
| Strong (≥ 2.0) | 158 | 100 | −37% |
| Moderate (1.0–2.0) | 726 | 705 | −3% |
| Weak (0.0–1.0) | 2,022 | 2,020 | — |
| Not a match (< 0.0) | 4,172 | 4,264 | +2% |
Our pipeline produces 37% fewer strong matches — not because it misses eligible trials, but because it is more selective. Fewer false-positive strong matches means clinicians spend less time reviewing irrelevant results. At scale, this translates to roughly 15 fewer unnecessary reviews per patient across a 400+ trial corpus.
Clinical Reasoning Quality
An independent LLM-as-judge evaluation (GPT-5) assessed the 15 most divergent cases for clinical reasoning quality. The Health Universe pipeline was preferred in 60% of cases (9/15), with a higher mean clinical soundness score (3.0 vs. 2.5 on a 1–5 scale).
Representative examples of superior reasoning:
01 — Age restriction detection
Correctly flagged a 75-year-old as ineligible for a pediatric trial that TrialGPT scored as strongly eligible.
02 — Criteria logic
Correctly interpreted an inclusion criterion that TrialGPT inverted into an exclusion, recognizing patient eligibility.
03 — Missing data handling
Applied clinical caution when required diagnoses were absent, while TrialGPT assumed eligibility by default.
Study Design
Two-arm controlled evaluation using 17 cancer patients and 417 cancer-related trials from the original TrialGPT datasets (SIGIR 2016, TREC CT 2021, TREC CT 2022). Both arms use GPT-4o for matching. Same data in, same trials — the only variable is the patient representation method.
Key Takeaway for Stakeholders
Health Universe's agentic pipeline is benchmarked against the published state-of-the-art. It achieves equivalent discriminative accuracy (ROC AUC = 0.736) with meaningfully better precision (+21%), accuracy (+8%), and clinical reasoning quality. The structured fact sheet architecture adds the interpretability and auditability that clinical deployment demands — without sacrificing performance. This is the approach that scales.
Ground truth labels cover 176 of 7,089 patient-trial pairs (2.5%). Classification and PR AUC metrics are computed on the ground-truth subset; agreement and score distribution metrics use the full dataset. All metrics are reported on a 17-patient oncology subset and are not directly comparable to the full 183-patient TrialGPT results. Reference: Jin et al., Nature Communications 2024 (DOI: 10.1038/s41467-024-53081-z). ©2026 Health Universe, Inc. Confidential. All rights reserved.