Protocol benchmark
What 1,007 protocols reveal about ambiguity and inconsistency.
HumanTrue surfaced 20,450 candidates in a first-pass integrity review. The public analysis focuses on the ambiguity and inconsistency that remain meaningful after review, with every finding linked to its source evidence.
Read the result correctly
20,450 candidates are the beginning of the analysis.
The first pass finds places where protocol language may conflict, depend on unstated context, or require interpretation. HumanTrue then reviews the evidence and analyzes retained ambiguity and inconsistency by priority, domain, therapeutic area, and portfolio.
The two integrity signals in this report
- Inconsistency: two instructions, definitions, values, or rules cannot both be followed as written.
- Ambiguity: the intended meaning can be resolved only by finding separated context or making an assumption.
- Priority reflects potential impact if the issue is not resolved.
- Every retained finding remains connected to the passages that support it.
Priority by therapeutic area
Complex study designs surface more findings.
Oncology has the largest absolute total. Hematology is fractionally higher per protocol among therapeutic areas represented by at least 20 protocols. This reflects study volume and design complexity, not therapeutic-area quality.
| Therapeutic area | Protocols | High priority | Clarify | Consider | Total |
|---|---|---|---|---|---|
| Oncology | 222 | 1037 | 2080 | 319 | 3436 |
| Infectious diseases | 169 | 639 | 1069 | 219 | 1927 |
| Immunology | 156 | 570 | 1056 | 176 | 1802 |
| Endocrinology / metabolism | 90 | 353 | 581 | 121 | 1055 |
| Neurology | 54 | 202 | 382 | 75 | 659 |
| Hematology | 38 | 197 | 333 | 59 | 589 |
Real findings
What inconsistency and ambiguity look like in practice.
These examples come from the analysis dashboard's All Findings view. Each includes the finding, review rationale, and referenced source snippets.
inconsistency
Conflicting treatment assignment
The synopsis reassigns additional Group A2 participants to Group A1 for 13-PCV. Section 3.1 keeps the same participants in Group A2 for influenza-vaccine visits in the next season.
- Synopsis
- Reassign to A1
- Section 3.1
- Remain in A2
ambiguity
Competing primary endpoint structures
The summary blends UAS7 with ISS7 and HSS7. Tables 2-1 and 2-2 instead define separate primary endpoint scenarios, requiring the reader to connect separated context.
- Protocol summary
- Blended objective
- Tables 2-1 and 2-2
- Separate scenarios
All Findings
The conclusion never stands alone.
The dashboard's All Findings tab makes every candidate inspectable rather than reducing the work to aggregate charts.
Source evidence
Protocol identifier, exact section or table, quoted passages, and referenced snippets.
Review reasoning
Why the passages conflict or require interpretation, with priority, classification, confidence, and rationale.
Governed resolution
The sponsor's approved interpretation can be versioned and reused without losing the path back to the source.
Method boundaries
A pattern report, not a sponsor scorecard.
- Public protocol benchmark
- A descriptive reference dataset of 1,007 publicly available protocols from completed Phase 1-4 studies, reviewed using a consistent classification and adjudication process. It supports corpus-level comparisons, not performance rankings or detection-accuracy claims.
- First-pass review
- The 20,450 total describes candidates surfaced for investigation. Retained findings drive the pattern analysis.
- Complexity context
- Counts vary with protocol volume, phase, therapeutic area, document age, amendments, and study design complexity.
- Responsible attribution
- Public protocols do not establish whether an issue originated with a sponsor, CRO, vendor, template, or amendment. Attribution requires provenance evidence.
- Versioning
- Results can change as rules, schemas, review methods, and the protocol corpus evolve. Detailed evaluations identify the report version used.
Evaluate your portfolio