Evidence
Reading a trial’s failure: what “why stopped” actually means
A free-text field on a registry record is doing more work in the industry’s models than almost anything else. Most of what it says is not a failure — and “completed” is not the same as “worked”.
Clinical Trial OS · · 4 min read
The argument in short
- Termination is an event, not a verdict. Most terminations are not efficacy failures.
- A trial stopped early for overwhelming benefit is a success wearing a failure’s status.
- “Completed” is an operational fact about the protocol, not a statement about the result.
On a ClinicalTrials.gov record, a terminated study carries a field called whyStopped. It is free text. It is often a single clause typed by a coordinator years ago. And it is, indirectly, one of the most consequential fields in the industry — because a great many probability-of-success models are trained on a label derived from it, and the derivation is usually one line of code that reads: terminated means failed.
It does not. Termination is an event in the operational life of a study. Failure is a statement about a drug. Conflating the two teaches every downstream model that a grant expiring in Ohio is the same kind of evidence as a futility boundary being crossed.
The categories that are actually in there
When you read the field properly, the reasons sort into a small number of genuinely distinct kinds. These are the ten we classify into, and the distinctions between them are the entire point:
| Category | What it means | Evidence about the drug? |
|---|---|---|
| Efficacy failure | Futility, endpoint not met, no benefit over comparator, conditional power too low | Yes — strong |
| Safety | Toxicity, serious adverse events, unfavourable benefit-risk, a DSMB safety recommendation | Yes — strong |
| Efficacy early stop | Stopped early FOR overwhelming benefit; efficacy boundary crossed at a planned interim | Yes — positive |
| Completed early | Enrolment target reached ahead of plan; closed to accrual having hit its number | No — operational success |
| Recruitment | Slow accrual, targets not met, no eligible patients, screen failures | No — feasibility signal |
| Sponsor decision | Portfolio or pipeline prioritisation, programme discontinued, merger or acquisition | No — usually none stated |
| Funding | Grant not renewed, budget cut, financial constraints | No |
| Regulatory | Clinical hold, competent-authority action, IRB or ethics decision | Sometimes — read the text |
| Logistics | Drug supply or manufacturing, site closure, PI departure, pandemic disruption | No |
| Other | Genuinely none of the above, or content-free text | Unknown |
Look at the third column. Two of those ten categories are strong evidence about the molecule. One of them is evidence in the opposite direction to the status it carries. Six are evidence about money, operations and corporate strategy. If you collapse the column into a binary, you have thrown away the only thing that made the field worth reading.
The success that looks like a failure
The most instructive case is efficacy_early_stop. A trial whose data monitoring committee halts it because the treatment arm is so clearly better that continuing would be unethical is recorded, in the registry, as terminated. Read naively it is a failure. It is the strongest possible positive result.
Its mirror image is completed_early: the study hit its enrolment target ahead of schedule and closed to accrual. That is an operational success and says nothing about efficacy either way. Both of these sit in a status field that a careless pipeline reads as bad news.
Censoring, not failure
The statistical word for a trial that stopped for reasons unrelated to the question is censored. A censored observation is not a negative one — it is an absence of information, and it must be handled as an absence. Scoring it as a failure does not merely add noise; it adds biased noise, in the direction of pessimism about anything that is hard to fund or hard to enrol.
Why this is hard to automate — and where we abstain
The field is written by humans under no editorial standard, and some of it is genuinely ambiguous. The shape that defeats a rules engine most often is a sentence with two clauses and no punctuation between them: an assertion that the enrolment target was reached, followed by a percentage of people. “Enrollment target reached 90% of patients enrolled…” — is that ninety per cent of patients who completed the study (an operational success), or ninety per cent of a target that was never met (a recruitment failure)? The two readings are grammatically identical.
We learned the hard way that refining a regular expression until it decides meaning is a losing game — each fix breaks a case the previous fix handled. The rule we settled on is that the classifier is allowed, and expected, to abstain. Ambiguous text returns “other”, which records no vote rather than a coin-flip. A wrong guess corrupts a training label permanently; an abstention costs one row.
“Other” is a real answer here, not a failure; a wrong guess corrupts a probability-of-success training label, an abstention simply records no vote.
“Completed” is not “worked”
The symmetric error is at the other end of the status field. COMPLETED means the protocol ran to its planned conclusion. It does not mean the primary endpoint was met, and it does not mean anyone ever found out — a completed trial may have posted no results, published no paper, and left the record with nothing but a date.
This is why an outcome label needs at least three states, not two. A reported result is one thing. A completed trial with no reported result is a different thing — genuinely unknown, and common enough that treating it as either success or failure will move your base rates. A terminated trial is a third thing whose meaning depends entirely on the text above.
What to ask
When someone shows you a historical success rate, a comparator analysis, or a PoS built on registry outcomes, there is one question worth asking before any other: how did you label the trials that stopped? If the answer is “terminated counted as failure”, you now know the direction of the bias, and roughly which indications it will hurt most — the ones that are hard to enrol and hard to fund.
Where this sits in the product
This classification runs across the registry corpus behind our design and verdict analyses, and every trial we cite carries its own status and stated reason. See the data sources we read and how it works.
Next
Related reading
Method
What “probability of success” can and cannot tell you
Ours is directional, and we say so on the number. Here is why an honest directional read is more useful than a precise-looking one — and what a PoS structurally cannot see.
Operations
Site selection is a data problem
Track record, competition density and the enrolment arithmetic are all measurable from public records — and the most common site-selection mistake is a counting error, not a judgement error.
See a verdict you can actually check.
Send us a protocol — or just a molecule and an indication. We'll return a fully cited feasibility assessment you can trace, line by line, back to public data — yours to defend in a bid, take to your board or investment committee, or hand to a regulator.