Why most feasibility assessments are unfalsifiable

Open almost any feasibility deck and you will find sentences like this one: “Site availability in Western Europe is adequate for the planned enrolment period.” It reads like a finding. It is not. There is no way for the person receiving it to disagree in any specific way, because there is nothing specific to disagree with. Adequate against what? Which sites? Counted when? Counted how?

This is the quiet failure of most trial feasibility work. Not that it is wrong — plenty of it is broadly right — but that it is unfalsifiable. Nobody downstream can locate the point where a judgement was made and test it. So the pack gets read for tone rather than content: who wrote it, how confident they sounded, whether the conclusion matches what the room already believed.

What falsifiability actually requires

Falsifiability sounds like a philosophy-seminar word. In practice it is four boring, mechanical properties. A claim is checkable when a reader who does not trust you can reconstruct it:

1. A named source. Not “public registries” — the register, by name, with its steward. ClinicalTrials.gov is a different thing from the WHO ICTRP, and both are different from a sponsor press release.
2. The exact record. An NCT number, a PMID, an application number, a filing. A source without a record identifier is a genre, not a citation.
3. A retrieval date. Registry records change. A site count read in March and a site count read in September are different measurements of a moving object, and only one of them is the one you shipped.
4. The rule. The step that turned records into the number. “Trials in this condition at this location” is not a rule until you say which statuses counted, which phases, and over what window.

Miss any one of the four and the claim collapses back into an assertion. Most feasibility packs miss three.

Why this happens to good analysts

It is not laziness. It is compression under a deadline. A feasibility question arrives three weeks before an investment committee, and answering it properly means touching a dozen public sources, each with its own query language, update cadence and idea of what a “trial” is. An analyst does the work, holds the reasoning in their head, and writes down the conclusion — because writing down the reasoning would take longer than the reasoning took.

The reasoning then evaporates. Six weeks later someone asks why the pack assumed twelve months of enrolment, and the honest answer is that a person who has since moved teams looked at some registry data and formed a view. The number survives; its justification does not. This is how a working assumption becomes a fact by attrition, and it is the single most common way a feasibility estimate goes into a protocol unexamined.

What a checkable assessment looks like

Concretely: every figure in the pack is attached to the record it came from, and the attachment survives the trip into a slide, a memo, and a board pack. Not a bibliography at the back — a link on the number itself. If the assessment says a city is contested, you should be able to click the count and see the trials, their statuses, and the date they were read.

That has a second-order effect people underrate. Once a number is traceable, it becomes updatable. A registry-derived count that carries its query can be re-run in September and compared with March. An assumption written into a slide cannot be re-run at all; it can only be re-argued.

It also changes the review meeting. When every claim is anchored, the conversation moves off “do we believe this?” and onto the two questions that actually deserve senior time: is this the right rule? and is this source fit for this purpose? Those are arguments worth having. “Does the analyst seem confident” is not.

The uncomfortable corollary

If you commit to falsifiability, you have to publish the gaps too. A cited assessment will have places where the public record simply does not answer the question — where a criterion has no measured retention, where a site has no reported enrolment history, where a comparator trial completed and never posted results. The honest move is to render those as absent rather than to fill them with a plausible default.

That is a harder document to sell internally, because a page with visible holes looks less finished than a page without them. It is also the only kind of page that is worth anything in a bid defence, an IC challenge, or a regulator conversation — because the holes were always there, and the only question was whether the reader would find out from you or from someone else.