Real world evidence got more usable and no less hard

The past week did not produce a neat breakthrough. It reinforced the same pattern in real world evidence, real world data, post market analytics, and health economics: more access, more guidance, and more pressure to prove that messy external data can support decisions without pretending it is clean trial data.

What changed

The regulatory direction kept moving toward broader use of real world data across the product lifecycle, including postmarket safety and effectiveness work, while still insisting that relevance, reliability, and documentation come first. FDA has also signaled a more permissive stance for some device submissions by removing a barrier to using de identified source data, which should make large claims, registry, and health system datasets easier to bring into submissions, but not easier to trust without work.

That matters because the value proposition has always been conditional. Real world evidence can complement trials, help with safety signals, long term outcomes, reimbursement, and uptake, but only if the data can be tied to a question that is narrow enough to answer and rigorous enough to defend.

Where the friction sits

The hardest part is still cohort definition. In real world data, the question is rarely whether there is enough data, it is whether the population actually matches the decision being made, whether the index event is defined consistently, and whether exclusions are doing hidden damage to external validity.

Missingness is the second trap. Real world datasets are often large but uneven, with laboratory values, vitals, site level fields, and outcomes missing in ways that are not random. That makes a dashboard look complete while the underlying analytic cohort quietly shifts under the analyst’s feet, especially when the missingness correlates with site, payer type, disease severity, or care setting.

Site bias is often stronger than teams admit. A system can appear nationally representative while still reflecting the coding habits, referral patterns, and follow up behavior of a narrow set of institutions. FDA’s focus on relevance and reliability is a direct warning against confusing dataset size with representativeness.

Interoperability is still a practical bottleneck, not a slogan. The technical problem is not only mapping vocabularies, but preserving provenance, time ordering, versioning, and study definitions across claims, EHR, registry, and device or payer feeds. That is why the useful work sits in pipeline design, data validation, and protocol enforcement rather than in the presentation layer.

Why adoption is hard

Adoption is hard because the decision threshold is higher than the analytics threshold. A team can build a polished dashboard quickly, but a decision that changes a trial design, label strategy, or market access plan needs traceable definitions, reproducible extraction logic, and an explicit account of how bias and uncertainty were handled.

That gap is where many programs stall. Data teams can ingest, normalize, and visualize, but operationalizing RWE means building controls around data provenance, consistency checks, cohort recreation, and protocol drift. FDA’s emphasis on documentation of data sources, study protocols, study element definitions, and quality controls points to the same operational reality.

The most common failure is mistaking statistical confidence for truth. A model or survival curve can look precise even when the cohort is unstable, the endpoint is proxy based, or the confounding structure is unresolved. In that case, the output is not evidence so much as a confident summary of the assumptions already baked into the pipeline.

What useful work looks like

The useful work is not broad data lake rhetoric. It is smaller and less glamorous:

It is making cohort definitions executable and versioned so the same population can be rebuilt later.

It is tracking missingness by field, site, and time so the team knows whether a result is data rich or simply under observed.

It is building comparability checks between EHR, claims, registry, and payer data before analysis starts, not after the result is already public.

It is preserving lineage from source to extract to analysis set so an auditor, regulator, or payer reviewer can see where the evidence came from.

It is also deciding when the dataset is too biased for the question. That is a useful answer, even if it is not a positive one.

What to watch next

The direction of travel is clear. Regulators are asking for more real world evidence, not less, and they are being more explicit about the standards that make it credible. The next step is less about collecting more data and more about whether life sciences teams can turn external data into governed, reproducible pipelines that survive contact with regulatory, access, and safety decisions.

The teams that get this right will be the ones that treat RWE as an engineering and methods problem first, and a storytelling problem last. If you are seeing the same gap between nice outputs and defensible decisions, it is worth comparing notes with peers who are living inside that friction too.