A result has to survive the handover

A computational result can look decisive on a screen. Its real test begins when another team tries to use it. They may work with a different population, different instruments, or records collected for another purpose. They may need to support a decision that the original study never considered. Every one of those differences can change the evidence required.

Our position is that the handover deserves to be treated as a research problem. In biodefence, computational analysis may inform public-health assessment or help prioritise investigation. In biotechnology, it may shape the next experiment or a development decision. In either setting, the receiving team needs to know what the result establishes, what remains uncertain, and where its usefulness has been tested.

This changes the standard for progress. A result should come with an account of the observations, assumptions, analytical choices, and intended use that produced it. A visually convincing chart cannot substitute for that record. The ambition should be to make a finding inspectable enough that a capable colleague can challenge it without having to reconstruct the project from memory.

The data may contain the laboratory as well as the biology

Measurements can reflect the circumstances in which they were made. Leek and colleagues described how laboratory conditions, reagent lots, and personnel differences produce batch effects in high-throughput biological data. When those differences line up with the biological question under study, they can support an incorrect conclusion. The problem is especially difficult when technical and biological differences are entangled.

A hypothetical example makes the issue concrete. If two comparison groups were measured under systematically different conditions, apparent separation between them may have more than one explanation. A downstream analyst needs that history to assess the finding. The example is a question about study structure, not evidence that any particular dataset is invalid.

This is why computational work should begin with the provenance of the observations. Who collected them? What changed during collection? Which records share the same underlying source? What was excluded, and why? These details should be recorded before a model is selected. More computing power cannot recover a design decision that was never documented, and a larger dataset can make an unrecognised systematic difference look more convincing.

Reference studies turn reproducibility into something measurable

The FDA-coordinated Sequencing Quality Control project provides a concrete example. Its 2014 consortium study compared RNA sequencing across multiple laboratory sites and platforms using reference material and built-in controls. It found that relative-expression measurements could be accurate and reproducible across sites and platforms under specified filtering conditions. It also found gene-specific biases across the technologies examined and cautioned against treating the methods as accurate measures of absolute expression.

The finding is conditional, and that is its strength. A technology can be useful for one measurement task without supporting every interpretation someone might attach to its output. Reproducibility becomes more informative when the object being reproduced is stated clearly: a relative difference, an absolute quantity, a ranking, or a decision.

Our reading is that shared reference work deserves a larger place in biological research. It gives teams a way to locate disagreements before they make broad claims about biological meaning. A common benchmark is useful when it exposes the conditions under which a method works and the conditions under which it fails. Merely producing a leaderboard would leave the most important questions unanswered.

Publish enough context for another team to work

The FAIR principles describe research information as findable, accessible, interoperable, and reusable. Their original formulation extends to tools and workflows as well as datasets. It calls for persistent identifiers, useful metadata, clear conditions of use, and provenance. It also explicitly accommodates authentication and authorisation. Responsible reuse can include controlled access to sensitive information.

For a global collaboration, that becomes a practical set of expectations. A receiving team should know which version it has, what the fields mean, how an output relates to its inputs, and what uses are permitted. The analysis should identify the software and reference resources that affect the result. These are proposed handover requirements; the exact package will depend on the discipline and the sensitivity of the material.

A downloadable file is only the beginning. If every interpretation requires a private conversation with its creator, the collaboration remains dependent on that person's availability. Good documentation allows the next team to spend its time testing the argument. It also makes disagreement more precise: researchers can distinguish a difference in data, an implementation choice, and a disagreement about interpretation.

Validation must match the intended decision

Clinical prediction offers a well-developed example of the distinction between a promising model and useful evidence. BMJ guidance published in 2024 defines external validation around a relevant dataset outside model development. It recommends assessing discrimination, calibration, and usefulness for decisions, with attention to the intended population and setting. Discrimination concerns how well predictions separate outcomes; calibration concerns how closely predicted risks correspond to what occurs.

These properties answer different questions. A model may rank cases reasonably while assigning probabilities that are poorly suited to the local population. Even a statistically strong result does not establish that acting on it will improve a service. The cost of reviewing alerts, the consequences of missed cases, and the available response all affect the practical judgement.

The broader lesson for computational biology is an inference, not a demand that every field adopt the same validation design. State the intended use first. Then choose an evaluation that could credibly support or reject that use. Reproducing an analysis on the original material and establishing that its conclusion holds in a new setting are different achievements. A report should say which one it demonstrates.

The sepsis evidence shows why evaluation must continue

A 2021 independent study evaluated the original Epic Sepsis Model at Michigan Medicine using 38,455 hospitalisations involving 27,697 adults. It reported a hospitalisation-level area under the receiver operating characteristic curve of 0.63. At the examined threshold, the model missed 67% of sepsis cases while flagging 18% of hospitalisations. These were findings from a particular model version, health system, and evaluation design; they are not a permanent verdict on every subsequent version.

The evidence did change. A 2026 prospective multicentre evaluation of version 2 examined 227,091 inpatient encounters across four US health systems. Encounter-level discrimination ranged from 0.82 to 0.92. The researchers reported improvement alongside institutional variation, low positive predictive values of 0.13–0.26, and a high alert burden. The studies used different settings and definitions, so their headline scores should not be treated as a controlled comparison.

The defensible conclusion is that performance has to be established for the version, population, and intended workflow under consideration. Updated evidence can justify revising an earlier judgement while leaving important deployment questions unresolved. Evaluation is part of maintaining a capability, not a certificate issued once.

Build a record that can support a hard decision

NIH's rigor and reproducibility guidance emphasises the strength of prior evidence, rigorous study design, relevant biological variables, and authentication of key research resources. These concerns belong in the design of a project, while choices can still be changed and weaknesses can be addressed.

We propose a concrete standard for research handover: provide enough information for a new team to explain why it trusts a result, where that trust ends, and what would change its assessment. That includes the intended use, the observations supporting it, important exclusions, uncertainty, unsuccessful checks, and known gaps. The record should make a negative conclusion as legible as a positive one.

This is demanding work. It competes with the pressure to produce a striking finding and move immediately to the next project. But a lab that wants its research to matter beyond its own computers has to invest in the evidence around the result. Biology becomes more useful when findings retain their meaning across teams, settings, and time—and when the conditions of that transfer remain open to examination.

← All writingNext announcement ↗