How we work

Share models, not data

The code travels. Patient data stay put. A model and its analysis plan are packaged in a container that each participating institution runs on its own patients, behind its own firewall. What comes back is a set of summary statistics, never patient-level values. This removes the privacy, contract and ownership barriers at once, because no patient data ever leave an institution.

Pre-register the analysis

The question is locked before anyone touches the data. A statistical analysis plan states the clinical question, the cohort definitions, the endpoints, the model exactly as published, and the performance measures. It is circulated to all sites, revised, and then locked and version-pinned. Each site runs the locked code; nothing is tuned after the results are seen. Reporting follows the TRIPOD statement.^5,6

At least three independent institutions

A validation counts only when it has been run at three or more institutions that had no part in developing the model. The developer's own cohort may be included as a fourth, but it is reported separately and never substitutes for an independent one. Three independent cohorts are the minimum for telling a model that generalises from one that happened to fit a single population.

Open analysis code

Analyses are automated and re-runnable: the same container at every site, so there is no dependency drift between institutions, with Jupyter notebooks and R scripts that any group can inspect and execute. A single interface runs the whole analysis plan. The repository for each validation is public, so the analysis can be audited and repeated by anyone. An example layout:

validate-models/
├── analysis_plan.md
├── notebooks/
│   ├── xerostomia_validation.ipynb
│   └── lymphopenia_validation.ipynb
├── R/
│   └── calibration.R
├── containers/
│   └── pycerr-segmentation.def
└── README.md

Aggregated results only

Sites return summary statistics, not patient records: performance measures, calibration curves and cohort descriptions, with small cells suppressed so that no value could identify a patient. Pooled analyses are run on these aggregates.

Discrimination and calibration, always both

Discrimination asks whether the model ranks patients in the right order, measured by the AUC or C-statistic. Useless rankings mean no clinical benefit. Calibration asks whether the model's probabilities are numerically correct, measured by calibration-in-the-large and the calibration slope. Miscalibrated probabilities mean wrong treatment thresholds. Every VALIDATE report includes both, because calibration drift between populations and treatment eras is the most common way a published model fails.

Inform, do not prescribe

Guidelines are not the consortium's to write. VALIDATE produces pre-registered, multi-cohort evidence about a published model and passes it to the professional societies. ESTRO, ASTRO and others consider that evidence in their own formal processes; recommendations are adopted into clinical pathways and planning standards from there; and clinicians use a model where the evidence supports it, documenting deviations where it does not.