Reproducible Research with RTpipeline¶

A guide to ensuring methodological rigor and reproducibility in radiotherapy data science

This document provides guidance for researchers—particularly PhD students and postdocs—on how to leverage RTpipeline's features to produce reproducible, publication-ready analyses.

Why Reproducibility Matters¶

In radiotherapy research, reproducibility failures often stem from:

Undocumented preprocessing - DVH interpolation methods, dose grid resolutions, and structure definitions are not specified
Environment drift - Package versions change between analysis runs
Configuration ambiguity - Key parameters (e.g., radiomics bin width) are buried in code
Data provenance gaps - No clear link between raw DICOM and final features

RTpipeline addresses these by encoding all preprocessing decisions in version-controlled configuration files and providing complete audit trails.

The Reproducibility Checklist¶

Use this checklist before submitting a manuscript or thesis:

Essential Elements¶

[ ] RTpipeline version recorded (git commit hash or release tag)
[ ] Complete config.yaml saved with results
[ ] Conda environment exported (conda env export > environment.yml)
[ ] QC reports reviewed and saved
[ ] Structure mapping dictionary documented
[ ] Random seeds fixed (where applicable)

For Radiomics Studies¶

[ ] PyRadiomics parameters documented (radiomics_params.yaml)
[ ] IBSI feature definitions referenced
[ ] Robustness analysis (ICC, CoV) completed
[ ] Perturbation chain parameters recorded

For Multi-Center Studies¶

[ ] Shared configuration bundle created and versioned
[ ] All centers record the same released version or immutable commit
[ ] Structure mapping harmonized across sites
[ ] QC criteria applied consistently

Configuration as Protocol¶

The Key Principle¶

Your config.yaml IS your preprocessing protocol. It should be:

Version-controlled (git)
Published as supplementary material
Citable (via DOI if possible)

Example Configuration Header¶

# RTpipeline Configuration for Prostate Toxicity Study
# Protocol Version: 1.2
# Date: 2025-01-15
# DOI: 10.5281/zenodo.XXXXXXX
#
# This configuration defines the complete preprocessing pipeline
# for the PROTECT-01 multi-center prostate radiotherapy study.
#
# Modifications from v1.1:
# - Updated ICC threshold from 0.75 to 0.90 per reviewer feedback
# - Added V75Gy metric for rectal dose-response analysis

version: "1.2"
study_id: "PROTECT-01"

Key Configuration Sections to Document¶

# Structure harmonization
structure_mapping:
  RECTUM:
    patterns: ["Rectum*", "RECT*", "rectum*"]
    required: true
    # Documentation: Maps all local rectum names to canonical RECTUM

# DVH computation parameters
dvh:
  interpolation: "linear"  # Method for DVH curve interpolation
  dose_units: "Gy"         # Absolute dose units
  volume_type: "relative"  # Relative (%) or absolute (cc)
  metrics:
    - Dmean
    - Dmax
    - D2cc
    - V70Gy

# Radiomics extraction
radiomics:
  binWidth: 25              # Fixed bin width in HU; report this project choice
  resampling_mm: [1, 1, 3]  # Isotropic x/y, 3mm z
  interpolator: "sitkBSpline"

# Robustness assessment
radiomics_robustness:
  enabled: true
  thresholds:
    icc:
      robust: 0.90      # Features with ICC >= 0.90 classified as robust
      acceptable: 0.75
    cov:
      robust_pct: 10.0  # Features with CoV <= 10% classified as robust

Environment Reproducibility¶

Dual Environment Architecture¶

RTpipeline uses two conda environments to resolve dependency conflicts:

Environment	Purpose	Key Dependencies
`rtpipeline`	Main processing	NumPy 2.x, TotalSegmentator, SimpleITK
`rtpipeline-radiomics`	Radiomics extraction	NumPy 1.x, PyRadiomics, `opencv-python-headless==4.8.1.78`, `rt-utils==1.2.7` (`--no-deps`)

RTSTRUCT rasterization¶

Radiomics and robustness convert RTSTRUCT contours to CT-grid masks through rt-utils/cv2.fillPoly. Reproducible runs require one OpenCV provider: opencv-python-headless==4.8.1.78 in the isolated NumPy 1.26 radiomics runtime. Install rt-utils==1.2.7 separately with python -m pip install --no-deps "rt-utils==1.2.7" after creating either radiomics YAML environment. NumPy, pydicom and dataclasses (declared by rt-utils) are explicit dependencies. Docker and scripts/install_local.sh perform this step. The Apptainer definition converts the Docker image published by this repository’s release workflow. Its mandatory %post environment check fails the build if the radiomics interpreter or pinned dependencies are missing or incompatible. Do not install RTpipeline with dependency resolution in this helper environment: use the source through PYTHONPATH or install the package with --no-deps.

RT-utils declares the GUI opencv-python distribution even though it uses only cv2. Therefore ordinary pip check reports that intentionally absent provider. Run PYTHONPATH=. python scripts/check_radiomics_environment.py in the radiomics environment; it validates the active rasterizer and permits only that one known metadata mismatch. Other dependency errors remain fatal. Third-party metadata is retained unchanged. The local installer removes competing provider wheels before reinstalling headless, because their installed files overlap.

Before radiomics RTSTRUCT mask preparation, the runtime verifies rt-utils 1.2.7, OpenCV 4.8.1, the sole pinned provider and the exact 23-pixel reference-triangle mask with vertices (8,9), (8,13), (14,5). OpenCV 4.11.0 fills 21 pixels on this example. A mismatch raises an error; there is no bypass. Launch radiomics and robustness in the radiomics runtime. CT calls from NumPy 2 re-execute the whole course driver in the isolated radiomics interpreter, resolved through RTPIPELINE_RADIOMICS_ENV_PREFIX or the existing environment discovery. A missing interpreter fails before course processing. NIfTI mask loading does not fill polygons.

The main NumPy 2 runtime pins opencv-python==5.0.0.93, matching the production main runtime. Main stages record imported OpenCV version, provider distribution name and version, module path, and probe errors in their existing provenance or completion records. They do not impose the radiomics 4.8.1 guard. Its RTSTRUCT utilities fill masks for custom structures, ROI repair, QC, review images and DVH crop labels. Its RTSTRUCT writers use OpenCV contour tracing. These paths retain their existing algorithms and do not establish equivalence to the radiomics mask convention.

Python environment launches set PYTHONNOUSERSITE=1, while retaining the existing PYTHONPATH and multiprocessing socket hook. Radiomics tables and completion receipts record rasterizer_backend, rasterizer_opencv_version, rasterizer_opencv_path, rasterizer_rt_utils_version, rasterizer_providers, and rasterizer_probe_error. Robustness tables and source-disposition sidecars also record these identities. The environment fingerprint includes the imported cv2 version, provider metadata, rt-utils version and reference mask; the absolute module path is recorded separately. Polygon rows are reusable only with the recorded single headless pin and rt-utils 1.2.7. Missing or different identities invalidate resume, checkpoints, publication, completion receipts, partial refresh, and cohort aggregation. Historical polygon outputs without the pinned rasterizer identity are not republished by this release. NIfTI and DICOM-SEG rows use distinct no-polygon-fill backend labels and do not require the polygon pin for reuse. These records describe the runtime used for the measurement; they do not establish clinical accuracy.

Custom structures are derived only in the main runtime. Its locked builder publishes RS_custom.dcm and records OpenCV 5.0.0 in rs_custom_meta.json. Radiomics and robustness consume that shared polygon artifact only when the code, configuration, source identities and main-runtime builder record are current. They never rebuild it. Missing, stale or legacy custom structures fail closed when required for extraction. The workflow orders radiomics after DVH, which builds the custom structures; robustness follows radiomics. The shared polygons are filled with OpenCV 4.8.1 for radiomics measurement. Derivation and final measurement rasterization therefore have separate, explicit runtime identities.

Exporting Environments¶

# Export main environment
conda activate rtpipeline
conda env export --no-builds > rtpipeline-environment.yml

# Export radiomics environment
conda activate rtpipeline-radiomics
conda env export --no-builds > rtpipeline-radiomics-environment.yml

Docker for Complete Reproducibility¶

For maximum reproducibility, use the Docker image with a specific version tag and record its digest.

# Use a specific version, not :latest
docker pull kstawiski/rtpipeline:3.0.0

# Record the image digest
docker inspect --format='{{.RepoDigests}}' kstawiski/rtpipeline:3.0.0

Document in your methods:

"All preprocessing was performed using RTpipeline Docker image kstawiski/rtpipeline:3.0.0 (SHA256: abc123...)."

Also record whether radiomics and robustness were executed through the default dual-environment setup (rtpipeline + rtpipeline-radiomics) or a custom local installation. When structures come from RTsegmentator, keep each course's Segmentation_CustomModels/rtseg_<model>/manifest.json and report the RTsegmentator version, task names and model references it records.

Methods Section Templates¶

Template 1: DVH Extraction¶

**Dose-Volume Histogram Analysis**

DVH metrics were extracted using RTpipeline (version 3.0.0; DOI: [release DOI]) [1].
DICOM RTSTRUCT, RTDOSE, and RTPLAN files were exported from [TPS name/version]
and processed using the following standardized protocol:

1. **Structure Harmonization:** Contour names were mapped to canonical nomenclature
   using a predefined dictionary (Supplementary Table S1). Structures named
   [list variations] were mapped to the canonical label [STRUCTURE_NAME].

2. **DVH Computation:** Cumulative DVH curves were computed using [interpolation
   method] interpolation with a dose resolution of [X] Gy. The following metrics
   were derived:
   - Mean dose (D_mean)
   - Maximum dose (D_max)
   - Dose to hottest 2cc (D_2cc)
   - Volume receiving ≥X Gy (V_XGy)

3. **Quality Control:** Cases with [QC criteria] were flagged and reviewed.
   [N] cases were excluded due to [reasons].

The complete preprocessing configuration is available as Supplementary File S2.

[1] RTpipeline software citation (see repository `CITATION.cff`).

Template 2: Radiomics with Robustness Assessment¶

**Radiomic Feature Extraction and Stability Assessment**

CT radiomic features were extracted using RTpipeline (version 3.0.0; DOI: [release DOI]) [1] with
PyRadiomics (version 3.0.1) [2] following Image Biomarker Standardisation
Initiative (IBSI) recommendations [3].

**Preprocessing:**
- Images were resampled to [X × X × X mm] isotropic voxels using
  [interpolation method]
- Intensity discretization used a fixed bin width of [X] HU
- [CT cropping method, if applicable]

**Feature Extraction:**
A total of [N] features were extracted from [structure(s)], comprising:
- Shape features (N = X)
- First-order statistics (N = X)
- Texture features: GLCM (N = X), GLRLM (N = X), GLSZM (N = X), GLDM (N = X)
- [Filtered features if applicable]

**Robustness Assessment:**
Feature stability was assessed using an RTpipeline-adapted NTCV chain inspired
by Zwanenburg et al. [4], not an exact reimplementation. For each ROI, [N]
perturbations were required:
- Gaussian noise injection (σ = 0, 10, 20 HU)
- Rigid translations (up to +/-4 mm)
- Reproducible random physical-space contour offsets (2 realizations)
- Distance-ranked volume adaptation to the closest voxel counts representing ±15%

Intraclass correlation coefficients ICC(3,1) were computed using Pingouin [5].
CoV was computed separately within each patient/course/ROI/source across
perturbations, then summarized by its cohort median; it was not computed from
pooled between-patient values.
Features were classified as:
- **Robust:** ICC ≥ 0.90 and CoV ≤ 10%
- **Acceptable:** ICC ≥ 0.75 and CoV ≤ 20%
- **Poor:** ICC < 0.75 or CoV > 20%

[N] features ([X]%) met robustness criteria and were retained for modeling.

**References:**
[1] RTpipeline software citation (see repository `CITATION.cff`).
[2] van Griethuysen JJM, et al. Cancer Res. 2017;77(21):e104-e107.
[3] Zwanenburg A, et al. Radiology. 2020;295(2):328-338.
[4] Zwanenburg A, et al. Sci Rep. 2019;9:614.
[5] Vallat R. JOSS. 2018;3(31):1026.

Template 3: Multi-Center Harmonization¶

**Multi-Center Data Harmonization**

Data from [N] centers were processed using RTpipeline ([released version and tag, or immutable post-tag commit/container digest]) [1]. Each center processed local DICOM exports using shared configs and versioned artifacts (Supplementary File S3). These inputs make consistent local processing operationally possible, but do not prove identical processing unless a bound contract verifies it.

**Harmonization Protocol:**
1. **Structure Mapping:** A consortium-wide structure dictionary was defined
   (Supplementary Table S1) mapping site-specific nomenclature to [N] canonical
   structure labels.

2. **Preprocessing Standardization:**
   - CT resampling: [X × X × X mm]
   - Dose grid interpolation: [method]
   - Field-of-view cropping: [region]-based using TotalSegmentator landmarks

3. **Quality Control:**
   - All centers declared the same QC criteria; bound packet checks and site receipts verified the criteria actually used (Supplementary Table S2)
   - [N] cases across all sites were excluded due to [reasons]

4. **Feature Extraction:**
   - DVH metrics: [list]
   - Radiomics features: [list or reference to table]

**Data Governance:**
Raw DICOM data remained at each institution. RTpipeline's current packet workflow supports distributed aggregate radiomics reliability analysis. The packets contain cohort-level reliability statistics, not outcomes or model updates. They do not themselves establish federated model training, secure aggregation, differential privacy, privacy guarantees, or outcome federation. External federated learning requires a separately implemented downstream system.

The RTpipeline configuration bundle is available at [repository or archive URL].

[1] RTpipeline software citation (see repository `CITATION.cff`).

Template 4: Structures from RTsegmentator¶

**Automatic Segmentation**

[Structures] were segmented on the planning CT with [model name] ([model reference])
[2], run through RTsegmentator version [X.Y.Z] [3] (task `[task name]`). RTsegmentator
maps each prediction onto the acquired CT grid and releases an RTSTRUCT only after
checking its image references, physical bounds and a contour-to-mask round trip
(Dice ≥ 0.95). RTpipeline (version 3.0.0) [1] imported each result only after
verifying every packaged file against the result manifest's SHA-256 checksums and
confirming that the RTSTRUCT referenced the course's planning CT series. Automatic
contours were [reviewed by / not edited by] [role] before analysis. [N] courses had
no structure found by the model and were [handled as].

[1] RTpipeline software citation (see repository `CITATION.cff`).
[2] Publication of the segmentation model.
[3] RTsegmentator software citation (see its `CITATION.cff`).

The values for the bracketed fields are recorded per course in Segmentation_CustomModels/rtseg_<model>/manifest.json (RTsegmentator version, task, model reference, licence and limitations) and in metadata/rtsegmentator_imports.json (job ID and outcome for every model, including models that found nothing).

Provenance Tracking¶

RTpipeline Output Structure¶

Each processed course includes provenance metadata:

Output/{PatientID}/{CourseID}/
├── metadata/
│   ├── case_metadata.xlsx       # DICOM tag extraction
│   ├── processing_log.txt       # Complete processing log
│   ├── config_snapshot.yaml     # Config used for this case
│   └── cropping_metadata.json   # CT cropping boundaries
├── qc_reports/
│   └── qc_summary.json          # Quality control flags
└── ...

Linking Outputs to Inputs¶

import pandas as pd

# Load metadata with DICOM provenance
metadata = pd.read_excel("Output/Patient001/Course001/metadata/case_metadata.xlsx")

# Key provenance fields
print("Source DICOM UIDs:")
print(f"  CT Series: {metadata['ct_series_uid'].iloc[0]}")
print(f"  RTSTRUCT: {metadata['rtstruct_sop_uid'].iloc[0]}")
print(f"  RTDOSE: {metadata['dose_sop_uid'].iloc[0]}")
print(f"  RTPLAN: {metadata['plan_sop_uid'].iloc[0]}")

Reproducibility Validation¶

Regression Testing¶

When updating RTpipeline versions or configurations:

# Run on reference dataset
rtpipeline run --config reference_config.yaml --input test_data/ --output test_output_v2/

# Compare outputs
python -c "
import pandas as pd
import numpy as np

v1 = pd.read_excel('test_output_v1/_RESULTS/dvh_metrics.xlsx')
v2 = pd.read_excel('test_output_v2/_RESULTS/dvh_metrics.xlsx')

# Numerical columns
numeric_cols = v1.select_dtypes(include=[np.number]).columns

# Compare
diff = (v1[numeric_cols] - v2[numeric_cols]).abs()
max_diff = diff.max().max()
print(f'Maximum absolute difference: {max_diff}')

if max_diff > 1e-6:
    print('WARNING: Outputs differ!')
    print(diff[diff > 1e-6].dropna(how='all'))
"

Hash Verification¶

For critical analyses, compute checksums:

import hashlib
import pandas as pd

def compute_df_hash(filepath):
    """Compute SHA256 hash of a DataFrame."""
    df = pd.read_excel(filepath)
    return hashlib.sha256(
        pd.util.hash_pandas_object(df).values
    ).hexdigest()

# Record hashes
hashes = {
    "dvh_metrics": compute_df_hash("_RESULTS/dvh_metrics.xlsx"),
    "radiomics_ct": compute_df_hash("_RESULTS/radiomics_ct.xlsx")
}

# Save for reproducibility verification
import json
with open("output_hashes.json", "w") as f:
    json.dump(hashes, f, indent=2)

Publication Checklist¶

Before submitting your manuscript:

Data Availability Statement¶

**Data Availability**

The RTpipeline configuration files used in this study are available at
[DOI/GitHub URL]. Due to [privacy/IRB/GDPR] restrictions, the raw DICOM
data cannot be shared. Aggregated results tables are provided as
Supplementary Data.

Code Availability Statement¶

**Code Availability**

All data preprocessing was performed using RTpipeline version 3.0.0
(https://github.com/kstawiski/rtpipeline; archived release DOI: [release DOI]).
The complete configuration bundle, including structure mapping dictionaries and
radiomics parameters, is available at [repository or archive URL]. Analysis
scripts for statistical modeling are available at [repository URL]. Report the
git commit or container digest as well when the bytes you ran differ from the
release tag.

Supplementary Materials to Include¶

config.yaml - Complete preprocessing configuration
structure_mapping.yaml - Structure name harmonization dictionary
radiomics_params.yaml - PyRadiomics extraction settings
environment.yml - Conda environment specification
qc_criteria.md - Quality control inclusion/exclusion criteria

Best Practices Summary¶

Area	Recommendation
Version Control	Git-track all configuration files
Environment	Use Docker or export conda environments
Configuration	Document all parameters in config files, not code
Provenance	Save config snapshots with each analysis
QC	Review and document all exclusions
Methods	Use templates with specific parameter values
Sharing	Publish config bundles with DOI