---
title: "Validating a shared definition across sites"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Validating a shared definition across sites}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment  = "#>",
  fig.width  = 7,
  fig.height = 4
)
```

This vignette illustrates how two sites that cannot share patient-level data, can still be used to derive and validate a simplified PTSD definition.

## The multi-site problem

Collaborations across clinics, registries, or countries are often bound by privacy rules and data-use agreements that forbid moving patient records between sites. Hence, pooling data is often not possible. However, as a simplified definition is fully described by the symptom indices it uses and the number required, optimization can take place at one site and validation at another without sharing the original data, only the symptom sets constituting the new definitions.

In this vignette, one site has data from the veterans with a high PTSD prevalence and another has population data with a low PTSD prevalence. The first site derives a definition and inspects which symptoms its optimization selects. It shares the definitions as a JSON file. The general-population site then applies that definition locally, and we compare how it performs in each sample.

## Requirements for the input data

Each site standardizes its own data with `rename_ptsd_columns()`: the 20 PCL-5 items in their standard order, scored 0 to 4, with no missing values, plus any identifier columns named in `id_col`. The full contract is in the [Getting started](getting-started.html) vignette. For this vignette, we subset the veteran sample to 120 rows to keep the optimization fast.

## Site 1: deriving the definition in the veteran sample

The veteran site optimizes on its own data. `compare_optimizations()` runs the three default rules and returns one object holding the results; nothing patient-level leaves the site.

```{r derive, message = FALSE}
library(PTSDdiag)
library(dplyr)

data("simulated_ptsd")
vet <- rename_ptsd_columns(simulated_ptsd[1:120, ],
                           id_col = c("patient_id", "age", "sex"))

comp_vet <- compare_optimizations(
  vet,
  n_top         = 10,
  score_by      = "balanced_accuracy",
  show_progress = FALSE
)
```

Before sharing anything, the site can look at which symptoms its optimization keeps selecting. `plot_symptom_frequency()` shows, for each rule, how often each of the 20 symptoms appears among the top combinations, with a pooled OVERALL row.

```{r figure, fig.alt = "Heatmap of PCL-5 symptom selection frequency in the veteran sample"}
plot_symptom_frequency(comp_vet, type = "relative")
```

The symptoms whose tiles are dark across the rules are the core symptoms the derivation settles on. These are stable features of the veteran sample, and the question for the collaboration is whether a definition built from them also works elsewhere.

## Sharing the definitions without sharing data

Rather than commit to a single winner, the site carries forward the top five combinations from each of the three rules, fifteen candidate definitions in all. This is closer to real practice, where a collaboration weighs several promising definitions rather than betting on one. `extract_definitions()` pulls them straight from the comparison object: it keeps the top `n` combinations of each rule and reads, for each, how many symptoms must be present and whether all four clusters are required, so the researcher chooses only how many to carry.

```{r definitions}
definitions <- extract_definitions(comp_vet, n = 5)

# The shared object: only symptom numbers and the rule to apply them
lapply(definitions, function(d) d$symptoms)
```

This object is everything the other site needs, and it holds nothing but symptom numbers and rules. No participant, score, or demographic is in it, so it can be shared freely.

In a real collaboration the definitions travel as files rather than as an R object. `write_combinations()` serializes one rule's combinations to a small JSON file; the optional `label` stores the rule's display name in the file, so tables produced at the receiving site are labelled by the derivation site automatically. A temporary file stands in for the transfer here.

```{r share-json}
json_file <- tempfile(fileext = ".json")
write_combinations(
  definitions[["4/6 Hierarchical"]]$symptoms, json_file,
  n_required  = 4,
  clusters    = list(B = 1:5, C = 6:7, D = 8:14, E = 15:20),
  label       = "4/6 Hierarchical",
  description = "Top 5 hierarchical combinations, veteran derivation sample"
)
```

At the other end, `read_combinations()` imports the file and `as_definitions()` turns one or several imported files into the same definitions structure `extract_definitions()` produces — `lapply(files, read_combinations)` handles a whole folder of rules in one line. The round trip reproduces the shared definition exactly:

```{r json-roundtrip}
received <- as_definitions(read_combinations(json_file))

all.equal(received[["4/6 Hierarchical"]]$symptoms,
          definitions[["4/6 Hierarchical"]]$symptoms)
```

## Evaluating every definition in a sample

Alongside the fifteen derived definitions we include ICD-11 as a fixed published benchmark. ICD-11 is the same rule everywhere, so each site computes it locally and nothing about it is shared. `evaluate_definitions()` applies each shared definition with its own rule, adds ICD-11, and scores all of them against the sample's full DSM-5-TR diagnosis. Because it needs only the definitions and a data frame, the identical call runs at either site.

## Performance in the derivation sample

The veteran site records how every definition performs on its own patients, the baseline the other site will be read against.

```{r perf-derivation}
evaluate_definitions(vet, definitions, include_icd11 = TRUE)
```

The first row is the full DSM-5-TR diagnosis itself, the reference. Below it are the fifteen derived definitions, grouped by rule, followed by ICD-11, each scored against the reference.

## Validating in the general-population sample

The general-population site receives only the definitions printed above. It runs the same evaluation on its own patients. The veteran site never sees these records, and this site never sees the veteran records.

```{r perf-validation}
data("simulated_ptsd_genpop")

# simulated_ptsd_genpop also carries paired CAPS-5 columns (C1..C20); here we
# use only the PCL-5 items, so we select those before standardising.
genpop <- rename_ptsd_columns(
  simulated_ptsd_genpop[, c("patient_id", "age", "sex", paste0("S", 1:20))],
  id_col = c("patient_id", "age", "sex")
)

evaluate_definitions(genpop, definitions, include_icd11 = TRUE)
```

Both tables are built the same way, so they line up row for row with the derivation table above.

## A tidy table for downstream analysis

The tables above are formatted for reading. For filtering, plotting, or combining results across sites, request the same evaluation as a plain analysis table with `tidy = TRUE`: one row per combination with `Approach`, `Rank`, `Combination`, the 2x2 counts, and numeric metrics. The layout matches `summarize_top_combinations()` on the derivation side, so tables from both sites stack with `rbind()` — no label parsing required.

```{r tidy-table}
tidy_gp <- evaluate_definitions(genpop, definitions, tidy = TRUE)
head(tidy_gp)
```

## Reading the two tables together

Each definition keeps its character across the two samples: the hierarchical rule is the most specific and the least sensitive, the three-of-six rule the most sensitive and the least specific, and the four-of-six and ICD-11 rules fall between them. What changes with the sample is the absolute level of performance. The positive predictive value of every definition falls in the lower-prevalence community sample, so a positive result there is less likely to mark a true case than in the clinic. Sensitivity also drops, most visibly for the strict hierarchical rule, because the community sample's symptom profiles are milder, while specificity holds or rises. These shifts are exactly what external validation is meant to reveal, and they are why a definition should not be transported across settings on its derivation performance alone.

The agreement on core symptoms in the figure and the performance of every definition in the two tables were obtained without moving a single patient record between the sites. In a full collaboration the second site would also optimize, and the two sites would combine their selections into a cross-site consensus; the [Validating abbreviated symptom definitions](validation.html) vignette covers the external-validation logic and the JSON mechanism in more depth.

## See also

- [Getting started](getting-started.html) for the single-cohort derivation workflow.
- [Comparing diagnostic criteria](comparing-criteria.html) for the symptom-frequency tools and multi-rule comparison.
- [Validating abbreviated symptom definitions](validation.html) for internal and external validation.
- [CAPS-5 workflow](caps5-workflow.html) for validation against a clinician-administered reference.