# VCC-2025 (H1 hESC) trivial-baseline analysis — GlioState own analysis (draft, pseudobulk only)

Run date 2026-10-16. Purpose: show what the cell-eval metrics that need no differential-expression (DE) step reward, using trivial predictors on the public 2025 Virtual Cell Challenge data.

## Data (public, not redistributed here)
Google Cloud bucket `gs://arc-institute-virtual-cell-atlas/virtual-cell-challenge/2025/` (anonymous HTTPS also worked on 2026-10-16: `https://storage.googleapis.com/arc-institute-virtual-cell-atlas/virtual-cell-challenge/2025/<path>`):
`train/adata_Training.h5ad` (150 perturbations, 221,273 cells), `validation/adata_Validation.h5ad` (50 perts, 98,927 cells), `test/adata_Test.h5ad` (100 perts, 170,846 cells); raw integer counts, 18,080 genes, control label `non-targeting`. Arc states the 2025 data (incl. held-out test perturbations) are downloadable from the Virtual Cell Atlas. Check Arc's terms before reuse.

## Reproduce
```
python3 -m venv venv && venv/bin/pip install h5py pandas scipy numpy
for s in Training:train Validation:val Test:test; do venv/bin/python 01_pseudobulk.py adata_${s%%:*}.h5ad ${s##*:}.npz; done   # ~10 min each, CPU only, streams the CSR matrix
venv/bin/python 02_metrics.py . .          # writes results.json (~1 min)
```
Reference code read: github.com/ArcInstitute/cell-eval commit 6928967 (2026-07-27): `metrics/_anndata.py`, `metrics/_impl.py`, `_score.py`, `_baseline.py`, `_pipeline/_runner.py`, `metrics/_de.py`.

## What was computed
Per file: scanpy-default `normalize_total` (median cell total of that file) + `log1p`, mean over cells per target = pseudobulk (cell-eval's "bulk"). Metrics re-implemented from cell-eval source on pseudobulk: MAE, MSE, MAE/MSE of delta, Pearson-delta, PDS (L1, L2, cosine; target gene excluded; 1 − rank/N). Predictors: control mean; mean of the 150 train perturbation means (cell-eval baseline construction); same with control added; cell-weighted train mean; + median train knockdown ratio applied to the target gene; leave-one-out mean of other evaluation perturbations (uses evaluation labels; not a valid predictor); a split-half noise ceiling (cells of each perturbation and of control split at random into two disjoint halves, seed 0; one half is "real", the other "prediction"; no Spearman–Brown correction); scale sensitivity (prediction delta = factor × other-half delta).

## Caveats (read before citing)
- NOT the official leaderboard computation: pseudobulk re-implementation, not `cell-eval run`; DE-based metrics (DES / overlap, precision, AUC, direction match) are NOT computed; no model was evaluated.
- 2025 in-context task on one cell line (H1). Nothing here is a 2026 zero-shot result.
- Normalisation uses each file's own median total; `cell-eval prep` gene filtering was not applied; official evaluation may differ in details.
- Split-half "ceiling" uses ~half the cells per perturbation, so it is a conservative noise ceiling; it is not the Spearman–Brown-corrected `--ceiling` of cell-eval.
- Scale-sensitivity uses noisy copies of the real data, not model outputs.
- Single seed, no confidence intervals; 50 and 100 perturbations only.