{"scope":{"layer":"general","tissue_scope":"unknown","species":"unknown","system":"unknown","evidence_level":"preprint","importance":"","importance_reason":""},"id":"VC0167","type":"preprint","title":"Genetic Perturbation Prediction Models Fail Out-of-Distribution: A Rigorous Benchmark and Systematic Evaluation","authors_or_org":"Tiwari K, Mahajan G, Bellad RP, Dhruw NK, Jha AK.","date":"2026-10-19","year":"2026","venue":"bioRxiv (preprint, not peer-reviewed)","url":"https://doi.org/10.64898/2026.10.16.759604","doi":"10.64898/2026.10.16.759604","pmid":"","brain_tumor_focus":"no","tags":["benchmark","critique","perturbation","perturb-seq","foundation-model"],"tag_slugs":["benchmark","critique","perturbation","perturb-seq","foundation-model"],"tag_labels":["Benchmark & validation","Critique","Perturbation","Perturb-seq","Foundation model"],"tag_labels_zh":["基准评测与验证","批评与反证","扰动","Perturb-seq 扰动测序","基础模型"],"unmapped_tags":[],"one_line_summary_en":"Preprint (not peer-reviewed): Tiwari et al. (Elucidata Corporation) introduce EL-PERTURB (Extrapolative Learning for PERTURBation Prediction), a benchmark that tests 20 perturbation-prediction models from six architecture classes on essential-gene and genome-wide Perturb-seq screens, using a stratified out-of-distribution split that places train and test sets in functionally distinct regions of transcriptional space. The authors report that performance degrades substantially under OOD conditions and that architectural complexity, including single-cell foundation-model embeddings, gives no reliable advantage over well-tuned simpler baselines. Company-authored; no independent replication yet.","one_line_summary_zh":"预印本（未经同行评审）：Tiwari 等（Elucidata 公司）提出 EL-PERTURB（Extrapolative Learning for PERTURBation Prediction）基准，在必需基因与全基因组 Perturb-seq 筛选数据上测试 6 类架构共 20 个扰动预测模型，采用分层分布外（OOD）划分，使训练集与测试集落在转录空间中功能不同的区域。作者报告：OOD 条件下模型性能显著下降，架构复杂度（包括单细胞基础模型嵌入）相对调优良好的简单基线没有稳定优势。为企业作者的研究，尚无独立复现。","verified":"yes","verification_source":"bioRxiv API details (title, 5 authors, posted 2026-10-19, v1, new results, systems biology, CC BY, corresponding institution Elucidata Corporation, abstract) + Crossref posted-content record fetched 2026-10-21; full text not read; KB id VC0167 (renumbered 2026-10-20)","page_url":"https://gliostate.com/library/VC0167/","api_url":"https://gliostate.com/api/items/VC0167.json","sameAs":["https://doi.org/10.64898/2026.10.16.759604"],"is_preprint":true,"facets":{"type":"https://gliostate.com/library/type/preprint/","year":"https://gliostate.com/library/year/2026/","tags":["https://gliostate.com/library/tag/benchmark/","https://gliostate.com/library/tag/critique/","https://gliostate.com/library/tag/perturbation/","https://gliostate.com/library/tag/perturb-seq/","https://gliostate.com/library/tag/foundation-model/"]},"cited_by":[],"related":[{"id":"VC0175","title":"Benchmarking biological knowledge-graph priors against single-cell foundation models for Perturb-seq prediction","shared_tags":["benchmark","perturb-seq","foundation-model","perturbation"],"page_url":"https://gliostate.com/library/VC0175/","api_url":"https://gliostate.com/api/items/VC0175.json"},{"id":"VC0179","title":"A split-control criterion for few-shot adaptation in single-cell perturbation prediction","shared_tags":["perturbation","benchmark","critique"],"page_url":"https://gliostate.com/library/VC0179/","api_url":"https://gliostate.com/api/items/VC0179.json"},{"id":"VC0180","title":"Transcriptional perturbation effects remain structured beyond the leading global trend","shared_tags":["perturbation","benchmark","critique"],"page_url":"https://gliostate.com/library/VC0180/","api_url":"https://gliostate.com/api/items/VC0180.json"},{"id":"VC0145","title":"A comparison of computational methods for expression forecasting.","shared_tags":["benchmark","critique","perturbation"],"page_url":"https://gliostate.com/library/VC0145/","api_url":"https://gliostate.com/api/items/VC0145.json"},{"id":"VC0144","title":"Simple controls exceed best deep learning algorithms and reveal foundation model effectiveness for predicting genetic perturbations.","shared_tags":["benchmark","critique","perturbation"],"page_url":"https://gliostate.com/library/VC0144/","api_url":"https://gliostate.com/api/items/VC0144.json"},{"id":"VC0174","title":"Emerging frontiers for AI in single-cell genomics","shared_tags":["foundation-model","perturb-seq"],"page_url":"https://gliostate.com/library/VC0174/","api_url":"https://gliostate.com/api/items/VC0174.json"}]}