{"id":"VC0146","type":"paper","title":"Deep learning perturbation models can outperform baselines on calibrated metrics.","authors_or_org":"Miller HE, Mejia GM, Leblanc FJA, Swain B, Wang B, de Lima Camillo LP.","date":"2026-10-01","year":"2026","venue":"Nature biotechnology","url":"https://doi.org/10.1038/s41587-026-03307-w","doi":"10.1038/s41587-026-03307-w","pmid":"42823510","brain_tumor_focus":"no","tags":["benchmark","evaluation-principles","critique","linear-baseline"],"tag_slugs":["benchmark","critique"],"tag_labels":["Benchmark & validation","Critique"],"tag_labels_zh":["基准评测与验证","批评与反证"],"unmapped_tags":[],"one_line_summary_en":"Miller et al. (Nature Biotechnology 2026; authors from Shift Bioscience, competing-interest statement in paper): positive-control baselines and a 'dynamic range fraction' calibration framework over 14 datasets and 18 metrics; argues MSE and Pearson delta are often miscalibrated and that deep-learning models can beat uninformative baselines on calibrated metrics. Counter-position to VC0048/VC0049/VC0052/VC0144/VC0145.","one_line_summary_zh":"Miller 等（Nature Biotechnology 2026；作者来自 Shift Bioscience，论文含利益冲突声明）：14 个数据集、18 个指标上的阳性对照基线与“动态范围分数”校准框架；认为 MSE 与 Pearson delta 常校准不良，在校准指标下深度学习模型可超过无信息基线。与 VC0048/VC0049/VC0052/VC0144/VC0145 立场相对。","verified":"yes","verification_source":"Europe PMC (metadata; preprint 10.1101/2025.10.20.683304 indexed as PPR1104931) + Nature article page via doi.org read 2026-10-16 (full text, incl. competing interests); PMID 42823510","page_url":"https://gliostate.com/library/VC0146/","api_url":"https://gliostate.com/api/items/VC0146.json","sameAs":["https://doi.org/10.1038/s41587-026-03307-w","https://pubmed.ncbi.nlm.nih.gov/42823510/"],"is_preprint":false,"facets":{"type":"https://gliostate.com/library/type/paper/","year":"https://gliostate.com/library/year/2026/","tags":["https://gliostate.com/library/tag/benchmark/","https://gliostate.com/library/tag/critique/"]},"cited_by":[{"id":"validation-linear-baselines-systematic-variation","type":"validation","title_en":"Linear baselines and systematic variation in perturbation prediction: what 'a strong baseline' means, and where the papers disagree","title_zh":"扰动预测中的线性基线与系统性变异：“强基线”指什么，各论文在哪里分歧","url":"https://gliostate.com/validation/linear-baselines-systematic-variation/","api_url":"https://gliostate.com/api/topics/validation-linear-baselines-systematic-variation.json"}],"related":[{"id":"VC0145","title":"A comparison of computational methods for expression forecasting.","shared_tags":["benchmark","critique"],"page_url":"https://gliostate.com/library/VC0145/","api_url":"https://gliostate.com/api/items/VC0145.json"},{"id":"VC0048","title":"Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines.","shared_tags":["benchmark","critique"],"page_url":"https://gliostate.com/library/VC0048/","api_url":"https://gliostate.com/api/items/VC0048.json"},{"id":"VC0144","title":"Simple controls exceed best deep learning algorithms and reveal foundation model effectiveness for predicting genetic perturbations.","shared_tags":["benchmark","critique"],"page_url":"https://gliostate.com/library/VC0144/","api_url":"https://gliostate.com/api/items/VC0144.json"},{"id":"VC0049","title":"Benchmarking foundation cell models for post-perturbation RNA-seq prediction.","shared_tags":["benchmark","critique"],"page_url":"https://gliostate.com/library/VC0049/","api_url":"https://gliostate.com/api/items/VC0049.json"},{"id":"VC0050","title":"Zero-shot evaluation reveals limitations of single-cell foundation models.","shared_tags":["benchmark","critique"],"page_url":"https://gliostate.com/library/VC0050/","api_url":"https://gliostate.com/api/items/VC0050.json"},{"id":"VC0078","title":"Biological domain shift and statistical nesting concerns in generative AI-based spatial tumor growth prediction for pediatric diffuse midline glioma.","shared_tags":["critique"],"page_url":"https://gliostate.com/library/VC0078/","api_url":"https://gliostate.com/api/items/VC0078.json"}]}