Holistic evaluation of large language models for medical tasks with MedHELM.

Saved in:
Bibliographic Details
Title: Holistic evaluation of large language models for medical tasks with MedHELM.
Authors: Bedi S; Stanford University School of Medicine, Stanford, CA, USA. suhana@stanford.edu., Cui H; Stanford University School of Medicine, Stanford, CA, USA., Fuentes M; Stanford University School of Medicine, Stanford, CA, USA., Unell A; Stanford University School of Medicine, Stanford, CA, USA., Wornow M; Stanford University School of Medicine, Stanford, CA, USA., Banda JM; Stanford Health Care, Palo Alto, CA, USA., Kotecha N; Stanford Health Care, Palo Alto, CA, USA., Keyes T; Stanford Health Care, Palo Alto, CA, USA., Mai Y; Center for Research on Foundation Models (CRFM) & Department of Computer Science, Stanford University, Stanford, CA, USA., Oez M; Microsoft Corporation, Redmond, WA, USA., Qiu H; Microsoft Corporation, Redmond, WA, USA., Jain S; Microsoft Corporation, Redmond, WA, USA., Schettini L; Microsoft Corporation, Redmond, WA, USA., Kashyap M; Stanford University School of Medicine, Stanford, CA, USA., Fries JA; Stanford University School of Medicine, Stanford, CA, USA., Swaminathan A; Stanford University School of Medicine, Stanford, CA, USA., Chung P; Stanford University School of Medicine, Stanford, CA, USA., Haredasht FN; Stanford University School of Medicine, Stanford, CA, USA., Lopez I; Stanford University School of Medicine, Stanford, CA, USA., Aali A; Stanford University School of Medicine, Stanford, CA, USA., Tse G; Stanford University School of Medicine, Stanford, CA, USA., Nayak A; Stanford University School of Medicine, Stanford, CA, USA., Vedak S; Stanford University School of Medicine, Stanford, CA, USA., Jain SS; Stanford University School of Medicine, Stanford, CA, USA., Patel B; Stanford University School of Medicine, Stanford, CA, USA., Fayanju O; Stanford University School of Medicine, Stanford, CA, USA., Shah S; Stanford University School of Medicine, Stanford, CA, USA., Goh E; Stanford University School of Medicine, Stanford, CA, USA., Yao DH; Stanford University School of Medicine, Stanford, CA, USA., Soetikno B; Stanford University School of Medicine, Stanford, CA, USA., Reis E; Stanford University School of Medicine, Stanford, CA, USA., Gatidis S; Stanford University School of Medicine, Stanford, CA, USA., Divi V; Stanford University School of Medicine, Stanford, CA, USA., Capasso R; Stanford University School of Medicine, Stanford, CA, USA., Saralkar R; Stanford University School of Medicine, Stanford, CA, USA., Chiang CC; Stanford University School of Medicine, Stanford, CA, USA., Jindal J; Stanford University School of Medicine, Stanford, CA, USA., Pham T; Stanford University School of Medicine, Stanford, CA, USA., Ghoddusi F; Stanford University School of Medicine, Stanford, CA, USA., Lin S; Stanford University School of Medicine, Stanford, CA, USA., Chiou AS; Stanford University School of Medicine, Stanford, CA, USA., Hong HJ; Stanford University School of Medicine, Stanford, CA, USA., Roy M; Stanford University School of Medicine, Stanford, CA, USA., Gensheimer MF; Stanford University School of Medicine, Stanford, CA, USA., Patel H; Stanford University School of Medicine, Stanford, CA, USA., Schulman K; Stanford University School of Medicine, Stanford, CA, USA., Dash D; Stanford University School of Medicine, Stanford, CA, USA., Char D; Stanford University School of Medicine, Stanford, CA, USA., Downing L; Stanford University School of Medicine, Stanford, CA, USA., Grolleau F; Stanford University School of Medicine, Stanford, CA, USA., Black K; Stanford University School of Medicine, Stanford, CA, USA., Mieso B; Stanford University School of Medicine, Stanford, CA, USA., Zahedivash A; Stanford University School of Medicine, Stanford, CA, USA., Yim WW; Microsoft Corporation, Redmond, WA, USA., Sharma H; Microsoft Corporation, Redmond, WA, USA., Lee T; Center for Research on Foundation Models (CRFM) & Department of Computer Science, Stanford University, Stanford, CA, USA., Kirsch H; Stanford Health Care, Palo Alto, CA, USA., Lee J; Stanford Health Care, Palo Alto, CA, USA., Ambers N; Stanford Health Care, Palo Alto, CA, USA., Lugtu C; Stanford Health Care, Palo Alto, CA, USA., Sharma A; Stanford Health Care, Palo Alto, CA, USA., Mawji B; Stanford Health Care, Palo Alto, CA, USA., Alekseyev A; Stanford Health Care, Palo Alto, CA, USA., Zhou V; Stanford Health Care, Palo Alto, CA, USA., Kakkar V; Stanford Health Care, Palo Alto, CA, USA., Helzer J; Stanford Health Care, Palo Alto, CA, USA., Revri A; Stanford Health Care, Palo Alto, CA, USA., Bannett Y; Stanford University School of Medicine, Stanford, CA, USA., Daneshjou R; Stanford University School of Medicine, Stanford, CA, USA., Chen J; Stanford University School of Medicine, Stanford, CA, USA., Alsentzer E; Stanford University School of Medicine, Stanford, CA, USA., Morse K; Stanford University School of Medicine, Stanford, CA, USA., Ravi N; eHealth Africa Clinics, Kano, Nigeria., Aghaeepour N; Stanford University School of Medicine, Stanford, CA, USA., Kennedy V; Stanford University School of Medicine, Stanford, CA, USA., Chaudhari A; Stanford University School of Medicine, Stanford, CA, USA., Wang T; Stanford University School of Medicine, Stanford, CA, USA.; Stanford Health Care, Palo Alto, CA, USA., Koyejo S; Center for Research on Foundation Models (CRFM) & Department of Computer Science, Stanford University, Stanford, CA, USA.; Stanford Institute for Human-Centered AI, Stanford, CA, USA., Lungren MP; Stanford University School of Medicine, Stanford, CA, USA.; Microsoft Corporation, Redmond, WA, USA., Horvitz E; Microsoft Corporation, Redmond, WA, USA.; Stanford Institute for Human-Centered AI, Stanford, CA, USA., Liang P; Center for Research on Foundation Models (CRFM) & Department of Computer Science, Stanford University, Stanford, CA, USA.; Stanford Institute for Human-Centered AI, Stanford, CA, USA., Pfeffer MA; Stanford Health Care, Palo Alto, CA, USA., Shah NH; Stanford University School of Medicine, Stanford, CA, USA.; Stanford Health Care, Palo Alto, CA, USA.; eHealth Africa Clinics, Kano, Nigeria.
Source: Nature medicine [Nat Med] 2026 Mar; Vol. 32 (3), pp. 943-951. Date of Electronic Publication: 2026 Jan 20.
Publication Type: Journal Article
Journal Info: Publisher: Nature Publishing Company Country of Publication: United States NLM ID: 9502015 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1546-170X (Electronic) Linking ISSN: 10788956 NLM ISO Abbreviation: Nat Med Subsets: MEDLINE
Database: MEDLINE Ultimate
Description
ISSN:1546-170X
DOI:10.1038/s41591-025-04151-2