Cryo-EM model validation recommendations based on outcomes of the 2019 EMDataResource challenge

Abstract

This paper describes outcomes of the 2019 Cryo-EM Model Challenge. The goals were to (1) assess the quality of models that can be produced from cryogenic electron microscopy (cryo-EM) maps using current modeling software, (2) evaluate reproducibility of modeling results from different software developers and users and (3) compare performance of current metrics used for model evaluation, particularly Fit-to-Map metrics, with focus on near-atomic resolution. Our findings demonstrate the relatively high accuracy and reproducibility of cryo-EM models derived by 13 participating teams from four benchmark maps, including three forming a resolution series (1.8 to 3.1 Å). The results permit specific recommendations to be made about validating near-atomic cryo-EM structures both in the context of individual experiments and structure data archives such as the Protein Data Bank. We recommend the adoption of multiple scoring parameters to provide full and objective annotation and assessment of the model, reflective of the observed cryo-EM map density.

Metadata

Item Type:	Article
Authors/Creators:	Lawson, Catherine L. Kryshtafovych, Andriy Adams, Paul D. Afonine, Pavel V. Baker, Matthew L. Barad, Benjamin A. Bond, Paul https://orcid.org/0000-0002-8465-4823 Burnley, Tom Cao, Renzhi Cheng, Jianlin Chojnowski, Grzegorz Cowtan, Kevin https://orcid.org/0000-0002-0189-1437 Dill, Ken A. DiMaio, Frank Farrell, Daniel P. Fraser, James S. Herzik, Mark A. Hoh, Soon Wen https://orcid.org/0000-0003-1039-8000 Hou, Jie Hung, Li Wei Igaev, Maxim Joseph, Agnel P. Kihara, Daisuke Kumar, Dilip Mittal, Sumit Monastyrskyy, Bohdan Olek, Mateusz Palmer, Colin M. Patwardhan, Ardan Perez, Alberto Pfab, Jonas Pintilie, Grigore D. Richardson, Jane S. Rosenthal, Peter B. Sarkar, Daipayan Schäfer, Luisa U. Schmid, Michael F. Schröder, Gunnar F. Shekhar, Mrinal Si, Dong Singharoy, Abishek Terashi, Genki Terwilliger, Thomas C. Vaiana, Andrea Wang, Liguo Wang, Zhe Wankowicz, Stephanie A. Williams, Christopher J. Winn, Martyn Wu, Tianqi Yu, Xiaodi Zhang, Kaiming Berman, Helen M. Chiu, Wah
Copyright, Publisher and Additional Information:	Funding Information: EMDataResource (C.L.L., A.K., G.P., H.M.B. and W.C.) is supported by the US National Institutes of Health (NIH)/National Institute of General Medical Science, grant no. R01GM079429. The Singharoy team used the supercomputing resources of the Oak Ridge Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science at the Department of Energy under contract no. DE-AC05-00OR22725. The following additional grants are acknowledged for participant support: grant no. NIH/R35GM131883 to J.S.R. and C.W.; grant no. NIH/P01GM063210 to P.D.A., P.V.A., L.-W.H., J.S.R., T.C.T. and C.W.; National Science Foundation grant no. (NSF)/MCB-1942763 (CAREER) and NIH/R01GM095583 to A.S.; grant nos. NIH/R01GM123055, NIH/R01GM133840, NSF/DMS1614777, NSF/CMMI1825941, NSF/MCB1925643, NSF/DBI2003635 and Purdue Institute of Drug Discovery to D. Kihara; grant no. NIH/R01GM123159 to J.S.F.; Max Planck Society German Research Foundation grant no. IG 109/1-1 to M.I.; Max Planck Society German Research Foundation grant no. FOR-1805 to A.C.V.; grant nos. NIH/R37AI36040 and Welch Foundation/Q1279 to D. Kumar (PI: BVV Prasad); grant no. NSF/DBI2030381 to D. Si.; Medical Research Council grant no. MR/N009614/1 to T.B., C.M.P. and M.W.; Wellcome Trust grant no. 208398/Z/17/Z to A.P.J. and M.W.; Biotechnology and Biological Sciences Research Council grant no. BB/P000517/1 to K.C. and Biotechnology and Biological Sciences Research Council grant no. BB/P000975/1 to M.W. Publisher Copyright: © 2021, The Author(s). Copyright: Copyright 2021 Elsevier B.V., All rights reserved.
Dates:	Published: 4 February 2021 Accepted: 21 December 2020
Institution:	The University of York
Academic Units:	The University of York > Faculty of Sciences (York) > Chemistry (York) The University of York > Faculty of Arts and Humanities (York) > English and Related Literature (York) The University of York > Faculty of Sciences (York) > Computer Science (York)
Funding Information:	Funder Grant number BBSRC (BIOTECHNOLOGY AND BIOLOGICAL SCIENCES RESEARCH COUNCIL) BB/T012935/1 BBSRC (BIOTECHNOLOGY AND BIOLOGICAL SCIENCES RESEARCH COUNCIL) BB/S005099/1 BBSRC (BIOTECHNOLOGY AND BIOLOGICAL SCIENCES RESEARCH COUNCIL) BB/L006383/1
Depositing User:	Pure (York)
Date Deposited:	08 Feb 2021 12:10
Last Modified:	30 Nov 2024 01:13
Published Version:	https://doi.org/10.1038/s41592-020-01051-w
Status:	Published
Refereed:	Yes
Identification Number:	10.1038/s41592-020-01051-w
Related URLs:	http://www.scopus.com/inward/record.url?...
Open Archives Initiative ID (OAI ID):	oai:eprints.whiterose.ac.uk:170879

Download

Published Version

Filename: s41592_020_01051_w.pdf

Description: s41592-020-01051-w

Licence: CC-BY 2.5

CLICK TO DOWNLOAD

CORE (COnnecting REpositories)

Cryo-EM model validation recommendations based on outcomes of the 2019 EMDataResource challenge

Abstract

Metadata

Download

Published Version

Export

Statistics