Artificial Intelligence (AI) has tremendous potential to advance healthcare and improve the lives of everyone. But successful clinical translation requires evaluating the performance of AI models on large and diverse real-world datasets. MLCommons, an open global engineering consortium dedicated to making machine learning better for everyone, announced on July 17, 2023, a major milestone toward addressing this challenge with the publication of “Federated Benchmarking of Medical Artificial Intelligence with MedPerf” in the Nature Machine Intelligence journal.
MedPerf is an open benchmarking platform that efficiently evaluates AI models on diverse real-world medical data and delivers clinical efficacy while prioritizing patient privacy and mitigating legal and regulatory risks. The publication in Nature Machine Intelligence is the result of a two-year global collaboration spearheaded by the MLCommons Medical Working Group, with participation of experts from more than 20 companies, more than 20 academic institutions, and nine hospitals across 13 countries, including Dana-Farber Cancer Institute.
