IBM Press Room – IBM and Sage Bionetworks announced today that the winners of the first phase of its DREAM Digital Mammography (DM) Challenge have developed algorithms that had 5% fewer false-positive errors in breast cancer screenings than recently published state of the art computerized…
In the first task, they developed an algorithm with a predictive accuracy of 80.3 percent, which is 5 percent more accurate than the runner up. In the second task, Nikulin and his team developed an algorithm that was 80.4 percent accurate. The group developed an algorithm with a predictive accuracy of 77.5 percent and outperformed the runner-up by more than 2 percent. Though the difference in accuracy between Guan’s and Nikulin’s teams was 2.9 percent, their performance was indistinguishable in the other metrics used to score the algorithms. Both winning teams used “Deep Learning, ” one of the most advanced artificial intelligence techniques capable of analyzing and interpreting images. Mammograms are widely considered the most accessible and cost-effective breast cancer screening method. However, the American Cancer Society and the United States Preventive Services Task Force recently issued changes to recommendations for when women should start having mammograms and how often they should get them. The changes are due, in part, to the large number of false-positive mammograms. One in 10 women undergoing screening mammography is recalled for a diagnostic workup, though fewer than 5 percent of the recalled women will eventually be found to have cancer. Recalled patients often experience stress and additional medical costs, and some require interventions, including unnecessary biopsies. Participating teams used hundreds of thousands of de-identified mammograms and clinical trial data provided by Kaiser Permanente Washington and the Icahn School of Medicine at Mount Sinai to create algorithms that can determine a woman’s cancer status in the 12 months following her mammogram. Eight teams with the best algorithms will now move on to the community phase of the challenge, where they will be invited to add outside expert collaborators. They will also share their source code publicly, including other challenge participants, in an attempt to foster cooperative learning. In the community phase, finalists will work together to develop an algorithm that can fully match the accuracy of an expert radiologist. “I am extremely pleased with the results of the competitive phase of the DM Challenge, ” said Gustavo Stolovitzky, Director at IBM Research and Founder of the DREAM Challenges. “By providing powerful computational resources and making available what is, to the best of our knowledge, the largest public mammography dataset ever released, we empowered hundreds of data scientists to contribute to the solution in the fight against breast cancer. Moreover, the code and methods generated during the DM Challenge are now available for anybody interested in building on these results to help solve this important public health problem.” “The innovation in this challenge stems not only from its final output—a set of robust models to aid clinicians in detecting breast cancer—but also the structure of the challenge itself. This challenge embodies a new paradigm for data sharing and cloud-hosted collaboration to tackle important questions in biomedicine, ” said Dr. Justin Guinney of Sage Bionetworks. “By working together as a community of researchers and using the best tools of science and technology, we have advanced set the framework for clinicians in the field of breast cancer detection.” If by the end of the community phase, the top eight teams—including Nikulin’s, Guan’s, and six others—can develop an algorithm that matches the expert radiologist performance of about 87.9 percent accuracy, they will receive a prize of up to $1 million. The DM Challenge was born out of the White House’s Cancer Moonshot initiative and is funded in part by the Laura and John Arnold Foundation. It was designed by an organizing committee that includes IBM, Sage Bionetworks, Kaiser Permanente Washington, the Icahn School of Medicine at Mount Sinai, and the U. S. Food and Drug Administration. The challenge relied on the technological advances of Sage Bionetworks and IBM—Sage provided Synapse, a collaborative platform to host the challenge, as well as science and engineering expertise; IBM research teams in the United States, Israel, and Australia built the infrastructure for the challenge within the IBM Watson Health SoftLayer cloud and contributed further engineering and data science expertise. Additional information about the DM Challenge is available here: First conceived by IBM in 2006, DREAM Challenges have addressed objectives that range from predictive models for disease progression to developing models for cell signaling networks. Designed and run by a community of researchers, DREAM Challenges invite participants to propose solutions, fostering collaboration and building communities in the process. The DREAM Challenges community shares a vision of open collaboration to leverage the “wisdom of the crowd” to improve human health and sciences. For more than seven decades, IBM Research has defined the future of information technology, with more than 3,000 researchers in 12 labs located across six continents. Scientists from IBM Research have been awarded six Nobel prizes, a U. S. Presidential Medal of Freedom, ten U. S. National Medals of Technology, five U. S. National Medals of Science, and six Turing Awards. The teams have also included 19 inductees into the U. S. National Academy of Sciences and 20 inductees into the U. S. National Inventors Hall of Fame. For more information about IBM Research, visit. scientific inquiries and discoveries. In pursuit of this mission, Sage Bionetworks has assembled an information commons for biomedicine supported by Synapse, an open compute space. The commons facilitates open research collaborations
