Trust, But Verify: Informational Challenges Surrounding AI-Enabled Clinical Decision Software



Trust, But Verify: Informational Challenges Surrounding AI-Enabled Clinical Decision Software

January 23, 2020 — 12:30PM–5:00PM

Duke-Robert J. Margolis, MD, Center for Health Policy

By 2021, the size of the health AI market is expected to be about 11 times what it was in 2014, growing from $600 million to an estimated $6.6 billion.1 With this explosion of health AI technology comes an urgent need to better understand how to effectively communicate information about these tools in order to harness benefits and mitigate risks. Clinical decision software enabled by AI represents a particularly important area of focus. Information around appropriate use, benefits, and risks is potentially more complicated to determine and/or disclose for AI-enabled clinical decision software than for traditional medical devices. For example, AI-enabled software that utilizes certain deep learning techniques can make recommendations, but there may be no way for a human to understand how the software came to that recommendation. Additionally, commercial competitiveness concerns may restrict how much information a company may want to release. In the absence of information about “how the software works,” what information do stakeholders want? What information do companies want to share?

To explore these issues and more, the Center for Innovation Policy at Duke Law and the Duke-Margolis Center for Health Policy are holding a public meeting to better understand how to incentivize innovation in this space while communicating necessary information to stakeholders on how to use these products safely and effectively. Specific topics include:

  • Informational needs around AI-enabled clinical software during regulation, adoption, and point-of-use;
  • How information needed for this software may differ from that needed for traditional medical products and therefore how to effectively communicate it; and
  • The role of innovation incentives, such as patents and trade secrecy, over information flow.

Note: This event will also be webcast. In-person attendance is limited, so please register at early.

If you have any questions please email

1 Accenture (2017) “Artificial Intelligence: Healthcare’s New Nervous System”


Funding for this meeting was made possible in part by a grant from the Greenwall Foundation. The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Greenwall Foundation nor does mention of trade names, commercial practices, or organizations imply endorsements.