Event Details

Artificial intelligence (AI) tools are increasingly being used in evidence synthesis...

Artificial intelligence (AI) tools are increasingly being used in evidence synthesis. But how do we decide whether using an AI tool is appropriate, and what are the considerations for performance and responsibility? This session will (1) highlight expectations of evidence synthesists and user’s of AI tools, and (2) introduce the framework on responsible handover of an AI tool to the evidence synthesis community (available in the Responsible use of AI in evidence SynthEsis (RAISE) recommendations and guidance; https://osf.io/fwaud/overview). It will open by introducing the expectations of evidence synthesists using AI tools as detailed in RAISE and Cochrane’s joint position statement of AI use in evidence synthesis. This will be followed by highlighting the responsible handover framework and its five domains; what is the purpose of the tool; where have the training, testing and validation data come from; is the AI tool validated and performing sufficiently well for use; usability and user capability; and transparency, licenses, availability & documentation. The session will end with discussion on how to interpret the findings after using the framework, as well as considerations for planning to use an AI tool in a systematic review and reporting its use.

Presenters: Ella Flemyng and Anna Noel-Storr, Cochrane

Ella Flemyng is Head of Editorial Policy and Research Integrity at the Cochrane Collaboration. She leads the team that works with the wider Cochrane community to define standards for trusted evidence synthesis, covering methods, research integrity and publication ethics. With over 10 years experience in STEM publishing, her work involves research to inform policies and standards, and mainstreaming best practice across the wider field. Ella is a Council Member on the Committee of Publication Ethics (COPE), Convenor of the joint AI Methods Group, part of the RAISE initiative management group, and investigator on DESTINY, a Wellcome-funded project creating digital infrastructure and tools to facilitate rapid evidence synthesis in climate and health.

Anna Noel-Storr is Head of Evidence Pipeline and Data Curation at the Cochrane Collaboration. An information specialist by background, Anna and her team manage systems in Cochrane aimed at helping to identify primary research and make Cochrane evidence more discoverable. She led the development of the Cochrane Crowd platform, Cochrane's citizen science initiative, which enables anyone with an interest in health to help identify and describe health research. Much of Anna's work involves testing and developing workflows that combine human and machine effort in the study identification stages of evidence synthesis. She's currently involved in several Wellcome funded initiatives, all aimed at making evidence synthesis more efficient and sustainable.