Many clinical trial technology vendors are racing to add AI to their offerings. But it’s sponsors and CROs who carry the risk of any AI tools they adopt.
Clinical trial teams need to ask the right questions to ensure that any AI tool they use is built, trained, validated, monitored, and governed in a compliant way. A thoughtful approach means studies can move quickly without sacrificing high-quality data.
In this article, you’ll find:
- What risks to consider when evaluating AI tools
- Why it’s important to consider a tool’s intended use
- What questions to ask before adopting any AI tool
Want the full list of questions? Download the infographic on key AI questions to ask in clinical trials.
Identifying risks with AI tools in clinical trials
AI tools can introduce different risks depending on how they are designed and used. Clinical trial teams should identify these risks before any tool is added to a clinical trial.
Risks include:
- Generative AI can produce incorrect or misleading outputs.
- AI agents can take actions that users did not expect.
- Models can reflect bias from their training data.
- Shared model environments can raise concerns about data isolation.
Understanding these risks helps teams choose appropriate safeguards, oversight, and validation requirements.
Curious about how Medrio thinks about AI? Read our AI approach article.
Evaluating an AI tool’s intended use
Start by defining exactly what the AI tool will do and how its output will be used.
The FDA’s draft guidance recommends:
- Considering the tool’s context of use, or its specific role and scope in addressing a defined question or decision.
- Matching the assessment to the risk created.
The draft guidance outlines a risk-based credibility assessment framework, which means that as an output becomes more important, it requires greater scrutiny. As risk increases, clinical trial teams need to take a more rigorous approach to oversight, validation, performance criteria, risk controls, and documentation.
For example, take a tool that summarizes documents versus one that impacts trial data. An AI tool that summarizes internal documents may pose less risk since it does not affect patient safety or study results. A tool that influences clinical data, safety decisions, or regulatory evidence requires much stronger evidence and controls.
The same technology can carry different risks in different settings.
Before comparing features, teams should define what questions AI will address, where it fits into the workflow, and how people will use its output. Then they can assess whether a vendor can demonstrate that the tool is credible for that specific purpose.
Asking questions before adopting AI tools
Use the following key questions to evaluate an AI tool before integrating it into your clinical trials.
What is the underlying AI model? How was it trained?
Understand the AI’s underlying algorithms and development process to support its reliability. This kind of transparency supports informed, strategic decisions that align AI capabilities with the rigor required in clinical research.
Why it matters: Each type of AI in clinical trials has different risks. Generative tools may produce plausible but incorrect information in clinical research. Agentic tools may take multi-step actions across connected systems. A careful approach to methods and training will reduce the risk of incorrect, misleading, or fabricated information.
How is clinical trial data quality ensured before AI processing?
Confirm safeguards are in place to validate clinical trial data before it is entered for AI processing, maintaining data integrity and patient safety. Carefully curated data will result in quality AI outputs.
Why it matters: For clinical trials, data isolation should be clear. If a vendor cannot explain how customer data is protected, this raises a red flag for sponsors and CROs.
What is the interpretability of the AI outputs?
Interpretability determines whether users can understand how an AI tool reached an output, what evidence supports it, and what limitations may affect it. Clinical trial stakeholders need to be able to interpret, trust, and communicate any AI-related outputs.
Why it matters: Clinical teams need to understand the basis for AI outputs. If users cannot interpret the output, they cannot defend the decision that follows.
How is the AI validated? Are there ongoing evaluations?
Clarify how the AI is validated and re-evaluated so it consistently performs as expected under real-world conditions. It’s essential to document these validation processes to provide an audit trail for regulatory reviews.
Why it matters: For AI embedded in clinical trial systems, validation should align with computer system validation expectations. It should also account for the fact that model behavior can change over time.
What are the potential biases in the AI models?
Ask how the vendor identifies, tests, and reduces bias in the model’s training data, design, and outputs. Review whether performance has been evaluated across relevant patient groups, sites, regions, and study populations.
Why it matters: Biased data can produce uneven or misleading results. In clinical trials, these factors can affect patient selection, risk detection, data interpretation, and reliability of study conclusions.
What regulatory considerations are addressed?
Maintain compliance with relevant regulations and guidelines so that data is legally acceptable upon submission. A vendor needs to be able to explain how any tool supports data integrity, privacy, security, auditability, human oversight, and regulatory review.
Why it matters: Sponsors and CROs remain responsible for the data and evidence submitted to regulators. When AI contributes to decisions about safety, effectiveness, or product quality, teams need documented evidence that the tool is credible for that specific use.
Want the full list of questions? Download the infographic on key AI questions to ask in clinical trials.
Supporting responsible AI adoption in clinical research
AI tools can support faster, more efficient clinical trials—but only when used with clear purpose and appropriate oversight.
Sponsors and CROs should treat AI evaluation as part of their broader risk management process. By asking the right questions before adoption, teams can reduce avoidable risk and choose tools that support data quality, patient safety, and regulatory confidence.
