Skip to main content
U.S. flag

An official website of the United States government

Here’s how you know

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

HTTPS

Secure .gov websites use HTTPS
A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

  • Environmental Topics
  • Laws & Regulations
  • Report a Violation
  • About EPA
Risk Assessment
Contact Us

Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique

On this page:

  • Overview
  • Downloads
Method: A two-round digital Delphi survey, followed by an online workshop with guided discussions, was performed. The Delphi participants were experienced with both in in vitro models and systematic review methods.   Results: Fifteen experts completed both Delphi rounds, and thirteen participated in the workshops. Of the 405 items in the bank, 372 were included for consideration when assessing potential for bias in an in vitro study. Both the literature sources and the focus group discussions contributed with concepts considered to be important for the assessment of the potential for bias in an in vitro study; 83%-100% of the items collected from the different literature sources were identified to be important and 91% (127) of the new items discovered in the focus group discussions were identified to be important.   Discussion:   The 372 retained items will be interpreted into a manageable set of study appraisal criteria and a supporting guidance that will constitute the INVITES-IN study appraisal tool. The high retention of items included in tools designed for assessment of human and animal studies to in vitro studies suggests that many items are generally applicable across multiple study designs. Therefore, tool development processes should benefit from drawing on assessment tools outside the immediate domain of interest. Tool development would also likely benefit from supplementing literature reviews with focus group discussions, as our results demonstrate that the use of focus group discussions with domain experts is a pragmatic and valuable approach to increasing coverage of items in a tool development process.   In conclusion, this study demonstrates the value of using rigorous methods to ensure a comprehensive dataset as the starting point for creation of an assessment tool.

Impact/Purpose

Context: In vitro toxicity studies are increasingly being included as evidence in systematic reviews and chemical risk assessments. INVITES-IN, a tool for assessing the internal validity of in vitro studies, is under development. The first step in the creation of INVITES-IN was the development of an “item bank” database of 405 concepts (“items”) of potential relevance for assessing the internal validity of in vitro toxicity studies. The items were gathered from focus groups and a purposive literature sample that included one systematic review and study appraisal tools for human, animal, and in vitro studies. In this paper we present the second step of the creation of INVITES-IN, i.e. the methods and results for identifying items for consideration when assessing the potential for bias in an in vitro study.  

Citation

Mathisen, G., C. Svendsen, G. Vist, T. Husøy, H. Ames, A. Bearth, M. Audebert, A. Bernhard, A. Beronius, E. Bruzell, E. Di Consiglio, M. Davenport, I. Druwe, R. Geci, U. Gundert-Remy, S. Hoffmann, T. Hartung, H. Hogberg, C. Hooijmans, Luci Lizarraga, J. Olker, P. Prieto, J. Robinson, A. Rooney, A. Sebollela, N. Smith, E. Spilioti, A. Spyropoulou, O. Tcheremenskaia, E. Testai, A. Wang, J. Zilliacus, AND P. Whaley. Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique. Taylor & Francis Group, London, UK1, (2025). [DOI: 10.5281/zenodo.14656764]

Download(s)

  • https://zenodo.org/records/14656764
DOI: Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique
  • Risk Assessment Home
  • About Risk Assessment
  • Risk Recent Additions
  • Human Health Risk Assessment
  • Ecological Risk Assessment
  • Risk Advanced Search
    • Risk Publications
  • Risk Assessment Guidance
  • Risk Tools and Databases
  • Superfund Risk Assessment
  • Where you live
Contact Us to ask a question, provide feedback, or report a problem.
Last updated on February 14, 2025
United States Environmental Protection Agency

Discover.

  • Accessibility Statement
  • Budget & Performance
  • Contracting
  • EPA www Web Snapshots
  • Grants
  • No FEAR Act Data
  • Privacy
  • Privacy and Security Notice

Connect.

  • Data
  • Inspector General
  • Jobs
  • Newsroom
  • Open Government
  • Regulations.gov
  • Subscribe
  • USA.gov
  • White House

Ask.

  • Contact EPA
  • EPA Disclaimers
  • Hotlines
  • FOIA Requests
  • Frequent Questions

Follow.