Skip to main content
U.S. flag

An official website of the United States government

Here’s how you know

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

HTTPS

Secure .gov websites use HTTPS
A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

  • Environmental Topics
  • Laws & Regulations
  • Report a Violation
  • About EPA
Risk Assessment
Contact Us

Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies

On this page:

  • Overview
  • Downloads
Introduction and objective: Systematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS) Program has developed an approach for systematic review of evidence of health effects from chemical exposures that includes structured approaches for literature search and screening, study evaluation, data extraction, and evidence synthesis and integration. This approach recognizes the need for developing outcome-specific criteria for study evaluation. Because studies are assessed at the outcome level, a study could be considered high quality for one investigated outcome, and low quality for another, due to differences in the outcome measures, analytic strategies, how relevant a certain bias is to the outcome, and how the exposure measure relates to the outcome. The objective of this paper is to illustrate the need for outcome-specific criteria in study evaluation or risk of bias evaluation, describe the process we used to develop the criteria, and summarize the resulting criteria. Methods: We used a process of expert consultation to develop several sets of outcome-specific criteria to guide study reviewers, improve consistency, and ensure consideration of critical issues specific to the outcomes. The criteria were developed using the following domains: outcome assessment, exposure measurement (specifically timing of exposure in relation to outcome; other exposure measurement issues would be addressed in exposurespecific criteria), participant selection, confounding, analysis, and sensitivity (the study's ability to detect a true effect or hazard).   Results: We discuss the application of this process to pregnancy-related outcomes (preterm birth, spontaneous abortion), other reproductive-related outcomes (male reproductive hormones, sperm parameters, time to pregnancy, pubertal development), chronic disease (diabetes, insulin resistance), and acute or episodic conditions (asthma, allergies), and provide examples of the criteria developed. For each outcome the most influential methodological considerations are highlighted including biological sample collection and quality control, sensitivity and specificity of ascertainment tools, optimal timing for recruitment into the study (e.g., preconception, specific trimesters), the etiologically relevant window for exposure assessments, and important potential confounders. Conclusions: Outcome-specific criteria are an important part of a systematic review and will facilitate study evaluations by epidemiologists with experience in evaluating studies using systematic review methods who may not have extensive discipline-specific experience in the outcomes being reviewed.

Impact/Purpose

This paper describes the development of outcome-specific criteria for study evaluation of epidemiology studies during systematic reviews. This is an important topic for IRIS assessments and a significant contribution to the systematic review literature.

Citation

Radke-Farabaugh, E., B. Glenn, A. Galizia, A. Persad, R. Nachman, T. Bateson, Michael Wright, A. Navas-Ascien, W. Arroyave, R. Puett, E. Harville, A. Pollack, J. Burns, C. Lynch, S. Sagiv, C. Stein, AND G. Cooper. Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies. Elsevier B.V., Amsterdam, NETHERLANDS, 130:104884, (2019). [DOI: 10.1016/j.envint.2019.05.078]

Download(s)

DOI: Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies
  • Risk Assessment Home
  • About Risk Assessment
  • Risk Recent Additions
  • Human Health Risk Assessment
  • Ecological Risk Assessment
  • Risk Advanced Search
    • Risk Publications
  • Risk Assessment Guidance
  • Risk Tools and Databases
  • Superfund Risk Assessment
  • Where you live
Contact Us to ask a question, provide feedback, or report a problem.
Last updated on September 27, 2021
United States Environmental Protection Agency

Discover.

  • Accessibility Statement
  • Budget & Performance
  • Contracting
  • EPA www Web Snapshots
  • Grants
  • No FEAR Act Data
  • Privacy
  • Privacy and Security Notice

Connect.

  • Data
  • Inspector General
  • Jobs
  • Newsroom
  • Open Government
  • Regulations.gov
  • Subscribe
  • USA.gov
  • White House

Ask.

  • Contact EPA
  • EPA Disclaimers
  • Hotlines
  • FOIA Requests
  • Frequent Questions

Follow.