Standards of Evidence

Standards of Evidence

Governments and healthcare policy makers around the world have decided that quality of care within their systems should improve, that it should be evidence based, and that it is in the public’s interest to ensure that this happens (Barlow, 2004; Institute of Medicine, 2001, 2015; McHugh & Barlow, 2012). Various organizations around the world have attempted to meet this mandate by tackling the thorny questions focusing on what constitutes best evidence and how much of it do we need before we designate a clinical process as "evidence based". The Institute of Medicine (2001) following the lead of Sackett (Sackett, Rosenberg, Gray, Haynes & Richardson, 1996) proffered that evidence based practice (EBP) "" is the integration of best research evidence with clinical expertise and patient values" (Institute of Medicine, 2001, pg. 147). One important component of EBP is the empirical support for specific interventions for applied disorders and problems. Here the focus is on the efficacy and effectiveness (clinical utility) of specific interventions that have earned the designation "Empirically Supported Treatment" (EST). Earlier on, the Division of Clinical Psychology (Division 12) of the American Psychological Association took a stab at developing criteria by which to judge a treatment as empirically supported noting very specifically the number of experimental studies that would be required. At around the same time the Agency for Healthcare Policy And Research of the United States Public Health Service (now the Agency for Healthcare Research and Quality) developed a set of criteria that ranked evidence in a hierarchical fashion from the most convincing (typically well done randomized clinical trials) ranging down to the consensus of expert opinion which, while still contributory, would be substantially less convincing than well done experimental studies. Also in the mid 1990"s, an American Psychological Association joint task force created a template for developing clinical practice guidelines (APA, 1995, since updated, APA 2002) that also concluded that these guidelines should be constructed based on a hierarchy of evidence which would then be evaluated by a competent committee of clinicians and clinical scientists.

Since that time, this hierarchical approach to evaluating evidence regarding interventions has been widely adopted around the world by government healthcare agencies, such as the National Health Service (NHS) in the UK. This group commissioned the National Institute for Health and Care Excellence (NICE) expressly for the purpose of developing the evidence base to guide healthcare in the NHS. In the United States both the APA and ApA have now developed sophisticated guidelines (e.g. American Psychiatric Association, 2018; American Psychological Association, 2019). Also, several years ago a major joint initiative of the National Institute of Mental Health and the Substance Abuse and Mental Health Services Administration (SAMSHA) in the United States focused on promoting, implementing and evaluating evidence based mental health practices within state mental health systems (NIH, 2004). These governmental and additional professional agencies have adopted a hierarchical approach to evaluating evidence with data from well done randomized clinical trials as the gold standard, but not excluding non-randomized but large studies conducted directly in clinical practice settings, and consensus among experts on the empirical grounding and totality of the evidence supporting a given intervention. And the American Psychological Association (2006) has published an up-to-date statement on evidence based practice also detailing this hierarchy of evidence, as well as the attention to the context in which the treatment is applied and the differential levels of clinical expertise necessary to apply it. It is important to note that both the APA (2005) statement and other important statements (e. g. Barlow, Bullis, Comer, & Ametaj, 2013) recognize that EPB goes beyond ESTs to include scientifically based approaches to disorders or problems for which ESTs may not yet exist. Here, extrapolation from existing evidence on interventions and ongoing careful measurement of progress becomes important. But the explosion of evidence on ESTs in recent years, and the sometimes premature or inappropriate dissemination of interventions that are not empirically supported require the existence of a forum for evaluating and publishing a series of interventions that meet the highest standards of our field.

In this series we aim to present cutting edge interventions that are both empirically grounded and have accumulated sufficient evidence to be disseminated to healthcare professionals as "empirically supported". The Board of Scientific Affairs of the "Treatments that Works" series at Oxford University Press will carefully evaluate the nature of the intervention and the hierarchy of evidence available to support it before making a decision to endorse and disseminate the treatment (or not). This decision will be made without regard to the theoretical or empirical origin of the intervention but will assess for the adequacy of existing randomized prospective clinical trials or, failing this, other types of evidence such as, clinical trials that do not meet this high criteria, effectiveness studies, meta-analytic evaluations, and expert consensus. For interventions that are preventive in nature, evidence may also include reduction in known risk factors, increases in behaviors known to be associated with resilience, as well as symptomatic reduction at various points in time. Of course, interventions with no more than consensual support would not meet the high criteria required for the series. The purpose of this review then, is to provide clinicians with our best judgment of new cutting-edge treatments with sufficient support to warrant consideration for inclusion in any clinical practice guideline.

The process of developing evidence for a given intervention, and the parameters under which it is effective really never ends. The task of evaluating this evidence presents a moving target that will be reconsidered from time to time on the basis of new and emerging data. The fundamental belief guiding our efforts is that, although established knowledge may never be as complete or detailed as we would like, the scientific method is the only way to demonstrate definitively whether a treatment works. Without that proof, everyone should be cautious and skeptical in recommending or applying a particular treatment to relieve suffering or enhance functioning. We sincerely hope that clinicians around the world and the individuals they serve will benefit from this series.


American Psychiatric Association. (2018). Practice guideline for the pharmacological treatment of patients with alcohol use disorder. Washington, DC: American Psychiatric Association.

American Psychological Association. (1995). Template for developing guidelines: Interventions for mental disorders and psychosocial aspects of physical disorders. Washington, DC: American Psychological Association.

American Psychological Association. (2002). Criteria for evaluating treatment guidelines. American Psychologist, 57, 1052-1059.

American Psychological Association (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271-285.

American Psychological Association. (2019). A clinical practice guideline for the treatment of depression across three age groups. Washington, DC: American Psychological Association. 

Barlow, D.H. (2004). Psychological treatments. American Psychologist, 59(9), 869-878.

Barlow, D. H., Bullis, J. R., Comer, J. S., & Ametaj, A. A. (2013). Evidence-based psychological treatments: An update and a way forward. In S. Nolen-Hoeksema, T.D. Cannon, & T. Widiger (Eds.), Annual Review of Clinical Psychology (Vol. 9, pp. 1-27). Palo Alto, CA: Annual Reviews.

Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.

Institute of Medicine (2015). Improving Diagnosis in Healthcare. Washington, DC: National Academy Press.

McHugh, R. K., & Barlow, D. H. (Eds.). (2012). Dissemination and implementation of evidence-based psychological interventions. New York: Oxford University Press.

National Institute of Health (2004). State implementation of evidence-based practices: Bridging science and service. (NIMH and SAMHSA RFA MH-03-007). Retrieved November 19, 2004.

Sackett, D. L., Rosenberg, W.M., Gray, J.A., Haynes, R.B., & Richardson, W.S. (1996). Evidence based medicine: What it is and what it isn"t. British Medical Journal, 312, 71-72.