Guidelines are proposed to put comparative effectiveness research on a more solid footing

“Comparative effectiveness research” (CER) is one of the newer catchphases to bring together clinical evidence and value to patients and society in new medical technologies; the concept subsumes aspects of outcomes research, health economics and various value-based analyses. The CER concept was one of the formative ideas behind the Patient-Centered Outcomes Research Institute (PCORI), established by the Patient Protection and Affordable Care Act of 2010, and now staffed up and running; at the same time, many health plans, pharmacy benefit managers and healthcare organizations are generating their own studies and programs—and making healthcare decisions based on those programs.

There’s only one problem here: there is no consensus, across all stakeholders, in what constitutes “good” CER, and how and when CER studies are appropriate evidence in making healthcare decisions. That problem, together with a proposed list of 13 guiding principles to address this gap, are the subject of a paper just published in the J. of Comparative Effectiveness Research [J. Comp. Effect. Res., 2012, 1(5)]. With funding from the National Pharmaceutical Council (NPC; Washington, DC), Drs. Bryan Luce, SVP of science policy at United BioSource Corp. (Bethesda, MD) and Robert Dubois (chief science officer at NPC) and five academicians developed the principles and vetted them using an iterative process of gathering opinions from successive groups of healthcare stakeholders and experts.

“The reality is, CER is more of an aspiration than a methodology,” says Luce. “If you gathered 10 CER experts in a room and asked them what it actually entails, you would not get a clear answer.” The paper cites the definition adopted by the Institute of Medicine:

CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policymakers to make informed decisions that will improve health care at both the individual and population levels.’

Luce and his coauthors goes on to say that to be useful, CER needs “a more general set of principles that include the process of planning and conducting CER studies, and the potential for CER to improve healthcare and health”—and the guiding principles it proposes address this process. Those principles include self-evident principles of bias and transparency, broad consideration of alternatives (to a particular therapy), heterogeneity (across patient populations), and generalizability (being able to adapt findings across “patients, settings, geography and systems of care”). The more difficult aspect several principles is to explicitly include “all relevant stakeholders and decision-makers” in a study’s design and conduct, which could include patients, payers, manufacturers, medical researchers, employers and others. Not all “stakeholders” are “decision-makers” (who would act on the results of a CER study); “for example, a sponsoring manufacturer is a stakeholder but not normally considered a decision-maker,” says the paper.

Another tough element is economics: “It has been a political decision not to include economics in PCORI or CMS decision-making,” says Luce, “but very obviously, CMS is concerned with the cost of care, and private insurers routinely look at economics. One of the guiding principles, “relevancy,” incorporates the economic dimension: When costs and cost—effectiveness are clearly relevant, CER studies should endeavor to assess them,” states the paper.

There are other efforts to formalize CER practices, such as the GRACE initiative, and PCORI is wrapping up a comment period on its Methodology Report right now. Luce says that his group hopes to follow up its paper with ongoing evaluation of how well CER studies and organizations meet these principles, and to reinforce that, while few if any CER studies could meet all of these “aspirational” goals, decisionmakers should keep the principles in mind when evaluating CER results. UBC, Luce’s employer, has a stake in the matter too—it is a leading provider of evidence-based medical research to manufacturers and others concerned with advancing medical technology.