Quality, not quantity, in observational research

Fig. 1: Review of first 24 Months of Pharmacovigilance Risk Assessment Committee Oversight

Few would argue

that high-quality observational research is needed to meet real-world evidence needs. In late 2014, Quintiles polled 135 biopharma customers to ask what they considered to be the most challenging aspect of providing evidence to health technology assessment (HTA) decision-makers. There were three response choices: understanding the diversity of stakeholder needs, varying real-world data access and availability, and generating quality evidence at the right time at the right cost. Most customers chose generating quality evidence at the right time at the right cost to be their greatest challenge.

However, while the need for quality is broadly acknowledged, its definition in the context of observational research is not always clear. This article offers a framework designed to deliver consistent quality in observational research based on five key features:

• Study design that provides data relevant to all key healthcare system stakeholders
• Ethical conduct
• Feasible based on the availability of time, budget and data
• Specialized in operational delivery, weaving together strategy, science and operations
• Results are communicated to stakeholders before, during and after study completion.

Why “quality” not “quantity”? There’s tremendous excitement in healthcare over the growing availability of “Big Data” including administrative claims and electronic medical records. In parallel, observational research leveraging secondary and primary data sources is becoming more widely used by healthcare decision-makers. As such, the expectation of what observational research can deliver is huge, and it has never been more important to deliver high-quality observational research to meet the demand for real-world evidence. However, the challenges inherent in observational research remain: selection bias, information bias, confounding, and unreliable or missing data. Delivering studies based on large numbers of people using poor-quality data may, at best, waste healthcare spend and, at worst, lead to poor healthcare decisionmaking.

Fig. 2: Key evidence requirements of payers

Relevance to stakeholders

Research by biopharma companies must meet the information needs of all healthcare stakeholders, including regulators, payers, physicians and patients.

Regulators—with a primary focus on the detection of safety signals and ensuring long-term effectiveness–require well-designed, scientifically sound studies. A Quintiles review of decisions made by the European Medicines Agency’s Pharmacovigilance Risk Assessment Committee (PRAC; Fig. 1) indicates that many Post-Authorisation Safety Study (PASS) protocols have been rejected due to issues such as: inadequate study design to meet objectives, sample size, statistical approach, missing timelines, enrollment strategy, representativeness, bias and confounding, and inappropriate design. In the US, big data has been a mainstay of drug safety using programs such as the MiniSentinel, which relies largely on administrative health insurance claims data and some electronic medical record information, but also has the capacity for supplementary prospective data collection. Further, FDA has shown its interest in prospective observational data as evidenced by granting line extensions based on registry data. For example, a recent line extension within the same extension was granted for bevacizumab using data from one clinical trial and two registry-based studies.

Payers bring a different perspective, one focused on determining value and coverage, monitoring usage within specific criteria, and ensuring cost effectiveness. HTA bodies have a common aim of determining the value of new technologies; however, there is heterogeneity among the agencies in how they operate and how different types of information influence decisionmaking. The balance between central and local decision-makers varies by market, as does the primary notion of value. This is demonstrated by a survey conducted by the European network for Health Technology Assessment, EUnetTHA, which found that all payers want to know whether a drug works and is safe, and many also examine value (Fig. 2). Real-world evidence is often required either as part of longer term post-market safety/efficacy monitoring or for conditional coverage and reimbursement re-evaluation. In addition, real-world evidence is needed for targeting differentiation and uptake at regional payer level or with providers who are increasingly involved in financial management and care pathways/efficiency. In the US, there is also a longstanding interest in quantitative evidence generation to support new trends that are starting to examine not just whether a particular treatment or diagnostic works, but how much of a difference it makes. For example, the National Oncologic PET Registry was used to evaluate the impact of PET scanning on cancer diagnosis and how much it affected physician decisionmaking. [1]

Fig. 3: Payer demands for benefit over standard of care [2]

Payers also want to see superiority over the existing standard of care. A recent analysis of HTA reports in non-small cell lung cancer (NSCLC) identified a strong preference for payers to demand benefit over existing standard of care in the primary endpoint (Fig. 3).

Publicly available data from UK National Institute for Care Excellence (NICE) oncology assessments indicate that payers are looking for hard endpoints, such as overall survival versus progression-free survival for oncology products. Payers also want to see evidence in the right patient population, with differences between the trial population and the requested reimbursement indication being the most frequently reported critique. Comparator selection is also critical for this stakeholder group.

In terms of evidence demands, physicians wish to obtain locally relevant evidence, advance science, improve patient care, ensure continued reimbursement and generate publications. Patients want to understand their treatment options, the associated risks and benefits, impacts on quality of life, convenience of administration, and affordability.

These diverse information needs should be reflected in biopharma companies’ research strategy, as they work to build a drug’s safety profile, evaluate efficacy to improve patient outcomes, prove value, secure reimbursement, enhance understanding of unmet patient needs, explore new indications, generate publications, and inform the product’s brand strategy.

Ethical research conduct

Due to their observational nature, epidemiological studies in the past were widely regarded as not raising any significant ethical issues and were commonly carried out without approval of an ethical review committee. This is no longer the case and the International Ethical Guidelines for Epidemiological Studies, prepared in 2009, call out the need to submit observational studies to ethical committees for review. [3] Prepared by the Council for International Organizations of Medical Sciences (CIOMS) in collaboration with the World Health Organization (WHO), the guidelines cover both observational and interventional research. They were designed for use, particularly by low-resource countries, in defining national policies on the ethics of biomedical research (and particularly in clinical trials of pharmaceuticals).

Observational studies usually require scientific and ethical review, although some observational studies, such as those utilizing publicly available or anonymous data, may not be subject to prior ethical review and approval under local regulations. If in doubt about whether a study needs ethical review, sponsors are encouraged to consult an ethical review committee or to submit their studies for review. Any observational study that is not submitted to an ethical review committee should receive administrative confirmation by a competent authority that the study is exempt from review. Also, physicians participating in observational studies should receive incentives commensurate with fair market value for the work performed, and payments should not represent an inducement to prescribe.

While observational research normally does not pose a risk of physical harm to individuals, this is not always the case. For example, in some nonexperimental studies, researchers may take blood or tissue samples; and even when an observational study involves only questionnaires or examination of medical records, subjects may be at risk of physical or psychological harm. Informed consent is needed when the research poses a risk for patients, and is normally required if the study involves data containing personal identifiers.

When populations and communities with limited resources are involved, any research should be responsive to the health needs and priorities of the participating population or community. In addition, any intervention, product developed or knowledge generated, must be made reasonably available for the benefit of these participants. Study sponsors should be aware that if a patient is switched from an affordable standard of care to an unaffordable new treatment in an observational study, the research may be characterized as exploitative and, therefore, unethical.

There is an ethical obligation to disseminate any study findings that are of potential scientific or public health importance. This should take place in a timely fashion through publications in the scientific literature and presentations at scientific conferences, workshops or symposia. It should be noted that scientific journals generally require that papers submitted for publication have received prior review by an ethical review committee. Authorship of study reports should follow the guidelines established by the International Committee of Medical Journal Editors, [4] and any potential conflicts of interest, financial and nonfinancial, should be disclosed. Agreement to adhere to these guidelines should be included in the study protocol.

ABOUT THE AUTHOR

Louise Parmenter, PhD is Global Head of Operations, Epidemiology & Outcomes Research, at Quintiles.

References

• Larson SM. Practice-Based Evidence of the Beneficial Impact of Positron Emission Tomography in Clinical Oncology. JCO May 1, 2008 vol. 26 no. 13 2083-2084; http://jco.ascopubs.org/content/26/13/2083.full
• Source: Quintiles HTA Accelerator
• http://www.ufrgs.br/bioetica/cioms2008.pdf
• http://www.icmje.org/