The framework for quality assurance (QA) has a long history in healthcare, stretching back centuries since the time of Hippocrates.1 Medicine evolved into an evidence-based care culture, which is reflected in modern practice.2 QA in healthcare is one of the top priorities for most providers and health organizations. This is because healthcare organizations recognize that QA programs can help prevent, detect, and potentially correct any problems that may arise in patients, which will translate into higher quality and standards of care.3 Whereas QA is a structured and continuous program, quality assessment is the method for data collection and its subsequent analysis to which its results are compared to predefined standards.4 QA programs have evolved significantly over time and are described below, from surveys evaluating hospital performance to establishing metrics for the digital age, like telehealth.
In the United States, an effort to achieve high-quality standards was developed gradually. Between 1995 and 2000, several quality improvement initiatives were performed.5 Eventually, the Health Care Finance Administration (HCFA) identified quality indicators to routinely assess the excellence of care administered to its Medicare beneficiaries.6 These quality indicators were broken into six domains of healthcare: safety, effectiveness, timeliness, patient-centeredness, efficiency, and equity.7 The last three domains, as identified by the Institute of Medicine (IOM), are patient-centered and revolve around patient satisfaction or, in other words, patient experience. In general, quality indicators for healthcare are a vital tool used to track and evaluate the performance of an individual provider and a healthcare system. The specific indicators developed by HCFA focused on reviewing the impact of an intervention on the health status of a patient (e.g., percentage of patients diagnosed with hypertension with controlled blood pressures). Although this appeared to be the gold standard of assessing quality, relying solely on outcome measures to demonstrate therapeutic effectiveness has many drawbacks. Outcomes are the result of many contributing factors, many of which are out of an individual healthcare provider's control.
Parallel to the efforts of the HCFA, in 1994 the Healthcare Cost and Utilization Project (HCUP) created a set of quality indicators for hospitals, which was derived from discharge data. The Agency for Healthcare Research and Quality (AHRQ) maintains these commonly used indicators, which incorporate structural measures that address a provider's or a system's capacity (e.g., ratio of providers to patients) and process measures suggest what a provider does to improve health (e.g., percentage of patients who receive a preventive service like a screening colonoscopy).8 Although AHRQ Quality Indicators (Qis) are used widely, many organizations have developed and use their own sets of quality indicators (Table 10.1). Most recently, some organizations have been developing quality indicators for telehealth, which will be described later in this chapter.
Domain | Definition | Example Indicators |
---|---|---|
Prevention Quality Indicators (PQIs) | Calculates inpatient admissions that may have been avoided with access to outpatient care. |
|
Inpatient Quality Indicators (IQIs): Provider-Level Volume | Focuses on volume and measures the total amount of admissions and procedures performed. |
|
IQIs: Mortality Indicators | Includes indicators that cover procedures where higher mortality is associated with poorer care. |
|
IQIs: Mortality Indicators with Varying Mortality Rates | Describes procedures where volumes vary throughout the country. These indicators cover procedures where higher mortality rate is associated with poorer care. |
|
IQIs: Utilization Indicators | Includes indicators that measure high or low rates of specific procedures. High or low rates for these indicators represent inappropriate care delivery. |
|
Patient Safety Indicators (PSIs): Provider-level | Uses hospital inpatient discharge data to understand patient safety. Focuses on postoperative complications, adverse events, challenges with procedures, and trauma related to birth. |
|
After these quality indicators were developed, public reporting efforts began in 2006 under the Physician Quality Reporting Initiative (PQRI). Under the Tax Relief and Health Care Act (TRHCA) of 2006, PQRI enacted a pay-for-reporting system. This program focused on providing a formal mechanism for healthcare providers to evaluate and improve the quality of the care they provide to Medicare beneficiaries.10 The program provided incentive payments, which consisted of a 1.5% bond on Medicare Part B fee-for-service (FFS) charges for successful reporting on three quality measures. Then, in 2008, the Medicare Improvement for Patients and Providers Act (MIPPA) was enacted, and the incentive payments were increased to 2%.5 Although this was the beginning of formalized quality assurance programs, the Patient Protection and Accountable Care Act which was signed into law in 2010 highlighted the importance of a continuous quality program in healthcare.5 The overarching goal of this law was to improve quality of care, expand access, and lower healthcare costs for Medicare and Medicaid beneficiaries. To achieve this goal, the Patient-Centered Outcomes Research Institute (PCORI) was launched and began researching the effectiveness of various clinical interventions determining which treatments were the most effective for various patient populations. These findings guided evidence-based treatment algorithms to ensure high-quality care is maintained. In 2015, the Medicare Access and CHIP Reauthorization Act (MACRA) was passed. This law transitioned payments to the Merit-based Incentive Payment System (MIPS), which incorporates alternative payment models and merit-based payment systems.11 MIPS combines three of CMS's value-based programs: the physician quality reporting system, the value-based payment modifier, and the Medicare electronic health record (EHR) incentive program.12 Although this value-base model can be utilized across inpatient and outpatient facilities, because of this combination of reporting programs many outpatient providers and organizations have elected to enroll in MIPS payment models. Overall, 280 quality measures have been identified (Table 10.2).11
Heart failure (HF): beta-blocker therapy for left ventricular systolic dysfunction (LVSD) |
Urinary Incontinence: assessment of presence or absence of urinary incontinence in women aged 65 or older |
Urinary incontinence: plan of care for urinary incontinence in women aged 65 years or older |
Chronic obstructive pulmonary disease (COPD): long-acting inhaled bronchodilator therapy |
Prostate cancer: avoidance of overuse of bone scan for staging low-risk prostate cancer patients |
Prostate cancer: combination androgen deprivation therapy for high-risk or very high-risk prostate cancer |
Preventive care screening: body mass index (BMI) screening and follow-up plan |
Telehealth encounters have surged during the COVID-19 pandemic. In a study by Koonin and colleagues, telehealth visits increased substantially from 2019 through 2020. For example, in the first quarter of 2020, there was a 50% increase in telehealth visits compared to the same period in 2019.14 In another report by the Mayo Clinic, in-person visits declined a dramatic 78% during the COVID-19 pandemic from mid-March to mid-April. However, during that same time, telehealth was widely utilized, and the health system reported an astonishing 10,880% increase in this digital healthcare service.15 Although there has been a large effort to develop quality measures for in-person visits in inpatient and outpatient settings, these assessments for telehealth are lacking. Patients have generally responded positively to telehealth as a means of obtaining healthcare. Studies have demonstrated that patients are generally satisfied with telehealth services and report that saving time is a primary benefit of digital platforms.16,17 However, for all the purported benefits that telehealth has provided for patients and healthcare providers, the need to ensure that patients are deriving the same benefit from a digital health visit that they would receive from an in-person visit remains. Prior to the COVID-19 pandemic, several studies questioned if a telehealth platform is a more advantageous way to measure, track, and report the quality of care given to patients.18 But with the widespread adoption of telehealth as a means of managing acute and chronic health conditions, the question of quality assessment for this visit type remains even more paramount.
Patient experience is often tied to essential quality indicators. However, measuring the patient experience has been challenging. In 2008, with passage of MIPPA there was also a provision to assess the patient experience as part of the quality assessment. A variety of surveys have been used to measure patient encounters in both hospitals and outpatient clinics, but MIPPA tied these metrics to financial reimbursements, as detailed above.5,19 A variety of surveys have been used in the ambulatory setting such as the Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS) survey for outpatient surgery and the National Ambulator Medical Care Survey (NAMCS), or the Ambulatory Care Experiences Survey (ACES). Press Ganey's CG-CAHPS is one of the most widely used and validated patient experience surveys. It assesses the patient experience of adults and children in ambulatory or specialized care (Table 10.3). By weaving in the patient perspective with these quality indicators, the approach to quality becomes more holistic and allows care to be patient-centered and embody a team-based approach. Efforts like this provide national benchmarks for patient outcomes in a structured manner that assists healthcare professionals and organizations to provide high-quality evidence-based care to their patients.
Domain | Subdomain | Measure Concept |
---|---|---|
Experience | Patient, family, and/or caregiver | Overall improvement in quality of life because services are received at home |
Experience | Patient, family, and/or caregiver | Patients able to interpret diagnosis and treatment instructions through telehealth |
Experience | Patient, family, and/or caregiver | Decrease in wait times for patients |
Financial impact/cost | Financial impact on health system or payer | Visit duration measured versus in-home care |
Financial impact/cost | Financial impact on care team | Decrease in no-show rate |
Access to care | Access for care team | In-person visit agreed to after a telehealth consultation |
Access to care | Access for care team and for patient, family, and/or caregiver | Number of interprofessional visits |
Effectiveness | System effectiveness | Amount of time to schedule a visit |
Effectiveness | System effectiveness | Amount of time to check in for a visit |
Effectiveness | Clinical effectiveness | Relationship of the telehealth modality to the therapeutic need of the patient |
Effectiveness | Operational effectiveness | Can telehealth offer similar quality of services across a population of similar patients? |
Each using the scale: 1 Very poor 2 Poor 3 Fair 4 Good 5 Very good
Friendliness/courtesy of the care providers
Explanations the care provider gave you about your problem or condition
Concern the care provider showed for your questions or worries
Care provider's efforts to include you in decisions about your treatment
Information the care provider gave you about medications (if any)
Instructions the care provider gave you about follow-up care (if any)
Degree to which the care provider talked with you using words you could understand
Amount of time the care provider spent with you
Your confidence in this care provider
Likelihood of your recommending this care provider to others
Assessing the patient experience with telehealth across different demographics and socioeconomic situations is essential for organizations. In general, patients appear to be very satisfied with the telehealth platform to access their healthcare providers. In a retrospective single-center cohort study in New York City, 38,609 Press Ganey patient satisfaction surveys were evaluated and found that scores were significantly higher (94.9% vs. 92.5%; p <0.001) compared to in-person visits.21 Compared to in-person visits, there are several ways beyond the survey to assess patient experience with telehealth. For example, other key data points can be passively pulled which illuminate a patient's preferences to the telehealth platform. For example, identifying patient visits as either telephone only, video only, or in-person can help determine patient preferences. Organizations can also use custom patient survey questions to glean information on telehealth quality and the patient experience. By asking questions regarding time or money saved, access to technology and preference of using this technology can help organizations understand the patient experience with telehealth. An alternative to developing internal patient survey questions is to use established questionnaires to measure the patient experience. Commonly used validated surveys include the Telehealth Usability Questionnaire (TUQ) or the Telemedicine Satisfaction and Usefulness Questionnaire.22 These questionnaires assess usefulness, ease of use, quality of the telehealth interface, quality of the interaction with telehealth, reliability of the system, and patient experience.23,24
With this infrastructure, benchmark reporting, and methodology being set at a national level, it is critical that individual provider groups and organizations understand how to perform a clinical quality audit. The clinical audit is a key part of the quality improvement process, but it is also a measure of effectiveness. In general, it requires measuring the outcomes achieved against established standards, which were reviewed above. It is crucial to understand that the clinical audit goes beyond data collection. Successful clinical audits measure patient outcomes against standards or defined quality indicators, with the expectation of improving that practice. Clinical audits should be repeated frequently to ensure that any changes made to a practice demonstrate improvement.25 The first step to a successful clinical audit is to identify the topic to be evaluated. The ideal topic should be one that meets the following benchmarks:25
High volume of work
Risky
Highly variable
Complex
Potentially innovative
An overview of how to perform a clinical audit is drawn in Figure 10.1. Following proper identification of a topic, the scope of the project should be developed along with the corresponding methodology. The goal of the audit could be to evaluate a new process (e.g., patient adherence to warfarin using a telehealth platform) and/or improvement of a current process.26 Once the topic is identified and its corresponding methodology is developed, quality indicators should be selected. With these two key pieces of information defined, data collection can begin. This data collection may be performed either prospectively or retrospectively. The data itself can be quantitative or qualitative and include elements such as surveys, interviews, and questionnaires.27 Discrete data points from medical records like lab results or diagnosis codes can be included. A patient's medical record is primarily the main source of information for most clinical audits, but it is important to know that it may be lacking all the required data points needed.25 Following data collection, a comparison of your collected data with the benchmark standards must occur. This is the cornerstone of the clinical audit. Results from this comparison can demonstrate whether a standard is met and ultimately that further improvement is needed. If results are close to the standard used, then the clinical audit can enter a monitoring phase.25,27 Lastly, the cycle ends with change implementation. Following a change, it is important to go back and perform the clinical audit again to ensure that the changes made were positive and align with the standards previously identified.27
Identifying clinical outcome targets is as important as identifying quality outcomes. Although this is currently ill defined and the overall data are lacking, the rapid adoption of telehealth offers an opportunity to compare outcome measures following telehealth and in-person visits among patients with chronic health conditions like heart disease and diabetes. Although telehealth may offer opportunities to increase patient compliance, pharmacists must follow recommended and evidence-based practices when they provide care through this modality.
To perform clinical audits of telehealth, it is important that key indicators be abstracted in a chart review (Table 10.3). Having components like a telehealth consent and indicators about mode of technology used for a patient visit can make this abstraction easier. Lastly, it is crucial that organizations evaluate medication safety within the telehealth context. Reviewing charts for adverse events and creating voluntary reporting tools to catch medication errors related to telehealth are essential.
When you are determining which metrics to track for a telehealth service, it may be worthwhile to evaluate any metrics that are being tracked for a similar in-person service. Often, the goals of a service may be very similar regardless of whether the service is being delivered in person or by using telehealth. To date, very little has been published on the effectiveness of pharmacy services provided via traditional methods (i.e., face to face) compared with telehealth services.
Routine clinical quality audits are one piece of the quality assurance puzzle. A key component of quality assurance for any organization is ensuring safe practices for patients. One way to improve practice is through a medication safety review. Medication errors and medication-related adverse events have serious ramifications in patient care. Errors due to healthcare are the fourth leading cause of death in the United States and the leading cause of preventable death.29 Medication errors are also associated with increased length of hospital stays and additional costs. Therefore, to prevent erroneous morbidity and mortality from medication-related adverse events or errors, detection of these events is a paramount step.
On a national level, various regulatory agencies and federal organizations collect data and create standardized recommendations or alerts, such as the Food and Drug Administration Med-Watch program, the Joint Commission on the Accreditation of Healthcare Organizations, and the Institute of Safe Medication Practices. However, individual organizations must also develop tools to analyze errors and identify areas to improve quality and make system changes. In order to monitor safe medication processes, organizations must evaluate both adverse events and medication errors. However, the methods used to evaluate each of these categories varies. Most organizations use chart review to detect adverse events. By utilizing monitoring tools within the electronic health record, incident reporting, or auditing claims data, adverse events can be passively pulled and demonstrate clinical outcomes for the entire patient population in question.30 Medication errors are primarily detected by direct observation and voluntary reporting. Most organizations have employee reporting tools such as the Patient Safety Net (PSN) through which healthcare providers can summarize a patient event and each event undergoes a root cause analysis. Detection methods commonly used to assess medication errors are outlined in Table 10.4. Organizations can use these events to implement changes. Interestingly, the adoption of telemedicine may result in a reduction of medication and medical errors. A study using remote pharmacist review of medication orders for three community hospitals found an increase in interventions resulting in improved patient safety.31 By leveraging this technology, patient safety can be optimized in new ways; however, more data are needed to support patient safety benefits.
Method | Advantages | Disadvantages | Efficacy |
---|---|---|---|
Chart abstraction/review |
|
| Gold standard to detect adverse events |
Claims data |
|
| Adverse events detected |
Incident reporting (sentinel events) |
|
|
|
Voluntary reporting |
|
|
|
Computer monitoring |
|
|
|
Direct care observation |
|
|
|
Patient monitoring |
|
|
|
At the time of this writing, standardized quality assessment and assurance approaches to telehealth pharmacy practice models have largely not been developed. In fact, national conversations about this topic are ongoing. However, thanks to the above-outlined robust quality initiatives that have taken place over the past three decades, we can incorporate existing approaches and modify them to a telehealth pharmacy practice visit. To perform a quality assessment, it is imperative that quality indicators be identified. Nonprofits like the National Quality Forum (NQF) are attempting to develop standardized quality indicators for telehealth that focus on access to care, financial impacts or cost, the telehealth experience, and effectiveness of treatment. However, benchmarks are currently not well defined and need to be further developed.
De Jonge V, Sint Nicolaas J, van Leerdam M, et al.Overview of the quality assurance movement in health care. Best Pract Res Clin Gastroenterol. 2011;25(3):337-347. doi: 10.1016/j.bpg.2011.05.001.
Guyatt G, Cook D, Haynes B. Evidence based medicine has come a long way. BMJ. 2004;329(7473):990-991. doi: 10.1136/bmj.329.7473.990.
Institute of Medicine (US) Committee to Design a Strategy for Quality Review and Assurance in Medicare; Lohr KN (ed). Medicare: A Strategy for Quality Assurance: Volume 1. Washington, DC: National Academies Press, 1990. 9, Methods of Quality Assessment and Assurance. https://www.ncbi.nlm.nih.gov/books/NBK235459.
Mainz J, Hammershøy EM, Worning AM, et al.Quality assessment, quality assurance and quality improvement in health care. Concepts and terminology. Ugeskr Laeger. 1992;154(17):1151-1156.
Marjoua Y, Bozic K. Brief history of quality movement in US healthcare. Curr Rev Musculoskelet Med. 2012;5(4):265-273. doi: 10.1007/s12178-012-9137-8.
Jencks SF, Cuerdon T, Burwen DR, et al.Quality of medical care delivered to Medicare beneficiaries: A profile at state and national levels. JAMA. 2000;284:1670-1676. doi: 10.1001/jama.284.13.1670.
Berwick DM. A user's manual for the IOM's “Quality Chasm” Report. Health Aff. 2002;21(3):80-90. doi: 10.1377/hlthaff.21.3.80.
Types of health care quality measures. (n.d.). https://www.ahrq.gov/talkingquality/measures/types.html. Accessed on May 18, 2021.
Farquhar M. AHRQ Quality indicators. In: Hughes RG (ed). Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality; 2008 Apr. Chapter 45. https://www.ncbi.nlm.nih.gov/books/NBK2664. Accessed May 18, 2021.
Dowd B, Li CH, Swenson T, et al.Medicare's Physician Quality Reporting System (PQRS): Quality measurement and beneficiary attribution. Medicare Medicaid Res Rev. 2014 Jun 25;4(2). doi: 10.5600/mmrr.004.02.a04.
Manchikanti L, Hammer M, Benyamin R, et al.Merit-based incentive payment system (MIPS): Harsh choices for interventional pain management physicians. Pain Physician. 2016;19:E15-E32.
Yon K, Sinclair J, Bentley OS, et al.Impact of quality measures performed through pharmacist collaboration with a primary care clinic. J Am Pharm Assoc. 2003; 2020;60(3 S): S97-S102. https://doi.org/10.1016/j.japh.2020.01.011.
2020 Annual Call for Quality Measures Fact Sheet. (2020, May 22). Retrieved May 06, 2021. https://www.cms.gov/files/document/2020-mips-call-quality-measure-overview-fact-sheet.pdf
Koonin L, Hoots B, Tsang C, et al.Trends in the use of telehealth during the emergence of the COVID-19 pandemic—United States, January-March 2020. Weekly. 2020;69(43):1595-1599.
Demaerschalk B, Blegen R, Ommen S. Scalability of telemedicine services in a large integrated multispecialty health care system during COVID-19. Telemed e-Health. 2021:96-98. http://doi.org/10.1089/tmj.2020.0290.
Polinski JM, Barker T, Gagliano N, et al.Patients’ satisfaction with and preference for telehealth visits. J Gen Intern Med. 2016 Mar;31(3):269-275. doi: 10.1007/s11606-015-3489-x.
Jacobs J, Ferguson J, Van Campen J, et al.Organizational and external factors associated with video telehealth use in the veterans’ health administration before and during the COVID-19 pandemic. Telemedicine and e-Health. 2021. doi: http://doi.org/10.1089/tmj.2020.0530.
Halpren-Ruder D, Chang A, Hollander J, et al.Quality assurance in telehealth: Adherence to evidence-based indicators. Telemed J E Health. 2019;25(7):599-603. doi: 10.1089/tmj.2018.0149.
Mazurenko O, Collum T, Ferdinand A, et al.Predictors of hospital patient satisfaction as measured by HCAHPS. J Healthcare Manag. 2017;62(4):272-283. doi: 10.1097/JHM-D-15-00050.
An overview of Version 3.0 of the CAHPS Clinician & Group Survey. (2015, June 17). Retrieved May 6, 2021. https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance/cg/about/cg_3-0_overview.pdf.
Ramaswamy A, Yu M, Drrangsholt S, et al.Patient satisfaction with telemedicine during the COVID-19 pandemic: Retrospective cohort study. J Med Internet Res 2020;22(9):e20786. doi: 10.2196/20786.
Chuo J, Macy ML, Lorch SA. Strategies for evaluating telehealth. Pediatrics 2020 Nov;146(5):e20201781. doi: 10.1542/peds.2020-1781.
Parmanto B, Lewis Jr A, Graham K, et al.Development of the telehealth usability questionnaire (TUQ). Int J Telerehabil. 2016;8(1);3-10. doi: 10.5195/ijt.2016.6196.
Bakken S, Grullon-Figueroa L, Izquierdo R, et al.Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. J Am Med Inform Assoc. 2006l 13(6):660-667. doi: 10.1197/jamia.M2146.
Esposito P, Dal Canton A. Clinical audit, a valuable tool to improve quality of care: General methodology and applications in nephrology. World J Nephrol. 2014;3(4):249-255. doi: 10.5527/wjn.v3.i4.249.
Buth S, D’Cruz L. Clinical audit—process and outcome for improved clinical practice. Dent Update. 2012;39:710-714. doi: 10.12968/denu.2012.39.10.710.
Soimakallio S, Alanen A, Jarvinen H, et al.Clinical audit: Development of the criteria of good practices. Radiat Prot Dosimetry. 2011;147(1-2):30-33. doi: 10.1093/rpd/ncr260.
Creating a framework to support measure development for telehealth. (2017, August). http://www.qualityforum.org/Publications/2017/08/Creating_a_Framework_to_Support_Measure_Development_for_Telehealth.aspx. Accessed May 18, 2021.
Light D, Lexchin J, Darrow J. Institutional corruption of pharmaceuticals and the myth of safe and effective drugs. J Law, Med Ethics. 2013;14(3):590-610. doi: 10.1111/jlme.12068.
Montesi G, Lechi A. Prevention of medication errors: Detection and audit. Br J Clin Pharmacol. 2009;67(6):651-655. doi: 10.1111/j.1365-2125.2009.03422.x.
Schneider PJ. Evaluating the impact of telepharmacy. Am J Health Syst Pharm. 2013;70:2130-2135. doi: 10.2146/ajhp130138.