The framework for quality assurance (QA) has a long history in healthcare, stretching back centuries since the time of Hippocrates.1 Medicine evolved into an evidence-based care culture, which is reflected in modern practice.2 QA in healthcare is one of the top priorities for most providers and health organizations. This is because healthcare organizations recognize that QA programs can help prevent, detect, and potentially correct any problems that may arise in patients, which will translate into higher quality and standards of care.3 Whereas QA is a structured and continuous program, quality assessment is the method for data collection and its subsequent analysis to which its results are compared to predefined standards.4 QA programs have evolved significantly over time and are described below, from surveys evaluating hospital performance to establishing metrics for the digital age, like telehealth.
In the United States, an effort to achieve high-quality standards was developed gradually. Between 1995 and 2000, several quality improvement initiatives were performed.5 Eventually, the Health Care Finance Administration (HCFA) identified quality indicators to routinely assess the excellence of care administered to its Medicare beneficiaries.6 These quality indicators were broken into six domains of healthcare: safety, effectiveness, timeliness, patient-centeredness, efficiency, and equity.7 The last three domains, as identified by the Institute of Medicine (IOM), are patient-centered and revolve around patient satisfaction or, in other words, patient experience. In general, quality indicators for healthcare are a vital tool used to track and evaluate the performance of an individual provider and a healthcare system. The specific indicators developed by HCFA focused on reviewing the impact of an intervention on the health status of a patient (e.g., percentage of patients diagnosed with hypertension with controlled blood pressures). Although this appeared to be the gold standard of assessing quality, relying solely on outcome measures to demonstrate therapeutic effectiveness has many drawbacks. Outcomes are the result of many contributing factors, many of which are out of an individual healthcare provider's control.
Parallel to the efforts of the HCFA, in 1994 the Healthcare Cost and Utilization Project (HCUP) created a set of quality indicators for hospitals, which was derived from discharge data. The Agency for Healthcare Research and Quality (AHRQ) maintains these commonly used indicators, which incorporate structural measures that address a provider's or a system's capacity (e.g., ratio of providers to patients) and process measures suggest what a provider does to improve health (e.g., percentage of patients who receive a preventive service like a screening colonoscopy).8 Although AHRQ Quality Indicators (Qis) are used widely, many organizations have developed and use their own sets of quality indicators (Table 10.1). Most recently, some organizations have been developing quality indicators for telehealth, which will be described later in this chapter.
AHRQ Quality Indicator Modules for Adult Patients9
Prevention Quality Indicators (PQIs)
Calculates inpatient admissions that may have been avoided with access to outpatient care.
Obstetric trauma due to vaginal delivery with instrument
Obstetric trauma from a vaginal delivery without an instrument
Obstetric trauma related to cesarean section delivery
Complication of anesthesia
Death in low-mortality diagnosis-related groups (DRGs)
Death in surgical inpatients with treatable serious complications (a previously known failure to rescue)
Transfusion reactions (AB/Rh)
After these quality indicators were developed, public reporting efforts began in 2006 under the Physician Quality Reporting Initiative (PQRI). Under the Tax Relief and Health Care Act (TRHCA) of 2006, PQRI enacted a pay-for-reporting system. This program focused on providing a formal mechanism for healthcare providers to evaluate and improve the quality of the care they provide to Medicare beneficiaries.10 The program provided incentive payments, which consisted of a 1.5% bond on Medicare Part B fee-for-service (FFS) charges for successful reporting on three quality measures. Then, in 2008, the Medicare Improvement for Patients and Providers Act (MIPPA) was enacted, and the incentive payments were increased to 2%.5 Although this was the beginning of formalized quality assurance programs, the Patient Protection and Accountable Care Act which was signed into law in 2010 highlighted the importance of a continuous quality program in healthcare.5 The overarching goal of this law was to improve quality of care, expand access, and lower healthcare costs for Medicare and Medicaid beneficiaries. To achieve this goal, the Patient-Centered Outcomes Research Institute (PCORI) was launched and began researching the effectiveness of various clinical interventions determining which treatments were the most effective for various patient populations. These findings guided evidence-based treatment algorithms to ensure high-quality care is maintained. In 2015, the Medicare Access and CHIP Reauthorization Act (MACRA) was passed. This law transitioned payments to the Merit-based Incentive Payment System (MIPS), which incorporates alternative payment models and merit-based payment systems.11 MIPS combines three of CMS's value-based programs: the physician quality reporting system, the value-based payment modifier, and the Medicare electronic health record (EHR) incentive program.12 Although this value-base model can be utilized across inpatient and outpatient facilities, because of this combination of reporting programs many outpatient providers and organizations have elected to enroll in MIPS payment models. Overall, 280 quality measures have been identified (Table 10.2).11
Prostate cancer: avoidance of overuse of bone scan for staging low-risk prostate cancer patients
Prostate cancer: combination androgen deprivation therapy for high-risk or very high-risk prostate cancer
Preventive care screening: body mass index (BMI) screening and follow-up plan
TELEHEALTH AND QUALITY ASSURANCE STRATEGIES
Telehealth encounters have surged during the COVID-19 pandemic. In a study by Koonin and colleagues, telehealth visits increased substantially from 2019 through 2020. For example, in the first quarter of 2020, there was a 50% increase in telehealth visits compared to the same period in 2019.14 In another report by the Mayo Clinic, in-person visits declined a dramatic 78% during the COVID-19 pandemic from mid-March to mid-April. However, during that same time, telehealth was widely utilized, and the health system reported an astonishing 10,880% increase in this digital healthcare service.15 Although there has been a large effort to develop quality measures for in-person visits in inpatient and outpatient settings, these assessments for telehealth are lacking. Patients have generally responded positively to telehealth as a means of obtaining healthcare. Studies have demonstrated that patients are generally satisfied with telehealth services and report that saving time is a primary benefit of digital platforms.16,17 However, for all the purported benefits that telehealth has provided for patients and healthcare providers, the need to ensure that patients are deriving the same benefit from a digital health visit that they would receive from an in-person visit remains. Prior to the COVID-19 pandemic, several studies questioned if a telehealth platform is a more advantageous way to measure, track, and report the quality of care given to patients.18 But with the widespread adoption of telehealth as a means of managing acute and chronic health conditions, the question of quality assessment for this visit type remains even more paramount.
Patient experience is often tied to essential quality indicators. However, measuring the patient experience has been challenging. In 2008, with passage of MIPPA there was also a provision to assess the patient experience as part of the quality assessment. A variety of surveys have been used to measure patient encounters in both hospitals and outpatient clinics, but MIPPA tied these metrics to financial reimbursements, as detailed above.5,19 A variety of surveys have been used in the ambulatory setting such as the Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS) survey for outpatient surgery and the National Ambulator Medical Care Survey (NAMCS), or the Ambulatory Care Experiences Survey (ACES). Press Ganey's CG-CAHPS is one of the most widely used and validated patient experience surveys. It assesses the patient experience of adults and children in ambulatory or specialized care (Table 10.3). By weaving in the patient perspective with these quality indicators, the approach to quality becomes more holistic and allows care to be patient-centered and embody a team-based approach. Efforts like this provide national benchmarks for patient outcomes in a structured manner that assists healthcare professionals and organizations to provide high-quality evidence-based care to their patients.
Select Quality Indicators for Telehealth: NQF Recommendations28
Patient, family, and/or caregiver
Overall improvement in quality of life because services are received at home
Patient, family, and/or caregiver
Patients able to interpret diagnosis and treatment instructions through telehealth
Patient, family, and/or caregiver
Decrease in wait times for patients
Financial impact on health system or payer
Visit duration measured versus in-home care
Financial impact on care team
Decrease in no-show rate
Access to care
Access for care team
In-person visit agreed to after a telehealth consultation
Access to care
Access for care team and for patient, family, and/or caregiver
Number of interprofessional visits
Amount of time to schedule a visit
Amount of time to check in for a visit
Relationship of the telehealth modality to the therapeutic need of the patient
Can telehealth offer similar quality of services across a population of similar patients?
Each using the scale: 1 Very poor 2 Poor 3 Fair 4 Good 5 Very good
Friendliness/courtesy of the care providers
Explanations the care provider gave you about your problem or condition
Concern the care provider showed for your questions or worries
Care provider's efforts to include you in decisions about your treatment
Information the care provider gave you about medications (if any)
Instructions the care provider gave you about follow-up care (if any)
Degree to which the care provider talked with you using words you could understand
Amount of time the care provider spent with you
Your confidence in this care provider
Likelihood of your recommending this care provider to others
Telehealth Pharmacy Practice Patient Experience
Assessing the patient experience with telehealth across different demographics and socioeconomic situations is essential for organizations. In general, patients appear to be very satisfied with the telehealth platform to access their healthcare providers. In a retrospective single-center cohort study in New York City, 38,609 Press Ganey patient satisfaction surveys were evaluated and found that scores were significantly higher (94.9% vs. 92.5%; p <0.001) compared to in-person visits.21 Compared to in-person visits, there are several ways beyond the survey to assess patient experience with telehealth. For example, other key data points can be passively pulled which illuminate a patient's preferences to the telehealth platform. For example, identifying patient visits as either telephone only, video only, or in-person can help determine patient preferences. Organizations can also use custom patient survey questions to glean information on telehealth quality and the patient experience. By asking questions regarding time or money saved, access to technology and preference of using this technology can help organizations understand the patient experience with telehealth. An alternative to developing internal patient survey questions is to use established questionnaires to measure the patient experience. Commonly used validated surveys include the Telehealth Usability Questionnaire (TUQ) or the Telemedicine Satisfaction and Usefulness Questionnaire.22 These questionnaires assess usefulness, ease of use, quality of the telehealth interface, quality of the interaction with telehealth, reliability of the system, and patient experience.23,24
Clinical Quality Audits
With this infrastructure, benchmark reporting, and methodology being set at a national level, it is critical that individual provider groups and organizations understand how to perform a clinical quality audit. The clinical audit is a key part of the quality improvement process, but it is also a measure of effectiveness. In general, it requires measuring the outcomes achieved against established standards, which were reviewed above. It is crucial to understand that the clinical audit goes beyond data collection. Successful clinical audits measure patient outcomes against standards or defined quality indicators, with the expectation of improving that practice. Clinical audits should be repeated frequently to ensure that any changes made to a practice demonstrate improvement.25 The first step to a successful clinical audit is to identify the topic to be evaluated. The ideal topic should be one that meets the following benchmarks:25
High volume of work
An overview of how to perform a clinical audit is drawn in Figure 10.1. Following proper identification of a topic, the scope of the project should be developed along with the corresponding methodology. The goal of the audit could be to evaluate a new process (e.g., patient adherence to warfarin using a telehealth platform) and/or improvement of a current process.26 Once the topic is identified and its corresponding methodology is developed, quality indicators should be selected. With these two key pieces of information defined, data collection can begin. This data collection may be performed either prospectively or retrospectively. The data itself can be quantitative or qualitative and include elements such as surveys, interviews, and questionnaires.27 Discrete data points from medical records like lab results or diagnosis codes can be included. A patient's medical record is primarily the main source of information for most clinical audits, but it is important to know that it may be lacking all the required data points needed.25 Following data collection, a comparison of your collected data with the benchmark standards must occur. This is the cornerstone of the clinical audit. Results from this comparison can demonstrate whether a standard is met and ultimately that further improvement is needed. If results are close to the standard used, then the clinical audit can enter a monitoring phase.25,27 Lastly, the cycle ends with change implementation. Following a change, it is important to go back and perform the clinical audit again to ensure that the changes made were positive and align with the standards previously identified.27
Telehealth Clinical Quality Audits
Identifying clinical outcome targets is as important as identifying quality outcomes. Although this is currently ill defined and the overall data are lacking, the rapid adoption of telehealth offers an opportunity to compare outcome measures following telehealth and in-person visits among patients with chronic health conditions like heart disease and diabetes. Although telehealth may offer opportunities to increase patient compliance, pharmacists must follow recommended and evidence-based practices when they provide care through this modality.
To perform clinical audits of telehealth, it is important that key indicators be abstracted in a chart review (Table 10.3). Having components like a telehealth consent and indicators about mode of technology used for a patient visit can make this abstraction easier. Lastly, it is crucial that organizations evaluate medication safety within the telehealth context. Reviewing charts for adverse events and creating voluntary reporting tools to catch medication errors related to telehealth are essential.
When you are determining which metrics to track for a telehealth service, it may be worthwhile to evaluate any metrics that are being tracked for a similar in-person service. Often, the goals of a service may be very similar regardless of whether the service is being delivered in person or by using telehealth. To date, very little has been published on the effectiveness of pharmacy services provided via traditional methods (i.e., face to face) compared with telehealth services.
Routine clinical quality audits are one piece of the quality assurance puzzle. A key component of quality assurance for any organization is ensuring safe practices for patients. One way to improve practice is through a medication safety review. Medication errors and medication-related adverse events have serious ramifications in patient care. Errors due to healthcare are the fourth leading cause of death in the United States and the leading cause of preventable death.29 Medication errors are also associated with increased length of hospital stays and additional costs. Therefore, to prevent erroneous morbidity and mortality from medication-related adverse events or errors, detection of these events is a paramount step.
On a national level, various regulatory agencies and federal organizations collect data and create standardized recommendations or alerts, such as the Food and Drug Administration Med-Watch program, the Joint Commission on the Accreditation of Healthcare Organizations, and the Institute of Safe Medication Practices. However, individual organizations must also develop tools to analyze errors and identify areas to improve quality and make system changes. In order to monitor safe medication processes, organizations must evaluate both adverse events and medication errors. However, the methods used to evaluate each of these categories varies. Most organizations use chart review to detect adverse events. By utilizing monitoring tools within the electronic health record, incident reporting, or auditing claims data, adverse events can be passively pulled and demonstrate clinical outcomes for the entire patient population in question.30 Medication errors are primarily detected by direct observation and voluntary reporting. Most organizations have employee reporting tools such as the Patient Safety Net (PSN) through which healthcare providers can summarize a patient event and each event undergoes a root cause analysis. Detection methods commonly used to assess medication errors are outlined in Table 10.4. Organizations can use these events to implement changes. Interestingly, the adoption of telemedicine may result in a reduction of medication and medical errors. A study using remote pharmacist review of medication orders for three community hospitals found an increase in interventions resulting in improved patient safety.31 By leveraging this technology, patient safety can be optimized in new ways; however, more data are needed to support patient safety benefits.
Commonly Used Methods to Assess Medication Errors and Adverse Events30
Uses standardized criteria
Gold standard to detect adverse events
Uses local data
Captures unapparent failures
Litigation based and may have legal implications
Adverse events detected
Incident reporting (sentinel events)
Captures active and unapparent failures
Only identifies severe, unexplained events or deaths
Can underestimate rates of adverse events
May cause feelings of blame, fear of punishment
Contributes to reports and alerting
Measures adverse events
Variety of sources can report events
Uses a structured and simplified form
Captures active and unapparent failures
Promotes a culture of safety
Variable quality of reporting
Individuals may feel the tool contributes to a blame culture
No data integration
Contributes to reports and alerting
Provides path for feedback and opportunities for corrective actions
Medication errors are detected
Multidata source integration
Adverse event prevention
Undetermined future risks
Inadvertent errors in data inserted
Captures prescribing faults
Captures prescription errors
Captures dispensing errors
Direct care observation
Captures active errors
Difficult to train
Data are centralized around administration errors
Data from outpatients
Tools are not standardized and vary (interviews, questionnaires, focus groups, etc.)
Can help direct future developments of patient-centered care
Maintaining Quality and Safety with Pharmacy Telehealth Practice
At the time of this writing, standardized quality assessment and assurance approaches to telehealth pharmacy practice models have largely not been developed. In fact, national conversations about this topic are ongoing. However, thanks to the above-outlined robust quality initiatives that have taken place over the past three decades, we can incorporate existing approaches and modify them to a telehealth pharmacy practice visit. To perform a quality assessment, it is imperative that quality indicators be identified. Nonprofits like the National Quality Forum (NQF) are attempting to develop standardized quality indicators for telehealth that focus on access to care, financial impacts or cost, the telehealth experience, and effectiveness of treatment. However, benchmarks are currently not well defined and need to be further developed.
De JongeV, Sint NicolaasJ, van LeerdamM, et al.Overview of the quality assurance movement in health care. Best Pract Res Clin Gastroenterol. 2011;25(3):337-347. doi: 10.1016/j.bpg.2011.05.001.10.1016/j.bpg.2011.05.001)| false
Institute of Medicine (US) Committee to Design a Strategy for Quality Review and Assurance in Medicare; LohrKN (ed). Medicare: A Strategy for Quality Assurance: Volume 1. Washington, DC: National Academies Press, 1990. 9, Methods of Quality Assessment and Assurance. https://www.ncbi.nlm.nih.gov/books/NBK235459.
Institute of Medicine (US) Committee to Design a Strategy for Quality Review and Assurance in Medicare; LohrKN (ed). Medicare: A Strategy for Quality Assurance: Volume 1. Washington, DC: National Academies Press, 1990. 9, Methods of Quality Assessment and Assurance. https://www.ncbi.nlm.nih.gov/books/NBK235459.)| false
JencksSF, CuerdonT, BurwenDR, et al.Quality of medical care delivered to Medicare beneficiaries: A profile at state and national levels. JAMA. 2000;284:1670-1676. doi: 10.1001/jama.284.13.1670.10.1001/jama.284.13.1670)| false
FarquharM.AHRQ Quality indicators. In: HughesRG (ed). Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality; 2008 Apr. Chapter 45. https://www.ncbi.nlm.nih.gov/books/NBK2664. Accessed May 18, 2021.
FarquharM.AHRQ Quality indicators. In: HughesRG (ed). Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality; 2008 Apr. Chapter 45. https://www.ncbi.nlm.nih.gov/books/NBK2664. Accessed May 18, 2021.)| false
DowdB, LiCH, SwensonT, et al.Medicare's Physician Quality Reporting System (PQRS): Quality measurement and beneficiary attribution. Medicare Medicaid Res Rev. 2014 Jun 25;4(2). doi: 10.5600/mmrr.004.02.a04.
DowdB, LiCH, SwensonT, et al.Medicare's Physician Quality Reporting System (PQRS): Quality measurement and beneficiary attribution. Medicare Medicaid Res Rev. 2014 Jun 25;4(2). doi: 10.5600/mmrr.004.02.a04.10.5600/mmrr.004.02.a04)| false
YonK, SinclairJ, BentleyOS, et al.Impact of quality measures performed through pharmacist collaboration with a primary care clinic. J Am Pharm Assoc. 2003; 2020;60(3 S): S97-S102. https://doi.org/10.1016/j.japh.2020.01.011.
YonK, SinclairJ, BentleyOS, et al.Impact of quality measures performed through pharmacist collaboration with a primary care clinic. J Am Pharm Assoc. 2003; 2020;60(3 S): S97-S102. https://doi.org/10.1016/j.japh.2020.01.011.10.1016/j.japh.2020.01.011)| false
DemaerschalkB, BlegenR, OmmenS.Scalability of telemedicine services in a large integrated multispecialty health care system during COVID-19. Telemed e-Health. 2021:96-98. http://doi.org/10.1089/tmj.2020.0290.
DemaerschalkB, BlegenR, OmmenS.Scalability of telemedicine services in a large integrated multispecialty health care system during COVID-19. Telemed e-Health. 2021:96-98. http://doi.org/10.1089/tmj.2020.0290.)| false
PolinskiJM, BarkerT, GaglianoN, et al.Patients’ satisfaction with and preference for telehealth visits. J Gen Intern Med. 2016 Mar;31(3):269-275. doi: 10.1007/s11606-015-3489-x.10.1007/s11606-015-3489-x)| false
JacobsJ, FergusonJ, Van CampenJ, et al.Organizational and external factors associated with video telehealth use in the veterans’ health administration before and during the COVID-19 pandemic. Telemedicine and e-Health. 2021. doi: http://doi.org/10.1089/tmj.2020.0530.
JacobsJ, FergusonJ, Van CampenJ, et al.Organizational and external factors associated with video telehealth use in the veterans’ health administration before and during the COVID-19 pandemic. Telemedicine and e-Health. 2021. doi: http://doi.org/10.1089/tmj.2020.0530.)| false
Halpren-RuderD, ChangA, HollanderJ, et al.Quality assurance in telehealth: Adherence to evidence-based indicators. Telemed J E Health. 2019;25(7):599-603. doi: 10.1089/tmj.2018.0149.10.1089/tmj.2018.0149)| false
MazurenkoO, CollumT, FerdinandA, et al.Predictors of hospital patient satisfaction as measured by HCAHPS. J Healthcare Manag. 2017;62(4):272-283. doi: 10.1097/JHM-D-15-00050.10.1097/JHM-D-15-00050)| false
RamaswamyA, YuM, DrrangsholtS, et al.Patient satisfaction with telemedicine during the COVID-19 pandemic: Retrospective cohort study. J Med Internet Res 2020;22(9):e20786. doi: 10.2196/20786.10.2196/20786)| false
BakkenS, Grullon-FigueroaL, IzquierdoR, et al.Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. J Am Med Inform Assoc. 2006l 13(6):660-667. doi: 10.1197/jamia.M2146.
BakkenS, Grullon-FigueroaL, IzquierdoR, et al.Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. J Am Med Inform Assoc. 2006l 13(6):660-667. doi: 10.1197/jamia.M2146.10.1197/jamia.M2146)| false
EspositoP, Dal CantonA.Clinical audit, a valuable tool to improve quality of care: General methodology and applications in nephrology. World J Nephrol. 2014;3(4):249-255. doi: 10.5527/wjn.v3.i188.8.131.5227/wjn.v3.i4.249)| false
LightD, LexchinJ, DarrowJ.Institutional corruption of pharmaceuticals and the myth of safe and effective drugs. J Law, Med Ethics. 2013;14(3):590-610. doi: 10.1111/jlme.12068.10.1111/jlme.12068)| false