Metrics serve a valuable purpose across business and healthcare settings. Identifying the correct metrics can lead to improved patient care, profitability, and success. Selecting the wrong metric can lead to patient dissatisfaction, employee discouragement, financial losses, and potentially even failure. By establishing key performance indicators (KPIs), one is able to focus work on the most important and meaningful metrics. If set properly, KPIs will provide valuable focus to leaders and front-line employees. A thoughtful approach to metric selection is an essential aspect of developing successful telehealth services.

Metrics can be broken down and organized in multiple ways. Each individual organization likely has their preferred method, which is often aligned to the mission, vision, and values of the organization. Understanding your organization's priorities will prove valuable in determining the best way to organize metrics for a telehealth service. Further, that understanding will help you prioritize which metrics to focus efforts around. Of equal importance, organizational priorities will help you determine which metrics are less important.

Many of the metrics used to assess telehealth services do not vary significantly from traditional metrics that you might choose to evaluate a service delivered by more traditional means. This can be helpful while establishing telehealth services because previously established metrics can be immediately employed in tracking and assessing services delivered remotely. Adjustments and metric additions will further enhance effective measurement of telehealth services. The discussion surrounding different types of metrics will include those you might find with ambulatory care services delivered face-to-face that can be applied to care delivered virtually, along with metrics that might be considered more exclusive to telehealth services.

In this chapter we divide metrics into five general categories: access to care, financial impact/cost, experience, effectiveness, and utilization. This breakdown aligns with the domains that the National Quality Forum (NQF) outlined in its telehealth report published in 2017 (which was introduced in Table 10.3). The classification of metrics was based on a comprehensive environmental scan of available publications surrounding telehealth measures.

Tool for telehealth pharmacy practice evaluation: Utilizing a framework to select, organize, and communicate key performance indicators will position one well for demonstrating value and justifying service expansion. Aligning metrics with organizational priorities and goals is important. Remember that continual improvement and refinement of metric tracking will be needed.

FIVE CATEGORIES OF METRICS

Access to Care

The access to care domain focuses on whether remote individuals can obtain needed information, expertise, or clinical services effectively. A significant driver behind implementing telehealth services is to improve patient access to care. It also includes connecting clinical experts to healthcare professionals in areas where those experts may be lacking.

As stated previously, meaningful access metrics will vary depending on your ambulatory practice, the services or information being delivered, and the priorities of the organization or practice. A few examples are provided in Table 11.1.

Table 11.1.

Examples and Definitions of Access Metrics

Access Metric Examples

Definition

Median lag time

Number of days to the next available appointment

Third next available

Average length of time in days between the request for an appointment and the third available appointment

Future capacity

Percent of time available to book patients for the next 4 weeks

Panel size

Number of unique patients attributed to an individual provider

Financial Impact

The financial impact domain includes measures that assess revenues, expenses, and cost savings attributed to delivery of a service. The ultimate decision as to whether a telehealth pharmacy practice service is implemented comes down to its financial viability. Clearly defined, accurate financial metrics can create a strong case for implementation or continued support of a service.

The financial impact of the delivery of care is of great concern to various individuals and groups. Patients, families, and/or caregivers value reduced out-of-pocket expense. Demonstrating reduced costs to patients through telehealth services is a valuable selling point. Telehealth services that deliver reduced out-of-pocket savings, while still delivering quality care, are attractive for patients. Members of the care team also celebrate opportunities to reduce a patient's costs.

Health systems value reduced overhead and operational costs. Depending on a health system's priorities, different metrics might be more meaningful. For example, per member per month (PMPM) savings would be meaningful to an organization operating in a value-based environment, where profit per encounter or relative value unit (RVU) might be more appropriate in a health system with more emphasis on fee-for-service.

Payers are drawn to measures that demonstrate reduced overall cost. PMPM savings (medical and pharmaceutical) are one example. Generic prescribing and formulary compliance percentages can also demonstrate value to payers. Examples and definitions of other metrics related to financials and cost are presented in Table 11.2.

Table 11.2.

Examples and Definitions of Metrics Related to Financials and Cost of Care

Financial Impact/Cost Metric Examples

Definition

Out-of-pocket cost savings

Patient costs for healthcare services or products received. Does not include insurance premiums.

Per member per month (PMPM)

Metric used by payers to describe the average cost of healthcare for each member

Profit or savings per encounter

Financial savings/profit per unit allows for scaling of services

% formulary compliance

Ratio of alignment with a payer’s formulary. Increased compliance leads to pharmaceutical expense savings

% generic prescribing

Ratio of generic medications prescribed compared to overall prescribing

Return on investment (ROI)

The financial return on an investment made or expenses realized

Relative value unit (RVU)

Measurement of provider productivity

Experience

The experience domain centers on the usability and participation in a telehealth pharmacy practice encounter. The focus on the consumer or patient experience is growing within healthcare. With more healthcare information from both credible and suspect sources easily accessible to patients, providing a valuable patient experience will help draw patients to quality care delivered by clinical professionals.

Customer service is valued across service industries. It is tied to consumer satisfaction and loyalty. The patient's experience also plays a role in the perceived quality of care. Metrics evaluating patient experience might include net promotor score or likelihood to recommend. Tracking and sharing stories of excellent customer service are also important.

Employee and provider experience are also valuable to measure. Happy and satisfied employees tend to lead to better patient care and better patient experience. For pharmacists delivering care to patients in ambulatory care settings, provider satisfaction can be a significant driver for both the care team and the broader health system. Examples of experience metrics can be found in Table 11.3.

Table 11.3.

Examples and Definitions of Experience Metrics

Experience Metric Examples

Definition

Net promotor score

Metric that shows a customer’s overall satisfaction of a service

Likelihood to recommend

An example of a customer service question that describes a customer’s likelihood to recommend your service

Provider satisfaction

Similar to customer satisfaction, measuring providers’ satisfaction for services provided by ambulatory care pharmacists via telehealth adds value to providers and their patients

Well-being index

Measure of healthcare professional burnout. With levels of burnout increasing, it is valuable to measure and share services that can improve the well-being of pharmacists and physicians.

Customer service stories

Stories that describe the care delivered connect to an individual’s emotions and can describe the individual impact that makes numbers and percentages more personal.

Effectiveness

The effectiveness domain encompasses a large variety of potentially important metrics for an ambulatory care pharmacy service. Quality measures and metrics are a core part of the effectiveness domain. System, clinical, operational, and technical effectiveness measures are meaningful. Examples and definitions of effectiveness metrics can be found in Table 11.4.

Table 11.4.

Examples and Definitions of Effectiveness Metrics

Effectiveness Metric Examples

Definition

Population health targets

Numerous priority population health targets could be identified and used.

Disease markers

Blood pressure, HbA1c, INR, cholesterol, etc.

Time to goal

More rapid achievement of therapy goals demonstrates the effectiveness of pharmacist and telehealth services.

Readmission rates

Reductions in costly readmission rates demonstrate impactful cost savings and improved quality of care.

Event occurrence rates

Reductions in complications or events demonstrate the effectiveness of services.

Medication therapy problems identified/resolved

Identification and resolution of medication therapy problems highlight the impact pharmacists have in improving patient safety and care quality.

Telehealth cases

Both telehealth encounters and ratios of telehealth encounters to overall patient encounters can be meaningful to detail the work being done.

System metrics might include effective sharing of information between different healthcare professionals and population health targets (i.e., medication adherence, health screening completion). Clinical metrics include both direct and indirect outcome and process measures of quality. Disease markers (HbA1c, blood pressure, etc.), readmission rates, event occurrence, and medication therapy problems are only some metrics that may be valuable to track.

Operational effectiveness might include metrics that demonstrate integration into the broader health system. Being able to speak to the volume and types of services delivered by various telehealth means (i.e., synchronous, asynchronous, remote patient monitoring, mobile health applications) can demonstrate operational effectiveness.

As with other telehealth domains, selection of metrics will depend greatly on individual practice and health-system priorities. Careful selection of the highest priority metrics can increase visibility and demonstrate value to multiple stakeholders.

Utilization

While the utilization of a service is not defined in the NQF telehealth report, it is an important consideration that is often overlooked. For those services that require a provider referral to the pharmacist, the number of referrals compared to the number of eligible patients should be measured. For any type of pharmacist service, evaluating how frequently patients are engaging in the service is also important. Evaluating the number of unique patient referrals compared to the number of unique patient initial visits can provide a snapshot of how well your patient engagement strategies are working (Table 11.5). Comparing the type of visit in which patients engage can also provide some insight into the uptake of in-person visits versus telehealth visits, helping to tailor the service to fit the needs of the population.

Table 11.5.

Examples and Definitions of Utilization Metrics

Utilization Metric

Definition

Percentage of population engaged in initial visit

Number of unique patients engaged in an initial visit/total population eligible for service

Percentage of population engaged in follow-up visits

Number of unique patients engaged in a second visit (follow-up visit)/number of unique patients engaged in an initial visit

Countless metrics exist across the domains of access to care, financial impacts, experience, effectiveness, and utilization. A common misstep is to track “everything.” While this approach seems reasonable, especially if data are readily available, it can dilute and even hide the most meaningful measures. Selecting KPIs will allow you and stakeholders to focus on the most important and most impactful metrics. Demonstrating value and continuous improvement across a few metrics is often preferable to measuring everything. Selecting metrics that align with organizational priorities and reimbursement strategies may help you to be the most successful.

This chapter provides a framework for understanding and organizing different types of metrics available for helping to communicate a powerful story about the work being delivered via telehealth. Understanding different types of metrics and prioritizing the most impactful will position you well to communicate meaningfully to stakeholders.

REFERENCE

1

National Quality Forum. Creating a framework to support measure development for telehealth. August 31, 2017. https://www.qualityforum.org/Publications/2017/08/Creating_a_Framework_to_Support_Measure_Development_for_Telehealth.aspx. Accessed May 13, 2021.