Creating Measures

“How will we measure change?”

In Designing and Testing Changes, you came up with a change you want to try. Before you test out the change, ask yourself how you’ll know whether the change made a difference. How will you measure it?

Types of Measures

  • Structure: Measures that look at resources, staff, and systems.

  • Process: Measures that look at workflow steps.

  • Outcome: Measures that look at whether patients are actually getting better.

  • Balancing: Measures that look to see if you are improving one area without negatively affecting another.

  • Data Quality: Measures that look at whether the data you’re relying on for your other measures is complete and accurate.

When you pick a change to try, you will usually come up with multiple measures related to the change. Depending on the type of change, you’ll probably make at least one structure or process measure; you may make outcome measures; and you will probably make one or more balancing measure and data quality measure.

When you think about measures, you think about data. And when you think about data, you probably think about your patient care reporting system and NEMSIS. But that’s not your only data source. Think more broadly about other data sources:

  • Dispatch
  • Billing
  • Personnel management and scheduling
  • Medical devices
  • Hospitals
  • Quality assurance reviews
  • … and even the various spreadsheets you might be keeping

Measure Details

Now, build out the details of each of your measures:

Name Short name of the measure.
Description A one-sentence description of what it measures.
Inclusion Criteria Which records/reports from your data this measure will look at.
Denominator What you are counting from the records/reports. Often, the denominator is simply a count of records.
Numerator The criteria for the records that “pass” the measure.
Baseline Your current performance on the measure.
Goal Where you want your performance to be, and when.
Worksheet

Measure Development Worksheet

NEMSQA Measures

The National EMS Quality Alliance (NEMSQA) is a great place to look for performance measures that have already been developed, rigorously tested, and implemented all over the United States. They’ve already done the work of figuring out all the measure details. Most of the NEMSQA measures are process measures, and a few are outcome measures.

Example: Measure DevelopmentChelsea is working on improving trauma care in her agency. One area for improvement is pain management. Luckily, NEMSQA has a few measures in this area, so Chelsea can save a bunch of work and use them:

  • Trauma-01 Pain Assessment of Injured Patients (a Process measure)
  • Trauma-03 Effectiveness of Pain Management for Injured Patients (an Outcome measure)
  • Trauma-08 Documentation of GCS, SBP, and Respiratory Rate (a Process measure)

That’s a good start, but she comes up with a couple more measures:

  • Pain Management Medication Complications (a Balancing measure)
  • Documentation of Traumatic Injuries (a Data System measure)

Chelsea uses the Measure Development Worksheet to work out the details for her additional measures. Here’s how she develops the “Pain Management Medication Complications” measure:

Name Pain Management Medication Complications
Description Percentage of pain medication administrations for trauma that had no complications
Inclusion Criteria On PCRs where Possible Injury is "Yes," Medication administrations where medication was acetaminophen, ibuprofen, fentanyl, hydromorphone, ketamine, ketorolac, morphine, or nitrous oxide
Denominator Count of medication administrations
Numerator Count of medication administrations where Medication Complication is "None"
Baseline 78%
Goal Maintain 75–80% for the next year (this is a balancing measure)

Tracking the Measures

Now that you’ve chosen some measures and worked out the details, it’s time to set up the process to track them:

  • Who will track the measures and how often?

  • How will the measures be visualized/displayed and communicated?

Example: Measure TrackingChelsea has identified these measures to track trauma pain management in her agency:

  • NEMSQA Trauma-01 Pain Assessment of Injured Patients (a Process measure)
  • NEMSQA Trauma-03 Effectiveness of Pain Management for Injured Patients (an Outcome measure)
  • NEMSQA Trauma-08 Documentation of GCS, SBP, and Respiratory Rate (a Process measure)
  • Pain Management Medication Complications (a Balancing measure)
  • Documentation of Traumatic Injuries (a Data System measure)

She hands the measures off to Maria. Their ePCR system already has reports built for the NEMSQA measures. Maria uses those reports as examples and builds reports for the two additional measures that Chelsea developed. Maria will report on the measures on the first Monday of each month. She will display them using line charts that show performance by month. She will email the charts to agency leadership and post print-outs on a bulletin board in the station.