Promising Path to Success [PPS]: - Quality Improvement Toolkit

OVERVIEW

This toolkit is designed to provide assistance in employing a Quality Improvement (QI)

framework to the development and study of quality improvement in settings where services are

provided to support with emotional, behavioral, or mental health (“E/B/MH”) conditions. We

employ a general Quality Improvement framework and provide supplementary materials specific

that may be of assistance throughout this document.

WHAT IS QUALITY IMPROVEMENT (QI)?

Quality improvement (QI) is a set of activities deliberatively and collaboratively undertaken with

the goal of improving a system’s processes and/or outcomes. Processes include a series of

actions or steps that are taken in order to achieve a particular end. In pediatric behavioral health

treatment settings, this might include improving practice to be consistent with best practice and

available evidence, including but not limited to screening and treatment of E/B/MH conditions.

Outcomes are the results, end products, or consequences of those processes. In pediatric

practice, this may include patient health or behavioral health outcomes thought to improve as a

result of the QI project.

QI originates from the business community, with a rich history dating back to the early 1900’s.

The use of QI methods in business particularly helped catapult Japan into a leadership position in

the auto and electronic industries following World War II. In the second half of the 20th

century, the US military, airlines industries, and health care organizations began to adopt QI

tools to decrease errors in their processes and improve outcomes.

Following publication of the Institute of Medicine’s Crossing the Quality Chasm

, the health care

system identified using QI methods as critical for improving health care processes and patient

outcomes.

WHY A QI FRAMEWORK TO PROMOTE PROVISION OF QUALITY

HEALTHCARE FOR CHILDREN WITH E/B/MH CONDITIONS?

We chose to employ a QI framework for two reasons:

• QI’s focus is on improving processes and outcomes and can be applied to almost any

topic. QI stresses first experimenting with small changes, where financial and

motivational risks are minimized, and then spreading to a larger population once

improvements have been tested. Given both complexity of delivery systems and potential

for unintended consequences, a QI framework provides opportunity for customization to

specific delivery system challenges and ongoing evaluation of these efforts.

• QI is likely a process that you are familiar in using for respective practices and capitalizes

on recent commitments in mental health care and continuous quality improvement.

Promising Path to Success [PPS]:

Quality Improvement Toolkit

WHAT IMPROVEMENT MODELS ARE AVAILABLE?

A number of different conceptual approaches and models exist to guide QI projects, whether the

Institute for Healthcare Improvement (IHI) Model for Improvement, Lean, Six Sigma, and the

FOCUS Plan-Do-Study-Act model. These models share similar conceptual frameworks for

improvement.

n the Appendix of this workbook.

We highlight below a modified version of the IHI’s Model for Improvement that highlights some

project management tools which may be helpful when planning for a QI effort. This modified

version has been used by several health systems in the Northeast (see Figure 1).

FIGURE 1. Modified IHI Model of Improvement

Phase 1. PRE-Work. Phase 1 consists of three PRE-work steps:

• Prioritizing a quality gap, or area for improvement

• “Readying” a team to work on the quality gap

• Educating the team about the quality gap by analyzing current structure, process, and

otucomes to identify factors that may provide an opportunity to improve the quality gap

Phase 2. PLANNING. During Phase 2, the QI team develops a charter, a living document that

organizes critical pieces of information about the project and answers three questions identified

in the IHI Model for Improvement, specifically:

• Aim: What the team aims to accomplish with the project.

• Measure: How the team will measure whether or not the changes made are an

improvement.

• Change: What change strategies, or planned changes, the team plans to

implement.

Ideas about measurement tools and change strategies may come from the research literature,

programs developed by similar organizations, or brainstorming activities. These can be

documented on a measurement planning form and a change strategy brainstorming form.

The project aim, measure(s), and change strategies are often detailed in a key driver diagram,

which captures an entire QI project in a single diagram and provides a measurement framework

for monitoring progress.

Often aims, measures, and change strategies change over time as a QI team learns more.

Similarly, the charter may need to be updated. The key driver diagram helps a team to stay

“grounded” about their overarching goal.

Phase 3. ACTION. During Phase 3, the QI team completes a series of small, discrete changes

using Plan-Do-Study-Act (PDSA) cycles. These changes build on the knowledge gained in

Phases 1 and 2 and from preceding PDSA cycles.

PDSA cycles involve 4 steps:

1. Plan: Identifying small tests of change to implement

2. Do: Implementing rapid, incremental changes

3. Study: Measuring and tracking anticipated and unanticipated consequences

4. Act: Reflecting on the results and adapting, adopting, or abandoning the change

Two types of data may be collected during PDSA cycles:

• Quantitative data, or numeric data collected from surveys, rating forms, or from

administrative data, such as a health record

• Qualitative data, or what people write or say about changes in a narrative format

Quantitative data from PDSA cycles are often captured in run charts or statistical process

control charts, that display changes in a project’s processes and outcomes over time. For an

introduction to run charts, Perla, Lloyd, and Murray describe the opportunities available for

employing run charts as an analytical tool to learn from variation in healthcare processes in an

introductory article published in BMJ Quality/Safety. For informative introduction to statistical

process control charts, please see the following reference, entitled “Use of Control Charts in

Healthcare,” written by Woodall, Adams, and Benneyan. (See references for additional details.)

Qualitative data help to understand people’s experiences with changes. This type of data also

may help identify unintended consequences, both positive and negative, of change strategies.

The IHI has templates for many of the tools suggested in this toolkit (like the charter, key

driver diagram, and measurement planning form) available through the IHI Open School.

See Additional QI Resources on page 20. Templates and descriptions are also in the

Appendix.

Phase 4. REFLECTION. If changes are positive, the QI team considers:

Sustaining successful changes,

Spreading successful changes, and

Measuring impact

WHAT DOES PHASE 1 (PRE-WORK) TYPICALLY INVOLVE?

TASK 1. Prioritizing the quality gap.

Identify the quality gaps relevant to your setting. These frequently arise out of concerns

identified in the course of everyday activities within your workplace. In our prior work, state

agencies and pediatric and child and adolescent psychiatrist leadership within a state have sought

to identify where there systems fail to meet the mental health guidelines set by the American

Academy of Child and Adolescent Psychiatry. Quality gaps can often be resolved on a number of

different levels by a variety of different actors.

TASK 2. Readying the team.

This step involves identifying the team members and setting up “operating agreements” for how

the team members will work together.

QI teams often craft operating agreements to help govern their work together. Because these

agreements may change over time, operating agreements are often viewed as “dynamic” and

subject to change.

Team members frequently include:

• 1-2 team leaders who direct day-to-day operations

•

A “management sponsor” or champion whose role is to make sure the team has the

appropriate resources and the commitment from the organization to move forward

•

A QI advisor or technical expert to assist the team in identifying appropriate QI methods

•

Content experts who may serve as consultants

•

Team members who represent different stakeholder groups or constituents who have a

stake in the outcome of the project and are interested in identifying and testing changes

Operating agreements are documents that specify terms on engagement:

• Ground rules to establish how team members will interact with each other

•

Guidelines for how team members will communicate with each other

•

Roles and responsibilities of each team member

•

How key decisions will be summarized

•

A timeline, or spreadsheet, with milestones, tasks, due dates, and the names of

responsible team members of each task

TASK 3. Educating ourselves about the quality gap.

The last step in Phase 1 involves team members educating each other about the quality gap to be

addressed and considering potential change strategies.

Team members share their own experiences and seek out data to inform understanding of

challenges. Topical areas where additional data may be of assistance include:

• The scope of the quality gap

• How different stakeholders may perceive the quality gap

• What change strategies may address the quality gap?

Example of a relevant resource to inform this work is the American Academy of

Pediatrics Implementation Guide providing resources to implement a number of

clinical guidelines endorsed by the Academy, with measures, key driver diagrams,

implementation tools, and educational materials, available here .

• How different stakeholder groups may perceive any proposed change strategies

• What other similar organizations are doing and how their organization compares

(often termed “benchmarking”)

Appendices provide examples of cause-and effect or “fishbone” diagrams and flowcharts/process

maps.

WHAT DOES PHASE 2 (PLANNING) INVOLVE?

In Phase 2, major goals are typically to organize the information learned so it is accessible to

team members and can be used to guide the project aims, measurement approaches, and

proposed change strategies.

Two tools that help in organizing information are the charter and the key driver diagram.

A charter usually includes:

AIM: What the team aims to accomplish with the QI project.

MEASUREMENT: How the team will measure if the change made is an improvement

CHANGE: What change strategies the team plans to implement to make an

improvement

An aims statement should be “SMART”: Specific, Measurable, Attainable, Realistic, and

Timely.

There are a number of QI tools that teams may use to portray any information when

educating themselves about a quality gap. Some examples include:

• Cause-and-effect (fishbone) diagrams that categorize issues

• Flowcharts and process maps that capture processes

• Graphs (bar graphs, pie charts, histograms, Pareto charts) that display data

• Spaghetti diagrams that map processes on a physical floorplan

In a charter, the aims statement gives a team a clear direction for the QI project and helps to

prevent wasted time and effort. An aims statement should be “SMART”: Specific, Measurable,

Attainable, Realistic, and Timely.

Your ideas for an aims statement:

______________________________________________________________________________

[For example, an aims statement might: By X date, X% of children and adolescents in foster care

prescribed psychotropic medication within X practice will have a documented informed consent

on record that is consistent with established protocols.]

Measures help evaluate processes and systems and help a team to know if changes lead to an

improvement. Data from these measures help the team make decisions about next steps.

Change strategies are actions the team takes to make improvements in the systems or process.

WHAT DOES PHASE 3 (ACTION) INVOLVE?

Phase 3 involves a series of PDSA cycles to test change strategies. In general, early PDSA cycles

are small, discrete changes. Typically, it is wise to use the PDSA cycle forms (See Appendix) to

document any changes made.

Both narrative and quantitative data can be used for PDSA cycles, including tools such as:

Observational data Participant observations

Narrative data Semi-structured interviews, focus groups

Quantitative data Run charts, statistical process control charts

Quantitative data from PDSA cycles can be displayed in run charts and statistical process

control charts.

• Run Chart – Simple line graph that displays observed data over time to determine

whether changes made are leading to improvement.

• Statistical Process Control Chart – Statistical tool used to distinguish between variation in

a process resulting from “common causes” and variation from “special causes.” Displays

observed data over time and determines if a process is stable or unstable.

DESCRIPTION AND TEMPLATES OF RELEVANT QI TOOLS

Charter Form

Measurement Development Form

Change Strategy Brainstorming Form

PDSA Cycle Form

Flowchart

Cause-and-Effect Diagram

Key Driver Diagram

Additional Resources on QI

Most descriptions and templates for QI tools are modified from the Institute for Healthcare

Improvement’s Open School. Originals available at www.ihi.org unless otherwise noted.

CHARTER FORM

A charter is used at the beginning of a QI project to explicitly state the project’s

objective, its team

members, aims, defined measures, and changes. QI projects

often change over time. The charter is a “living document” that is updated throughout

the project.

Project Title:

Team Leader:

Executive Sponsor:

Team Members, Names and Roles (i.e. Team Leader(s), Champion, Sponsor etc.):

Name:

Position and Organization or Department:

Patients/Clients/Family Who Will Benefit:

Types of Clinical and Administrative Staff,

Suppliers,

etc. Involved:

Aim Statement (What are we trying to accomplish? Numerical target for improvement, over

what time?):

Measures (How will we know if we are improving?):

Structure Measures:

Outcome Measures:

Process Measures:

Balancing

Measures:

Change Ideas (What can we try that will result in an improvement?):

Business Case (Are health system costs reduced by addressing the problem?):

Link to Organizational Strategy:

Term of Project (Start and Stop Dates):

Project Budget:

Anticipated Milestones:

Estimated Time Required for Staff Participation:

MEASURE DEVELOPMENT FORM

A measure development form helps you to define each measure you will be using in your QI

project.

Measure

Type (i.e. outcome, process, balancing)

Rationale (why this measure is needed)

Operational definition of measure

(mathematical, what or who to count &

how to count it)

Exclusions (any observations not included

in the measure)

Useful stratifiers (which variables or

factors are useful to look at, e.g., different

times of day, time of week)

Data collection & sampling method (how

you will collect data, how frequently)

Display (how the data will be displayed for

evaluation, e.g., run chart, control chart,

etc.)

Goal of the measure (what improvement

would look like)

Source (citations/references)

Modified from the IHI tools at www.ihi.org, 2013.

CHANGE STRATEGY BRAINSTORMING FORM

A change strategy brainstorming form is used when thinking about which change strategies are

appropriate for your aim. It may help to think about why you are considering this specific change

strategy (rationale), who may be involved (stakeholders), and any barriers or facilitators that may

be involved.

Potential Change Strategy

Rationale

for

Change

Strategy

Source

(e.g.,

literature,

other

program)

Stakeholders

Who Need to

be Involved

Potential

Barriers to

Change

Strategy

Potential

Facilitators

to Change

Strategy

PLAN-DO-STUDY-ACT (PDSA) CYCLE FORM

The PDSA cycle form helps a QI team to articulate why they are implementing a change

strategy, what they hope to see happen, and what they learned.

Objective for this PDSA Cycle: Date:_________________

Is this cycle used to:

develop  test, or  implement a change?

What question(s) do we want to answer with this PDSA cycle?

____________________________________________________________________________________________

PLAN

1. What is our plan? (Consider: Who, What, When, Where, and How?)

2. What is our data collection plan? (Consider: Who, What, When, Where, and How?)

3. What do we predict will happen?

____________________________________________________________________________________________

_____________________________________________________________________________________________

1. How did we carry out the change or test?

2. What data did we collect?

3. What worked? What didn’t? Why?

____________________________________________________________________________________________

_____________________________________________________________________________________________

STUDY

1. How did we analyze the data, and what were the results?

2. How do our pediatricians compare to the date?

____________________________________________________________________________________________

_____________________________________________________________________________________________

ACT

1. Based on the learnings in “study”, above what are our next steps?

______________________________________________________________________________

FLOW CHART

Flowcharts allow you to draw a picture of the way a process actually works so that you can

understand the

existing process and develop ideas about how to improve it. A high level

flowchart, showing 6 to 12 steps, gives the panoramic view of the process. These

flowcharts show clearly the major blocks of activity or the major system components in a

process. High-level flowcharts are especially useful in the early phases of the project. A

detailed flowchart is a close-up view of the process, typically showing thousands of steps.

These flowcharts make it easy to identify rework loops and complexity in the process.

Detailed flowcharts are useful after teams have pinpointed issues or when they are making

changes in a process.

Using a flowchart has a variety of benefits:

• It helps to clarify complex processes.

• It identifies steps that do not add value to the internal or external customer

including: delays, needless storage and transportation, unnecessary work,

duplication, added expense, before

breakdowns in communication.

• It helps team members gain a shared understanding of the process and use

this knowledge to

collect data, identify problems, focus discussions, and

identify resources.

• It serves as a basis for designing new processes.

Directions for creating a flowchart:

1. Get the “right” people involved – those who know the process best.

2. Agree on the use of the flowchart and the level of detail required.

3. Decide on the format of the flowchart - high level of detail.

4. Define the first and last steps in the process (by observing, brainstorming, or

consulting with people

responsible for each step).

5. Document the process steps in sequence. Note that some steps are parallel, that is,

they happen the

same time. Describe the process as it really exists, not the ideal.

Include what happens when there

are problems in the process.

6. Use decision symbols, choose the most natural branch, and continue to the end.

7. Use “clouds” or notes for unfamiliar steps and continue to the end.

8. When you reach the last step, go back to fill the branches.

9. Read through the flowchart to check for accuracy and completeness.

10. Assign action items to fill in unfamiliar steps and verify accuracy.

11. When the flowchart is complete and accurate, analyze it, use it, and keep it up to

date.

Modifying a flowchart:

All tools are meant to be helpful. The flowchart can be annotated to include names of

people, barriers identified, and any other information helpful to the team.

Common Flow Chart Symbols

CAUSE-AND-EFFECT (FISHBONE) DIAGRAM

A cause-and-effect diagram, also known as an Ishikawa (after its developer) or “fishbone” (after

its shape) diagram, is a graphic tool used to explore and display the possible causes of a certain

effect. It will help to

identify potential changes to test for your quality improvement project.

Why is it such a valuable tool to you and your team?

• It helps teams understand that there are many causes that contribute to an effect.

• It visually displays the relationship of the causes to the effect and to each other.

• It helps identify areas of improvement within your project.

What is a cause? Causes are parts of a system and forces outside a system that directly influence

the outcome, or aim, of your project. For example, one cause of low hand hygiene compliance

rate is provider

behavior. In other words, getting providers to change their behavior (and wash

their hands) directly

influences the compliance level within the unit or hospital.

There are lots of causes that contribute to a certain effect. Take poor hand hygiene, for example.

What are

the contributing factors? Sometimes clinicians are too busy. Sometimes they wash

their hands but may not use proper hand washing techniques. Other times the gel dispenser in

the hospital is broken or the gel

dispenser may work properly, but it is empty.

Consider these six categories of causes:

1. Materials: Supply, design, availability, and maintenance

2. Methods and Process: Steps in care process and steps in supply chain

3. Environment: Staffing levels and skills, workload and shift patterns, administrative and

managerial support, and physical plant, policies, and regulations

4. Equipment: Any equipment/tools needed to get the job done

5. People: Staff knowledge and skills/training, competence, patient behavior, and supervision

6. Measurement: Data collection, definition of measures, and sampling issues

What are the steps required to construct a cause-and-effect diagram?

1. Write what you are trying to change in a box on the right-hand side of the page.

2. Draw a long horizontal line to the left of the box.

3. Decide on the categories of causes that the team wants to use. As mentioned above,

useful

categories of causes in a classic fishbone diagram include Materials, Methods

and Process, Environment, Equipment, People, and Measurement. Another way to

think of categories is in terms

of causes at each major step in the process.

(Note: These categories can vary depending on your

project. Manufacturing sometimes

follows the “5 M’s”: man, machines, materials, methods, and

measurement.

Nonmanufacturing systems sometimes use the “5 P’s”: patrons, people, provisions,

places, and procedures. Just make sure the categories you chose fit your project.)

4. Draw diagonal lines above and below the horizontal line (these are the “fishbones”), and

label with the categories you have chosen.

5. Brainstorm and collect a list of cause for each category.

6. List the cause on each fishbone. If a cause has a secondary cause, draw a branch bone to

show relationships among the causes.

7. Develop the causes by asking, “Why?” until you have reached a useful level of detail –

that is, when the cause is specific enough to be able to test a change and measure its

effect.

Fishbone Chart Template

Fishbone Example: Patient Waiting Time

KEY DRIVER DIAGRAM

A driver diagram is a powerful tool that helps you to translate a high level improvement goal into a logical

set of underpinning goals (“drivers”) and projects. It captures an entire change program in a single

diagram and also provides a measurement framework for monitoring progress.

To create a driver diagram:

• Start with a clearly defined, measurable aim. It should describe what you intend to

achieve and by when.

• Get a group of subject matter experts together. Ask them to brainstorm the areas where

improvement is needed. Consider using a flowchart or cause-and-effect diagram.

• Cluster the ideas to create an agreed set of “drivers.’’ Make sure you use language like

“improve” or “decrease” for each driver.

• Discuss the need for new drivers or whether some of the drivers should be eliminated (if

they are

wrong or immaterial).

• Identify the links between the drivers to create primary, secondary, and possibly tertiary

drivers.

• Select change strategies that you believe will impact upon your drivers.

1. The outcome or aim

The driver diagram starts with a clearly defined and measurable outcome or aim.

2. Primary drivers

The overall outcome or aim is linked to primary drivers, or factors that are believed to have a direct

impact

on the aim. This first set of underpinning factors are referred to as primary drivers because

they ‘drive’ the achievement of your main outcome or aim. These drivers may act independently or in

concert to achieve

the overall aim.

3. Lower level drivers

The process of breaking down an aim can continue to lower levels to create secondary or tertiary

drivers,

and even further if required. The driver diagram can then capture change strategies you plan

to implement

through PDSA cycles that address these different drivers (i.e., workout 3x/wk, walk the

dog, stand while on

the phone).

4. Projects, interventions, changes

The ultimate goal of a driver diagram is to define the range of change strategies that you may want to

undertake.

KEY DRIVER SAMPLE

ADDITIONAL QI RESOURCES

Books:

Langley GJ, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide. 2nd Ed.

San Francisco, CA: Jossey-Bass; 2009

Ogrinc GS, Headrick LA, Moore SM, Barton AJ, Dolansky MA, Madigosky WS. Fundamentals of

Healthcare

Improvement: A Guide to Improving Your Patients’ Care. 2nd Ed. Oakbrook Terrace, IL: Joint

Commission

Resources and IHI. 2012.

Jackson TL. Mapping Clinical Value Streams (Lean Tools for Healthcare Series). Boca Raton, FL: Taylor

and

Francis. 2013.

Papers:

Woodall WH, Adams BM, Benneyan JC, and VISN 1 Engineering Resource Center. Statistical Methods

in Healthcare, Faltin F, Kenett R, Ruggeri F, eds., Wiley, 2011

Perla R, Provost L, Murray S. The run chart: a simple analytical tool for learning from variation in

healthcare

processes. BMJ Qual Saf 2011. 20, 46-51.

Pronovost et al. Creating high reliability in health care organizations. Health Research and Educational

Trust, 2006. 1599-1617.

Scoville R, Little K. Comparing Lean and quality improvement. IHI White Paper. 2014

Gawande, A. Big med.

The New Yorker. 2012. 53-63.

Websites:

Institute for Healthcare Improvement: www.ihi.org

IHI Open School:

http://www.ihi.org/education/ihiopenschool/Pages/default.aspx

Lean

Enterprise Institute: www.lean.org

Adapted from:

Daudelin, D, Mackie TI, and Leslie, LK. Quality Improvement Toolkit for Pediatric Practices, Tufts Clinical and

Translational Sciences Institute, Tufts Medical Center, 2012.