Project Governance – Creating a Value-Added Clinical Operations Metrics Program – Part 1

I love metrics.  In particular, I love the idea of using metrics to monitor the progress and the “health” of a program, and even to proactively identify project issues before they become critical.  An ideal metrics program will also help identify where to look – which studies in a program and what aspects of those studies – to understand what may be going wrong.  Who wouldn’t love that idea?

The problem with many metrics programs is that they quickly become too complicated (let’s measure everything because, well, we can) or too burdensome (ensuring the source data are complete and available and analyses and presentations are done in near-real-time).  Metrics programs need to be selective in what they highlight to managers if they are to help us get our work done.

This short series of articles will provide guidance and explore some of the challenges we face in Clinical Operations as we endeavor to implement metrics to help manage a portfolio of clinical programs.

Part 1 – First, Know What’s Critical for Your Success

Thanks to great software and your local Microsoft Excel expert, there’s no shortage of measurements we can produce.  With all the things we can measure, we need to acknowledge the difference between what you should monitor, what you should manage and what you should focus on as a goal or be rewarded on.  It’s really important that you select and focus on the measurements that will help you do your work.

A modern automobile has a lot of computers and microprocessors on board.  But the dashboard still displays the speedometer and the fuel gauge most prominently.

The first thing you need to establish is what’s important to you.  The Metrics Champion Consortium (which is a collaboration of various stakeholders from across the entire Clinical Trials industry) talks about identifying Critical Success Factors and Key Performance Questions. Your Critical Success Factors are the things that really define success or failure.  For example, if your program needs to recruit its first patient in order to trigger a tranche of venture capital funding, or to complete recruitment to enable filing ahead of a competitor, then start-up and recruitment metrics need to be a focus.  If you’ve recently completed recruitment and are anticipating filings, then your focus may need to change to quality and compliance metrics in anticipation of upcoming inspections.  When you set up your metrics program, you want to include metrics for both speed and quality.

Key Performance Questions help figure out what you want to know about that Success Factor and will help you identify what you will actually measure and how it will be presented.  For example, if your CSF is about start-up and recruitment, the question would probably be “Are we achieving study site set-up as quickly as planned?” and “Are patients being enrolled as quickly as planned?”

The final framing question is to determine who the metrics are for.  The metrics used by study teams will be different than the ones needed by senior managers.  That’s because the Key Performance Questions that each of those groups ask may also be different.  But of course they should be consistent and feed into one another.

In Part 2 of this series, I will discuss things you’ll need to keep in mind when implementing your metrics program. Then in Part 3, I’ll talk about ensuring you have other ways to understand the full context for the measures you’re seeing.