Chapter 5: What Does This Outcome Measurement Really Mean?

Introduction

What would it take for you to add a test or measure to a patient's assessment or to replace a measure you are currently using? When faced with this question, one of our colleagues answered this way: “To take the extra time and effort required to administer and score a new measure, it must provide additional clinically useful information and lead to more confident decisions than obtained from my current assessment methods.”

There are three important points raised in this statement. One is that the measure must provide a greater amount of useful information than is currently available. The second point introduces the notion of confidence. In mentioning confidence, this physical therapist is acknowledging that the information obtained from measures contains error and that some measures will be better than others for a given purpose. The final concept considers the reality of time. In a busy practice, one must efficiently maximize information and confidence in clinical decisions.

What's Ahead

The previous chapter dealt with random error and its application in determining the confidence in a measured value. It did not consider the degree to which the measure was assessing the characteristic of interest. In this chapter, we build on the previous one by considering the extent to which valid inferences can be drawn from a measure's values. Historically, validity focused on the integrity of a measure and it was not uncommon to see validity referred to as a property of a measure. For example, “the 6-minute walk test has a high level of validity.” Subsequently, it was realized that validity is not a property of a measure, but rather of a measure's scores or values.¹ This distinction is important because it directs one's attention to the context in which measurements are obtained. The context includes patient characteristics, clinical setting, and conditions of measurement. When considering validity today, our attention is no longer restricted to validity coefficients and measured values, but rather the emphasis is now on the inferences we can confidently draw from the measured values.² In other words, “What does this outcome measurement really mean?” In this chapter, we answer this question by responding to two inextricably linked questions:

1. To what extent does a measure assess what it is intended to measure within a declared context?

2. What is the interpretation of a measured value?

The first question considers traditional validation procedures and the latter question addresses interpretability. Interpretability comments on the extent to which qualitative meaning can be assigned to quantitative values.³

Part 1: To What Extent Does the Measure Assess What It Is Intended to Measure?

The Concept Behind This Question

Applied in a reliability context, the term true value represents the average of a conceptually infinite number of measurements when only random error ...