Appendix C: Power and Sample Size

INTRODUCTION

In Chapter 18 we introduced the concept of power as an important consideration in testing the null hypothesis. The purpose of this appendix is to describe statistical procedures for power analysis and estimation of sample size for studies using the t-test, analysis of variance, correlation, multiple regression, and chi-square for contingency tables. These procedures are based on the work of Cohen.¹ For each procedure, formulas are provided, followed by specific examples of their use.

THE EFFECT SIZE INDEX

In power analysis we are concerned with five statistical elements: the significance criterion (α), the sample size (n), sample variance (s²), effect size (ES), and power (1 – β). These elements are related in such a way that given any four, the fifth is readily determined.

Effect size is a measure of the magnitude of difference or correlation. The larger the observed effect, the more likely it will result in a significant statistical test (given a specific alpha level). An effect size index is a statistic that represents effect size using a standardized value that is universally applicable for all units of data, just as t, F and r are unit free. A different form of effect size index is used for each statistical procedure.

It is a simple process to calculate a sample effect size index following completion of a study. We know the sample size, and we can calculate the actual variance, means, correlations, or proportions in the data. This information can then be used to determine the degree of power achieved.

During planning stages of a study we use effect size to determine how many subjects will be needed. But because data are not yet available, the researcher must make an educated guess as to the expected effect size. This hypothesis is often based on previous research or pilot data, where studies can provide reasonable estimates for mean differences, correlations and variances. When such data are not available, the effect size estimate may be based on the researcher's opinion of a clinically meaningful difference; that is, the researcher can determine how large an effect would be important. For example, suppose we were interested in studying two treatments for improving shoulder range of motion in patients with adhesive capsulitis. We might say that the results of the treatments should differ by at least 20 degrees, or we would not consider the difference to be meaningful. Therefore, if we observed a difference this large, we would want it to be significant. This would be the effect size we would propose. Similarly, for a correlational study we could propose that a correlation of at least .60 would be important. These types of clinical judgments can then be used to guide the estimation of sample size.

Conventional Effect Sizes

...