The simplest experimental comparison involves the use of two independent groups within a randomized controlled trial (RCT). This design allows the researcher to assume that all individual differences are evenly distributed between the groups at baseline, and therefore, observed differences after the application of an intervention to one group should reflect a significant difference, one that is unlikely to be the result of chance.
The statistical ratio used to compare two means is called the t-test, also known as Student’s t-test. The purpose of this chapter is to introduce procedures that can be applied to differences between two independent samples or between scores obtained with repeated measures. These procedures are based on parametric operations and, therefore, are subject to assumptions underlying parametric statistics.
The Conceptual Basis for Comparing Means
The concept of testing for statistical significance was introduced in Chapter 23 in relation to a one-sample test. The process for comparing two sample means is similar, with some important variations. To illustrate this process, suppose we divide a sample of 100 subjects into two randomly assigned groups (n = 50), one experimental and one control, to determine if an intervention makes a difference in their performance. Theoretically, if the treatment was ineffective (H0), we would expect all subjects in both groups to have the same response, with no difference between groups.
But if the treatment is effective, all other factors being equal, we would expect a difference, with all subjects within the experimental group getting the same score, and all subjects within the control group getting the same score, but scores would be different between groups. This is illustrated in Figure 24-1A, showing that everyone in the treatment group performed better than everyone in the control group.
Four sets of hypothetical distributions with the same means but different variances. A) All subjects in each group have the same score, but the groups are different from each other. There is no variance within groups, only between groups. B) Subjects’ scores are more spread out, but the control and experimental conditions are still clearly different. The variances of both groups are equal. C) Subjects are more variable in responses, with greater variance within groups, making the groups appear less different. D) The variances of the two groups are not equal.
Now think about combining all 100 scores. If we were asked to explain why some scores in this sample were different, we would say that all differences were due to the effect of treatment. There is a difference between the groups but no variance within the groups.
Of course, that scenario is not plausible, so let’s consider the more realistic situation where ...