The simplest experimental comparison involves the use of two independent groups created by random assignment. This design allows the researcher to assume that all individual differences are evenly distributed between the groups, so that the groups are equivalent at the start of the experiment. Statistically, the groups are considered random samples of the same population, and therefore, any observed differences among them should be the result of sampling error or chance. After the application of a treatment variable to one group, the researcher wants to determine if the groups are still from the same population, or if their means can be considered significantly different.
Comparisons can also be made using a repeated measures design. A researcher may be interested in looking at the difference between two conditions or performances by the same group of subjects. In this case, the subjects serve as their own control, and the researcher wants to determine if the conditions are significantly different.
The purpose of this chapter is to introduce procedures for evaluating the comparison between two means using the t-test and confidence intervals. These procedures can be applied to differences between two independent samples or between scores obtained with repeated measures. These procedures are based on parametric operations and, therefore, are subject to all assumptions underlying parametric statistics.
THE CONCEPTUAL BASIS FOR COMPARING GROUP MEANS
The concept of statistical significance for comparing means is based on the relationship between two sample characteristics: the mean and the variance. The difference between group means indicates the degree of separation between groups (the effect size). Variance measures tell us how variable the scores are within each group. Both of these characteristics represent sources of variability that are used to describe the extent of treatment effects.
Suppose we wanted to compare two randomly assigned groups, one experimental and one control, to determine if treatment made a difference in their performance. Theoretically, if the experimental treatment was effective, and all other factors were equal and constant, all subjects within the treatment group would achieve the same score, and all subjects within the control group would also achieve the same score, but scores would be different between groups. As illustrated in Figure 19.1A, everyone in the treatment group performed better than everyone in the control group. Consider all the scores in this sample for both groups combined. If we were asked to explain why these scores were different, we would say that all differences were due to the effect of treatment. There is a difference between the groups, but no variance within the groups.
Four sets of hypothetical distributions with the same means, but different variances. In (A) all subjects in each group received the same score, but the groups were different from each other. There is no variance within groups. In (B) the ...