Skip to Main Content

Chapter 24: Foundations of Statistical Inference

Introduction

The simplest experimental comparison involves the use of two independent groups within a randomized controlled trial (RCT). This design allows the researcher to assume that all individual differences are evenly distributed between the groups at baseline, and therefore, observed differences after the application of an intervention to one group should reflect a significant difference, one that is unlikely to be the result of chance.

The statistical ratio used to compare two means is called the t-test, also known as Student’s t-test. The purpose of this chapter is to introduce procedures that can be applied to differences between two independent samples or between scores obtained with repeated measures. These procedures are based on parametric operations and, therefore, are subject to assumptions underlying parametric statistics.

The Conceptual Basis for Comparing Means

The concept of testing for statistical significance was introduced in Chapter 23 in relation to a one-sample test. The process for comparing two sample means is similar, with some important variations. To illustrate this process, suppose we divide a sample of 100 subjects into two randomly assigned groups (n = 50), one experimental and one control, to determine if an intervention makes a difference in their performance. Theoretically, if the treatment was ineffective (H0), we would expect all subjects in both groups to have the same response, with no difference between groups.

But if the treatment is effective, all other factors being equal, we would expect a difference, with all subjects within the experimental group getting the same score, and all subjects within the control group getting the same score, but scores would be different between groups. This is illustrated in Figure 24-1A, showing that everyone in the treatment group performed better than everyone in the control group.

Figure 24–1

Four sets of hypothetical distributions with the same means but different variances. A) All subjects in each group have the same score, but the groups are different from each other. There is no variance within groups, only between groups. B) Subjects’ scores are more spread out, but the control and experimental conditions are still clearly different. The variances of both groups are equal. C) Subjects are more variable in responses, with greater variance within groups, making the groups appear less different. D) The variances of the two groups are not equal.

A set of four graphs, A, B, C, and D, depict the four pairs of hypothetical distributions.
View Full Size||Download Slide (.ppt)

Now think about combining all 100 scores. If we were asked to explain why some scores in this sample were different, we would say that all differences were due to the effect of treatment. There is a difference between the groups but no variance within the groups.

Error Variance

Of course, that scenario is not plausible, so let’s consider the more realistic situation where subjects within ...

Pop-up div Successfully Displayed

This div only appears when the trigger link is hovered over. Otherwise it is hidden from view.

This site uses cookies to provide, maintain and improve your experience. MH Privacy Center Close