7.3 Inference Of The Difference Of Two Means

Let's delve into the fascinating realm of statistical inference, specifically focusing on drawing conclusions about the difference between two population means. This is a crucial skill in various fields, from scientific research to business analytics, allowing us to determine if observed differences are statistically significant or simply due to random chance.

Why Compare Two Means?

Imagine you're a marketing manager testing two different advertising campaigns. You want to know which campaign leads to a higher average conversion rate. Or perhaps you're a medical researcher comparing the effectiveness of a new drug against an existing treatment. In both scenarios, the core question is: Is there a significant difference in the average outcome between two groups? This is where inference of the difference of two means comes into play.

The Core Concepts

Before diving into the specifics, let's solidify some foundational concepts:

Population Mean (μ): The average value of a variable for the entire population of interest. This is often unknown and needs to be estimated.
Sample Mean (x̄): The average value of a variable calculated from a sample drawn from the population. This is our best estimate of the population mean.
Independent Samples: Samples drawn from two populations where the selection of one sample does not influence the selection of the other. This is a critical assumption for many of the methods we'll discuss.
Null Hypothesis (H0): A statement that there is no difference between the population means (μ1 - μ2 = 0). This is the hypothesis we're trying to disprove.
Alternative Hypothesis (H1): A statement that there is a difference between the population means. This can take several forms:
- μ1 - μ2 ≠ 0 (two-tailed test: the means are different)
- μ1 - μ2 > 0 (one-tailed test: mean 1 is greater than mean 2)
- μ1 - μ2 < 0 (one-tailed test: mean 1 is less than mean 2)
Significance Level (α): The probability of rejecting the null hypothesis when it is actually true (Type I error). Common values are 0.05 (5%) and 0.01 (1%).
P-value: The probability of observing a sample difference as extreme as, or more extreme than, the one observed, assuming the null hypothesis is true. A small p-value suggests strong evidence against the null hypothesis.

The Different Scenarios

The specific method used for inference of the difference of two means depends on a few key factors:

Whether the samples are independent or dependent (paired).
Whether the population standard deviations are known or unknown.
Whether the population distributions are approximately normal.

Let's explore each scenario in detail:

1. Independent Samples, Population Standard Deviations Known

This is the simplest case. If we know the population standard deviations (σ1 and σ2) for both groups, we can use a z-test. This scenario is relatively rare in practice because population standard deviations are usually unknown.

Test Statistic:
```
z = (x̄1 - x̄2) - (μ1 - μ2) / sqrt( (σ1^2 / n1) + (σ2^2 / n2) )
```
Where:
- x̄1 and x̄2 are the sample means
- μ1 and μ2 are the population means (usually μ1 - μ2 = 0 under the null hypothesis)
- σ1 and σ2 are the population standard deviations
- n1 and n2 are the sample sizes
Degrees of Freedom: Not applicable in this case. We use the standard normal distribution (z-distribution).
Decision Rule: Compare the calculated z-statistic to the critical value from the z-distribution based on the chosen significance level (α) and the type of test (one-tailed or two-tailed). If the absolute value of the z-statistic exceeds the critical value, we reject the null hypothesis. Alternatively, compare the p-value to the significance level. If the p-value is less than α, we reject the null hypothesis.

2. Independent Samples, Population Standard Deviations Unknown, Equal Variances Assumed

In this more common scenario, we don't know the population standard deviations, but we can assume that they are equal (σ1 = σ2). This allows us to pool the sample variances to get a better estimate of the common population variance.

Test Statistic: We use a t-test.
```
t = (x̄1 - x̄2) - (μ1 - μ2) / sqrt( Sp^2 * (1/n1 + 1/n2) )
```
Where:
- x̄1 and x̄2 are the sample means
- μ1 and μ2 are the population means (usually μ1 - μ2 = 0 under the null hypothesis)
- Sp^2 is the pooled variance
Pooled Variance (Sp^2):
```
Sp^2 = ((n1 - 1) * s1^2 + (n2 - 1) * s2^2) / (n1 + n2 - 2)
```
Where:
- s1^2 and s2^2 are the sample variances
Degrees of Freedom: n1 + n2 - 2
Decision Rule: Compare the calculated t-statistic to the critical value from the t-distribution based on the degrees of freedom, significance level (α), and the type of test (one-tailed or two-tailed). If the absolute value of the t-statistic exceeds the critical value, we reject the null hypothesis. Alternatively, compare the p-value to the significance level. If the p-value is less than α, we reject the null hypothesis.
Checking the Assumption of Equal Variances: We can use statistical tests like Levene's test or the F-test to formally test the assumption of equal variances. However, these tests can be sensitive to departures from normality. A rule of thumb is to check if the ratio of the larger sample variance to the smaller sample variance is less than 4. If the assumption of equal variances is violated, we should use the next scenario.

3. Independent Samples, Population Standard Deviations Unknown, Unequal Variances Assumed

This is the most general case for independent samples. We don't know the population standard deviations, and we cannot assume they are equal. This requires using a modified t-test, often called Welch's t-test.

Test Statistic:
```
t = (x̄1 - x̄2) - (μ1 - μ2) / sqrt( (s1^2 / n1) + (s2^2 / n2) )
```
Where:
- x̄1 and x̄2 are the sample means
- μ1 and μ2 are the population means (usually μ1 - μ2 = 0 under the null hypothesis)
- s1^2 and s2^2 are the sample variances
- n1 and n2 are the sample sizes
Degrees of Freedom: The degrees of freedom are calculated using a more complex formula (Welch-Satterthwaite equation):
```
df = ( (s1^2 / n1) + (s2^2 / n2) )^2 / ( ( (s1^2 / n1)^2 / (n1 - 1) ) + ( (s2^2 / n2)^2 / (n2 - 1) ) )
```
The result is usually rounded down to the nearest whole number.
Decision Rule: Compare the calculated t-statistic to the critical value from the t-distribution based on the calculated degrees of freedom, significance level (α), and the type of test (one-tailed or two-tailed). If the absolute value of the t-statistic exceeds the critical value, we reject the null hypothesis. Alternatively, compare the p-value to the significance level. If the p-value is less than α, we reject the null hypothesis.

4. Dependent Samples (Paired Samples)

In this scenario, the samples are related or paired in some way. For example, we might measure the blood pressure of the same individuals before and after taking a medication. The key is that each observation in one sample has a corresponding observation in the other sample. We analyze the differences between the paired observations.

Calculate the Differences: For each pair, calculate the difference (d = x1 - x2).
Calculate the Mean Difference (d̄): Calculate the average of the differences.
Calculate the Standard Deviation of the Differences (sd): Calculate the standard deviation of the differences.
Test Statistic: We use a t-test.
```
t = (d̄ - μd) / (sd / sqrt(n))
```
Where:
- d̄ is the mean difference
- μd is the population mean difference (usually μd = 0 under the null hypothesis)
- sd is the standard deviation of the differences
- n is the number of pairs
Degrees of Freedom: n - 1
Decision Rule: Compare the calculated t-statistic to the critical value from the t-distribution based on the degrees of freedom, significance level (α), and the type of test (one-tailed or two-tailed). If the absolute value of the t-statistic exceeds the critical value, we reject the null hypothesis. Alternatively, compare the p-value to the significance level. If the p-value is less than α, we reject the null hypothesis.

Confidence Intervals for the Difference of Two Means

In addition to hypothesis testing, we can also construct confidence intervals to estimate the plausible range of values for the difference between two population means.

General Form:
```
(x̄1 - x̄2) ± (critical value) * (standard error)
```
The critical value is obtained from the z-distribution or t-distribution, depending on the scenario. The standard error depends on the scenario as well (population standard deviations known/unknown, equal/unequal variances, paired samples).
Example (Independent Samples, Unequal Variances):
```
(x̄1 - x̄2) ± tα/2, df * sqrt( (s1^2 / n1) + (s2^2 / n2) )
```
Where tα/2, df is the critical value from the t-distribution with the appropriate degrees of freedom.

Assumptions and Considerations

Normality: The t-tests and z-tests rely on the assumption that the population distributions are approximately normal, or that the sample sizes are large enough (typically n > 30) for the Central Limit Theorem to apply. If the data are severely non-normal and the sample sizes are small, non-parametric tests (like the Mann-Whitney U test for independent samples or the Wilcoxon signed-rank test for paired samples) may be more appropriate.
Independence: The independent samples t-tests rely on the assumption that the samples are independent. If the samples are not independent, the paired t-test should be used.
Equal Variances (for the Pooled t-test): If using the pooled t-test, it's important to check the assumption of equal variances. If the variances are not equal, Welch's t-test should be used.
Outliers: Outliers can significantly affect the sample means and standard deviations, and therefore the results of the hypothesis tests. It's important to identify and address any outliers before performing the analysis.
Sample Size: Larger sample sizes provide more statistical power, making it more likely to detect a true difference between the population means if one exists.

Example: Comparing Exam Scores

Let's say we want to compare the average exam scores of two different teaching methods. We have two independent groups of students:

Group 1 (Method A): n1 = 35, x̄1 = 78, s1 = 8
Group 2 (Method B): n2 = 40, x̄2 = 82, s2 = 6

We want to test if there's a significant difference in the average exam scores between the two methods at a significance level of α = 0.05.

Hypotheses:
- H0: μ1 - μ2 = 0 (There is no difference in average exam scores)
- H1: μ1 - μ2 ≠ 0 (There is a difference in average exam scores)
Assumptions: We'll assume the exam scores are approximately normally distributed and that the samples are independent. We'll also need to check the assumption of equal variances. The ratio of the larger variance to the smaller variance is 8^2 / 6^2 = 1.78, which is less than 4. So, we'll proceed with the pooled t-test.

Pooled Variance:

Sp^2 = ((35 - 1) * 8^2 + (40 - 1) * 6^2) / (35 + 40 - 2) = 50.21

Test Statistic:

t = (78 - 82) - 0 / sqrt( 50.21 * (1/35 + 1/40) ) = -2.48

Degrees of Freedom:
```
df = 35 + 40 - 2 = 73
```
Critical Value: For a two-tailed test with α = 0.05 and df = 73, the critical value from the t-distribution is approximately ±1.993.
Decision: Since the absolute value of the calculated t-statistic (2.48) is greater than the critical value (1.993), we reject the null hypothesis.
Conclusion: There is a statistically significant difference in the average exam scores between the two teaching methods. Method B appears to lead to higher scores.

Using Software for Calculations

While understanding the formulas is crucial, in practice, you'll typically use statistical software packages like R, Python (with libraries like SciPy), SPSS, or Excel to perform these calculations. These tools automate the calculations and provide p-values, making the process much more efficient. The output from these packages will usually include the t-statistic, degrees of freedom, p-value, and confidence interval.

Beyond the Basics

Effect Size: While hypothesis testing tells us if there's a statistically significant difference, it doesn't tell us the size of the difference. Effect size measures, such as Cohen's d, can be used to quantify the practical significance of the difference.
Power Analysis: Power analysis helps determine the sample size needed to detect a statistically significant difference, given a certain effect size and significance level.
Non-Parametric Tests: As mentioned earlier, if the assumptions of normality are not met, non-parametric tests like the Mann-Whitney U test or the Wilcoxon signed-rank test can be used.

Conclusion

Inference of the difference of two means is a powerful statistical tool for comparing groups and drawing meaningful conclusions. By understanding the different scenarios, assumptions, and methods, you can confidently analyze data and make informed decisions in various fields. Remember to always consider the context of your data, check the assumptions of the tests, and interpret the results carefully. Using statistical software will streamline the calculations and help you focus on the interpretation of the results.