Covariance measures how two variables change together across paired data. It tells whether values of X above their mean tend to occur with values of Y above or below its mean. A positive covariance suggests the variables move in the same direction, while a negative covariance suggests they move in opposite directions.
This matters because covariance is the foundation for understanding correlation, regression, and patterns in scatter plots.
Key Facts
- Sample covariance: s_xy = Σ[(x_i - x_bar)(y_i - y_bar)] / (n - 1)
- Population covariance: cov(X,Y) = Σ[(x_i - μ_x)(y_i - μ_y)] / N
- Positive covariance means above-average X values tend to pair with above-average Y values, and below-average X values tend to pair with below-average Y values.
- Negative covariance means above-average X values tend to pair with below-average Y values, or the reverse.
- Correlation rescales covariance: r = s_xy / (s_x s_y)
- Covariance has units equal to the units of X times the units of Y, so its size is hard to compare across different data sets.
Vocabulary
- Covariance
- Covariance is a measure of how two variables vary together relative to their means.
- Deviation
- A deviation is the difference between a data value and its mean, such as x_i - x_bar.
- Scatter plot
- A scatter plot is a graph of paired data points that shows the relationship between two quantitative variables.
- Correlation
- Correlation is a standardized version of covariance that ranges from -1 to 1.
- Mean
- The mean is the arithmetic average of a set of values.
Common Mistakes to Avoid
- Treating covariance size as easy to interpret is wrong because covariance depends on the units and scales of both variables.
- Forgetting to subtract the means is wrong because covariance is based on paired deviations, not on the raw x and y values themselves.
- Using n instead of n - 1 for sample covariance is wrong when estimating covariance from a sample because n - 1 gives the standard unbiased estimator.
- Assuming positive covariance proves causation is wrong because covariance shows association, not that one variable causes the other.
Practice Questions
- 1 For the paired data (1,2), (2,4), (3,5), compute the sample covariance s_xy.
- 2 A data set has sample covariance s_xy = 18, standard deviation s_x = 6, and standard deviation s_y = 5. Compute the correlation r.
- 3 Two scatter plots both show positive covariance, but one uses height in centimeters and weight in kilograms while the other uses height in meters and weight in grams. Explain why comparing the covariance numbers directly can be misleading.