Exploratory Data Analysis, or EDA, is the first careful look at a dataset before formal modeling or hypothesis testing. It helps you summarize what the data look like, find unusual values, and choose useful graphs. EDA matters because real data are often messy, with missing entries, measurement errors, skewed distributions, and hidden patterns.
A good EDA can prevent misleading conclusions later.
Understanding Statistics: Exploratory Data Analysis
Before calculating anything, identify what each column represents. Some columns hold categories, such as bus route or eye color. Others hold numerical measurements, such as travel time or temperature.
These types need different summaries and graphs. Check the unit for every measurement. A height recorded in centimetres cannot be mixed with one recorded in metres without conversion.
Look for impossible entries, such as a negative age or a test score above the maximum. Missing values need attention too.
A blank might mean a student skipped a question, a sensor failed, or the value was never measured. Those cases can affect results in different ways.
The shape of a distribution tells a story that one average cannot tell alone. A symmetric distribution has similar spread on both sides of its center. A skewed distribution has a longer tail on one side, often because a few values are much larger or smaller than most others.
House prices commonly have right skew because a small number of very expensive homes pull the upper tail outward. In such cases, the median often describes a typical value better than the mean.
Compare measures of center with measures of spread. Two classes can have the same average score while one class has scores packed closely together and the other has scores scattered widely.
Graphs help reveal patterns that tables can hide. A histogram groups numerical values into intervals, making clusters, gaps, and skew easier to see. The chosen interval width matters.
Very wide intervals can hide important structure, while very narrow intervals can make random variation look meaningful. A box plot is useful for comparing several groups because it shows the middle half of each group and flags values far from the rest. A scatter plot is used for two numerical variables, such as hours of sleep and reaction time.
A correlation near zero does not prove that no relationship exists. The pattern may curve, or separate into groups. Even a strong correlation does not show that one variable causes the other.
Students meet exploratory analysis whenever they track sports results, compare screen time with sleep, study survey responses, or investigate a science experiment. Keep the context in view while working. An unusual value may be a typing mistake, but it may be a real and important event.
Do not delete it just because it looks inconvenient. Check the original source when possible, then record any decision you make. Watch for small samples, biased groups, and measurements taken under different conditions.
A survey of one friendship group cannot describe every student. Good exploratory work leaves a clear trail from raw data to the graphs and summaries used later.
Key Facts
- Mean = (sum of all values) / n
- Median is the middle value after sorting the data from least to greatest.
- Range = maximum value - minimum value
- Interquartile range: IQR = Q3 - Q1
- A common outlier rule is value < Q1 - 1.5IQR or value > Q3 + 1.5IQR.
- Correlation r measures the strength and direction of a linear relationship, with -1 <= r <= 1.
Vocabulary
- Exploratory Data Analysis
- Exploratory Data Analysis is the process of summarizing and visualizing data to understand its main features before formal modeling.
- Distribution
- A distribution describes how often different values or ranges of values occur in a dataset.
- Outlier
- An outlier is a data value that is unusually far from the rest of the data.
- Scatterplot
- A scatterplot is a graph that shows paired numerical data as points to reveal relationships, clusters, and trends.
- Correlation
- Correlation is a statistic that describes how strongly two numerical variables move together in a linear pattern.
Common Mistakes to Avoid
- Using only the mean, which is wrong when the data are strongly skewed or contain outliers because the mean can be pulled toward extreme values.
- Ignoring the units and context, which is wrong because a number has little meaning unless you know what was measured and how it was collected.
- Treating correlation as causation, which is wrong because two variables can move together due to a lurking variable or coincidence.
- Deleting outliers automatically, which is wrong because outliers may represent errors, rare events, or important discoveries that need investigation.
Practice Questions
- 1 For the dataset 4, 6, 7, 7, 8, 10, 22, find the mean, median, range, and identify the value most likely to be an outlier.
- 2 A class has quiz scores 12, 15, 16, 18, 20, 21, 21, 24. Find Q1, Q3, and IQR, then use the 1.5IQR rule to check whether any score is an outlier.
- 3 A scatterplot of study time versus exam score shows an upward trend with one student who studied many hours but scored low. Explain what pattern, outlier, and possible next question an EDA should report.