The data science workflow is a repeatable process for turning raw data into useful evidence. It matters because good conclusions depend on more than running a model or making a chart. A strong workflow begins with a clear question, uses careful data collection and cleaning, and ends with communication that helps people make decisions.
The process is iterative, so results often lead back to better questions or improved data.
Key Facts
- A clear statistical question identifies the population, variables, and goal of the analysis.
- Data cleaning includes handling missing values, correcting errors, removing duplicates, and checking units.
- Exploratory data analysis uses summaries and graphs such as mean, median, standard deviation, histograms, and scatterplots.
- Mean = sum of values / number of values.
- A common train-test split is 80 percent training data and 20 percent testing data for evaluating model performance.
- Prediction error for one case can be written as residual = observed value - predicted value.
Vocabulary
- Data Science Workflow
- A structured sequence of steps used to define a problem, prepare data, analyze patterns, build models, and communicate results.
- Exploratory Data Analysis
- The process of using graphs and summary statistics to understand patterns, outliers, and relationships in data.
- Feature
- A variable used as an input to a statistical model or machine learning algorithm.
- Model
- A mathematical or computational representation used to describe data, explain relationships, or make predictions.
- Residual
- The difference between an observed value and the value predicted by a model.
Common Mistakes to Avoid
- Starting with a vague question is wrong because the data, variables, and analysis method cannot be chosen well without a clear goal.
- Skipping data cleaning is wrong because missing values, duplicate records, and inconsistent units can create misleading patterns.
- Using the test data while building the model is wrong because it makes performance look better than it will be on new data.
- Reporting only a final number is wrong because conclusions need context, uncertainty, assumptions, and clear visual communication.
Practice Questions
- 1 A dataset has 1,250 rows. If 80 percent is used for training and 20 percent is used for testing, how many rows go into each set?
- 2 A model predicts house prices of 210,000 dollars, 315,000 dollars, and 420,000 dollars. The observed prices are 200,000 dollars, 330,000 dollars, and 400,000 dollars. Find each residual using residual = observed value - predicted value.
- 3 A team finds that a prediction model works well on old customer data but poorly on new customer data. Explain which parts of the data science workflow should be revisited and why.