Feature engineering is the process of turning raw data into useful inputs that a machine learning model can learn from. Raw data can be messy, incomplete, or hard for a computer to compare directly. Good features help reveal patterns, such as how study time relates to test scores or how pixels relate to an image label.
This matters because better features can make an AI system more accurate, fair, and easier to understand.
A feature engineering pipeline usually starts with cleaning data, then transforming it into numbers, categories, or measurements a model can use. Common steps include filling missing values, scaling numbers, encoding categories, and creating new features from existing data. For example, a model may learn better from “hours slept last night” and “hours studied this week” than from a messy student survey spreadsheet.
Feature engineering connects computer science with statistics because it uses both logical data processing and mathematical pattern finding.
Key Facts
- A feature is an input variable used by a machine learning model, such as age, height, temperature, or word count.
- Feature engineering turns raw data into cleaner, more useful features before training a model.
- Normalization can scale values with x' = (x - min) / (max - min).
- Standardization can rescale values with z = (x - mean) / standard deviation.
- Categorical data often needs encoding, such as red, blue, green becoming 0, 1, 2 or separate yes/no columns.
- Better features can improve model accuracy, but biased or irrelevant features can lead to poor predictions.
Vocabulary
- Feature
- A feature is a measurable input used by a machine learning model to make a prediction or classification.
- Raw data
- Raw data is information collected before it has been cleaned, organized, or transformed for analysis.
- Encoding
- Encoding is the process of converting non-number data, such as categories or labels, into numbers a model can use.
- Normalization
- Normalization is a scaling method that changes numerical values to a common range, often from 0 to 1.
- Training data
- Training data is the set of examples a machine learning model uses to learn patterns.
Common Mistakes to Avoid
- Using raw categories without encoding is wrong because most machine learning models cannot directly calculate with words like small, medium, and large.
- Forgetting to scale features with very different units is wrong because a large-number feature like income can overpower a smaller-number feature like rating.
- Filling missing values without thinking is wrong because replacing blanks with 0 can accidentally add false meaning to the data.
- Choosing features that leak the answer is wrong because the model may look accurate in practice problems but fail on real future data.
Practice Questions
- 1 A dataset has test scores with min = 50 and max = 100. Use x' = (x - min) / (max - min) to normalize a score of 80.
- 2 A feature has mean = 20 and standard deviation = 5. Use z = (x - mean) / standard deviation to standardize x = 35.
- 3 A model predicts whether a plant is healthy using color, height, number of leaves, and the label healthy or not healthy. Explain which input should not be used as a feature during training and why.