Feature engineering is the process of turning raw data into useful inputs that a machine learning model can learn from. Raw data can be messy, incomplete, or hard for a computer to compare directly. Good features help reveal patterns, such as how study time relates to test scores or how pixels relate to an image label.
This matters because better features can make an AI system more accurate, fair, and easier to understand.
A feature engineering pipeline usually starts with cleaning data, then transforming it into numbers, categories, or measurements a model can use. Common steps include filling missing values, scaling numbers, encoding categories, and creating new features from existing data. For example, a model may learn better from “hours slept last night” and “hours studied this week” than from a messy student survey spreadsheet.
Feature engineering connects computer science with statistics because it uses both logical data processing and mathematical pattern finding.
Understanding AI & Machine Learning: Feature Engineering Basics
A model does not understand the meaning behind a column. It only detects numerical relationships in the examples it receives. This creates an important design choice.
A date of 14 March 2025 is not usually useful as one large number. It may be more useful when separated into day of the week, month, holiday status, or time since an earlier event. For a shop, the day of the week can capture weekend buying habits.
For a school attendance system, the number of days since the term began may show a different pattern. Useful features represent parts of the situation that could affect the result.
Some features are made by combining measurements. A person’s height and weight can produce a ratio that may carry more information than either value alone in some health studies. A journey record with distance and travel time can produce average speed.
Text messages can be represented by word counts, message length, or the presence of selected words. Images can be described through pixel values, edges, or regions of color. Modern neural networks can often learn many image and text features themselves, but people still make important choices about data cleaning, labels, and what information the system is allowed to use.
Care is needed because a feature can accidentally reveal the answer. This is called data leakage. Imagine a model designed to predict whether a patient will be admitted to hospital.
A field entered after the admission decision, such as an assigned ward, would make the model appear very accurate during testing. It would fail in real use because that field is unavailable when the prediction is needed. Another common mistake is calculating an average or scaling rule using the full dataset before separating training examples from test examples.
Information from the test set then leaks into training. Rules for cleaning and transforming data should be learned from the training set, then applied unchanged to new data.
Features can affect fairness as well as accuracy. A postcode might help predict delivery time, yet it can indirectly reflect wealth, local services, or past discrimination. Removing an obvious sensitive field does not always remove bias, since other columns can act as substitutes.
Students should inspect where each feature came from, when it was recorded, and which groups may be represented poorly. They should compare errors across groups rather than trusting one overall accuracy score.
A simple feature that has a clear real-world meaning is often easier to check than a complicated feature with no explanation. When learning this topic, focus on the prediction task, the timing of every piece of data, and whether a feature would truly be available at the moment a decision is made.
Key Facts
- A feature is an input variable used by a machine learning model, such as age, height, temperature, or word count.
- Feature engineering turns raw data into cleaner, more useful features before training a model.
- Normalization can scale values with x' = (x - min) / (max - min).
- Standardization can rescale values with z = (x - mean) / standard deviation.
- Categorical data often needs encoding, such as red, blue, green becoming 0, 1, 2 or separate yes/no columns.
- Better features can improve model accuracy, but biased or irrelevant features can lead to poor predictions.
Vocabulary
- Feature
- A feature is a measurable input used by a machine learning model to make a prediction or classification.
- Raw data
- Raw data is information collected before it has been cleaned, organized, or transformed for analysis.
- Encoding
- Encoding is the process of converting non-number data, such as categories or labels, into numbers a model can use.
- Normalization
- Normalization is a scaling method that changes numerical values to a common range, often from 0 to 1.
- Training data
- Training data is the set of examples a machine learning model uses to learn patterns.
Common Mistakes to Avoid
- Using raw categories without encoding is wrong because most machine learning models cannot directly calculate with words like small, medium, and large.
- Forgetting to scale features with very different units is wrong because a large-number feature like income can overpower a smaller-number feature like rating.
- Filling missing values without thinking is wrong because replacing blanks with 0 can accidentally add false meaning to the data.
- Choosing features that leak the answer is wrong because the model may look accurate in practice problems but fail on real future data.
Practice Questions
- 1 A dataset has test scores with min = 50 and max = 100. Use x' = (x - min) / (max - min) to normalize a score of 80.
- 2 A feature has mean = 20 and standard deviation = 5. Use z = (x - mean) / standard deviation to standardize x = 35.
- 3 A model predicts whether a plant is healthy using color, height, number of leaves, and the label healthy or not healthy. Explain which input should not be used as a feature during training and why.