Converting Raw Data into Features
What is Feature Engineering?
- Feature Engineering is a crucial step in the process of predictive modeling.
- It involves the transformation of a given feature with the objective of reducing the modeling error for a given target.
- In most cases, appropriate transformation of data is an essential prerequisite step before model creation.
Feature Engineering is the process of tranforming raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data.
— Professor Andrew Ng
- ML models, such as neural networks, accept a feature vector and provide a prediction.
- These models learn in a supervised fashion where a set of feature vectors with expected outputs are provided.
- Features are represented as real numbered vectors because the feature values must be multiplied by the model weights.
- Better feature result in faster training and more accurate predictions.
- Feature Vectors can be:
— numerical
— categorical
— bucketized
— crossed - Engineering new features from existing features is a common practice. These engineered features are essentially calculated fields based on the values of other features.
- Reducing the dimensions of your feature space will result in fewer relationships between variables so you are less likely to overfit your model.
- Feature Engineering is an art and iterative process. It involves domain knowledge, intuition, and trial and error.
Coming up with features is difficult, time-consuming, [and] requires expert knowledge. ‘Applied machine learning’ is basically feature Engineering.
— Professor Andrew Ng
What Makes a Good Feature?
- Be Related to the Objective: Don’t use random data.
- Be Known at Prediction Time: New values for features must be available so the model can make a prediction.
- Be Numeric with Meaningful Magnitude: The data has to be numerical because that is the form that is amenable to machine learning.
- Have Enough Examples: Without enough samples, the model cannot learn patterns.
- Bring Human Insight to Problem: You decide why a particular feature might matter for a particular problem.
Resources
Source Credit: https://medium.com/google-cloud/what-is-feature-engineering-b51e731b05a9?source=rss—-e52cf94d98af—4
