I have been doing some reading on data pre-processing and feature engineering including feature selection, feature importance and feature construction. My understanding is that Feature engineer is applied in data preprocessing stage. Additionally, feature importance is sometimes checked when modelling data with machine learning tools and sometimes its done internally the model.
My question:
Is feature engineering always implemented in pre-processing stage or sometimes could be performed in later stages?
Thank you,
Shaima
It depends on the purpose of the task and the initial data used for processing. If your system operates on raw data and makes high-level decisions, then you need to build a hierarchy of features, starting with low-level features and ending with high-level ones, such as symantic relationships between complex patterns. In this way, the features are extracted both at the initial stages and at the last stages of the processing. This is usually followed by a classifier or a module that makes the decision on high-level features. For example, this is the case in Convolutional Neural Networks.
It's normally done before training the model. So yes, at image preprocessing is correct. To be honest, in most cases you return to feature engineering after training. It can then be used to enhance the accuracy of you model.
Cheers, Raoul
Feature engineering basically consists in extracting features from raw data. Thus, it is used in the data preprocessing step, in order to generate a proper input dataset for your machine learning algorithm. Furthermore, you can iteratively go back to the feature engineering step and test the effect of different features on the model performance, using an exploratory technique such as grid search, as used in feature selection and hyperparameter tuning.
The basic purpose of feature engineering is to make the raw data more meaningful so that machine learning algorithms can understand it more easily. It does not matter you consider it preprocessing or else, but it must be done before training the classifier.
Remember that, whatever steps you do for training data must be done with testing data too before passing it to the model for prediction.
Please note that
my exam
Conference Paper PhD exam 1
has good information
about this.
Please look at my
First PhD exam
It is in my research.
I hope you like it.
Dear Shaima Mohammed Alghamdi ,
The purpose of pre-processing is to simplify the handling of large amounts of data. for instance, in the case of text mining, it is done by using certain natural language processing techniques (NLP), including text parsing, data purification, Part of Speach tagging (PoS). These strategies will minimize the usage of noisy data, which is the major reason for the performance deterioration of various algorithms of classification.
on the other hand, Feature Engineering is considered the most important step for building Machine Learning models. It assists in the handling of a huge amount of data; by selecting and extracting the most representative feature vector.
- Feature selection aims to speed up the used algorithms, improve learning accuracy, and enhance model comprehensibility. Besides, finding an optimal feature subset, for dimension reduction, by removing irrelevant and redundant information from the used dataset.
- Feature Extraction aims to give representative weight to each feature in the used feature vector.
So, First data must pass through a pre-processing step to minimize the noise to as minimum as possible, then comes the Feature Engineering to effectively build the used feature vector for different mining tasks.
- Badges
- Science topic