I have a dataset and want to study the factors that positively or negatively affect a target variable. Based on features related to academic performance, demographics, etc., I want to aggregate these features into different factors and determine which factors most influence the target variable.
thank you very much in advance
Possibly, you could find inspiration in
@incollection{KarKul:88,
author={M. K\'{a}rn\'{y} and R. Kulhav\'{y}},
title={Structure determination of regression-type models for adaptive
prediction and control},
year=1988,
booktitle={{B}ayesian Analysis of Time Series and Dynamic
Models},
editor={J.C. Spall},
publisher={Marcel Dekker},
address={N.Y. },
}
Certainly, Here is that compact instruction;
Steps for Factor Analysis
- Data Preparation: Clean and standardize data.
- Perform Factor Analysis: Select the amount of factors, and extract them.
- Factor Rotation: To make this solution more interpretable we may now rotate this (varimax for example).
- Interpret Factors: Examine factor loadings.
- Create Factor Scores: Compute Scores for Each Data Point
- Analyze Influence on Target: Analyze the effect of factor scores in the target variable using regression
thank you very much Abdus Sobhan
I will try to follow these instructions
Abdus Sobhan, Can you please, propose some machine-learning techniques I should use to perform this task?
Mohamed el Jihaoui Of course, these are some machine-learning techniques to conduct factor analysis and look into their effect on the target variable.
Principal Component Analysis (PCA): Used for Dimensionality Reduction.
Factor Analysis (FA): We use it to discover the latent factors.
Linear Regression: For examining the impact of the factor score on the target variable.
Random Forest: For determining feature importance and interactions.
Gradient Boosting Machine(GBM): For prediction of an additional feature or hard code feature influence idea.
Support Vector Machines (SVM): for Factor scores with Classification tasks.
Hello Mohamed el Jihaoui ,
I recommend using the following feature selection approach. After addressing data issues such as feature standardization, employ a method like BestFirst to systematically explore subsets of features based on their predictive power.
Next, an evaluator, utilizing a basic classifier algorithm such as a decision tree, assesses the merit of each subset by measuring its predictive performance against the chosen evaluation metric. Once the optimal subset of features is identified, it is exclusively used to train the specified classifier, which also employs a decision tree as the base classifier.
To ensure reliable and generalizable results, conduct these steps within a 10-fold cross-validation framework for both feature selection and classification.
NB: PCA won't be useful in your case. This is because it transforms the data into a new set of uncorrelated features, which can be hard to interpret in terms of the original features.
Kind regards,
Dr. Samer Sarsam
- Badges
- Science topic