Suppose that I want to relate three variables, and a causal link between each two of them is proposed depending on the available literature, and I may too propose that one of these variables is the mediating variable.
So my thesis supervisor told me that to test cause-effect relationships we can use regression.
But I am not sure what is the research design then?
I know studying causal relationships can apply in experimental, quasi-experimental and ex post facto designs, depending on the variables and the conditions. But I don't know what research design allows multiple regression?
Look at the link below. In a Multiple Regression Analysis, there are eight Assumptions. Study them.
https://statistics.laerd.com/spss-tutorials/multiple-regression-using-spss-statistics.php
Multiple linear regression can be used pretty much whenever you have a continuous dependent variable (the general linear model entails a lot of other approaches as a special case, for example, t test and ANOVA).
You can also study mediation models using multiple regression, but it is more practical to use path analysis in that case. Path analysis can easily handle designs with multiple dependent and/or intervening variables. Path analysis allows you to estimate all path (regression) coefficients in a single step when you have one or more mediator variables and to test the indirect (mediated) effect for significance. In multiple regression, this is more complicated because you have to run separate regression models for the mediator(s) and the outcome variable(s).
Regression analysis is used to describe the relationships between a set of independent variables and the dependent variable. Visit following link for more detail
https://statisticsbyjim.com/regression/when-use-regression-analysis/
I would like to add to Christians answer that it is true that multiple regression and /or more specifically mediation / path analyses can be used to test for causal effects. But this alone is not sufficient to prove causality. For this you need a strong theory and an appropriate research design, which allows you to test the causality of your focal variables and to exclude the influence of other variables. So, you have to put much more effort in your research, BEFORE collecting the data.
This a very useful tutorial on the roles of covariates http://dagitty.net/learn/graphs/roles.html
The consequences are important:
"the covariates in a statistical analysis can have a variety of different roles from a causal inference perspective: they can be mediators, confounders, proxy confounders, or competing exposures. If a suitable set of covariates can be identified that removes confounding, we may proceed to estimate our causal effect using a multivariable regression model. In linear regression models, there are only two types of variables: the dependent variable (DV) and independent variables (IVs, or predictors). No further distinction is made between the IVs – specifically, the exposure is by no means a "special" IV and is treated just like any other IV. Thus, as you can see, there is a conceptual mismatch between causal theory (DAG) that leads us to formulate a multivariable regression model (that highlights the exposure-outcome relationship and associated statistical adjustment for confounding) and the regression model itself. This conceptual mismatch can easily lead to misinterpretation of the results from a multivariable regression model. "
(source: http://dagitty.net/learn/graphs/table2-fallacy.html)
There is no escaping of the need to think; you just cannot press a button.
YOU will use multiple linear regression if you have one continuous dependent variable. If you have more than one continuous variable, the appropriate model is multivariate model.
Hello Nour Salim. Before you get too excited about inferring causation (should you eventually find a statistically significant indirect effect), take a look at some of Klaus Fiedler's articles on mediation analysis. Here is one, for example:
Article Unwarranted inferences from statistical mediation tests – An...
HTH.
Kelvyn Jones points you in the direction of daggitty, which allows you to draw causal diagrams of the relationships between your variables, and is an incredibly useful tool in clarifying your hypothesis and translating it into a testable statistical model.
However, his most important contribution, and one that I shall happily repeat, is the dictum –
There is no escaping of the need to think; you just cannot press a button.
Or, as I tend to put it : The software should work; the analyst should think.
Hi Nour Salim . You can use multiple linear regression when you want to know: How strong the relationship is between two or more independent variables and one dependent variable (e.g. how rainfall, temperature, and amount of fertilizer added affect crop growth). See the link: https://www.scribbr.com/statistics/multiple-linear-regression/
Also, Multiple regression analysis allows researchers to assess the strength of the relationship between an outcome (the dependent variable) and several predictor variables as well as the importance of each of the predictors to the relationship, often with the effect of other predictors statistically eliminated. See the link: https://www.sciencedirect.com/topics/economics-econometrics-and-finance/multiple-regression-analysis
You can use the software SPSS for the multiple regression analysis . See the link for details: https://statistics.laerd.com/spss-tutorials/multiple-regression-using-spss-statistics.php
When you have more than 1 predictor / explanatory variables, and the correlation between the response and the predictors is linear. Neural Networks follow a similar procedure, but you can't explain the predictors.
Regression perfectly applies to a Cause-and-Effect/ Impact evaluation Design, be it experimental or not, for granted that causality is on stake. If the latter does not apply, use a correlation or any other associative (dissociative) statistical measurement. Multiple régression fits to both experimental and non-experimental designs, where causation is on stake.
Multiple regression is a higher level of the ANOVA method results and works well for getting insight casual relationships among independent variables > a dependent variable.
Be careful trying to make causal statements from regression output. Here is a quote that you might ask your supervisor to comment on:
Naive usage of regression modeling was blamed for nearly all the ills of sociology, everything from stripping temporality and context from the mainstream, \dots{} the suppression of attention to explanatory mechanisms, \dots{} the denial of causal complexity, \dots{} and the destruction if mathematical sociology. \\ \phantom{k} \hfill \citet[p.~13]{MorganWinship2015}
This is from the Morgan and Winship textbook on causes. Several other good books on causal inference exist.
Further, looking at some of the previous statements, you should be cautious treating these as useful and in some cases may mislead you. For example, from @Anuraj Nayarisseri
"Regression perfectly applied to a Cause-and-Effect" no, see quote above and any recent textbook on causation
"the latter does not apply, use a correlation" I am interpreting this as saying this does apply to correlation. If he means bivariate correlation, this at least is different than the conditional associations of the multiple regression, but it is unclear.
"Multiple regression is a higher level of .the ANOVA method" which again is misleading as ANOVA is just "a convenient method of arranging the arithmetic" \citep[p.~52](Fisher1934wishart} (this is from Fisher's discussion of Wishart's paper in JRSS in 1934) for presenting regression results.
Please, see the following helpful link:
https://statisticsbyjim.com/regression/when-use-regression-analysis/
Maybe you can consider the recursive least squares algorithm (RLS). RLS is the recursive application of least squares (LS) regression algorithm, so that each new data point is taken in account to modify (correct) a previous estimate of the parameters from some linear (or linearized) correlation thought to model the observed system. The method allows for the dynamical application of LS to time series acquired in real-time. As with LS, there may be several correlation equations with the corresponding set of dependent (observed) variables. For RLS with forgetting factor (RLS-FF), acquired data is weighted according to its age, with increased weight given to the most recent data.
Years ago, while investigating adaptive control and energetic optimization of aerobic fermenters, I have applied the RLS-FF algorithm to estimate the parameters from the KLa correlation, used to predict the O2 gas-liquid mass-transfer, hence giving increased weight to most recent data. Estimates were improved by imposing sinusoidal disturbance to air flow and agitation speed (manipulated variables). The power dissipated by agitation was accessed by a torque meter (pilot plant). The proposed (adaptive) control algorithm compared favourably with PID. Simulations assessed the effect of numerically generated white Gaussian noise (2-sigma truncated) and of first order delay. This investigation was reported at (MSc Thesis):
Thesis Controlo do Oxigénio Dissolvido em Fermentadores para Minimi...
When we have one dependent variable and two or more independent variables, the advisable statistical data analyses technique is a multiple regression analysis.
- Badges
- Science topic
- Similar topics
- Psychology