MISUSE OF LINEAR MODELING

Linear regression has often been misused to be the holly grail of proving relationship forecast. There is always a built-in tendency to conclude that Y changes at the rate of “b” for every unit change in X, reading from the linear equation: Y + a + bX + c. Actually, this equation is used for a rudimentary forecast given an array of X: (x1, x2, …,xn) and Y: (y1, y2, …, yn). Some financial or economic data may not be linear. For non-linear data, linear modeling may not be appropriate.

CORRELATION COEFFICIENT

When working with financial information, some data may not behave in a straight line fashion, especially cost functions or productivity. Instead of using linear equation as a starting point, one may look at correlation coefficient as a tool to measure the association between two data arrays. In so doing, use the following as a guide:

X quantitative: Y quantitative = Pearson correlation

X quantitative: Y ordinal = Biserial correlation

X quantitative: Y nominal = Point biserial correlation

X ordinal: Y quantitative = Biserial correlation

X ordinal: Y ordinal = Spearman rho or tetrachoric correlation

X ordinal: Y nominal = Rank biserial correlation

X nominal: Y quantitative = Point biserial correlation

X nominal: Y ordinal = Rank biserial correlation

X nominal: Y nominal = Phi, L, C & Lambda

The correlation coefficient interval runs between negative one (-1) and positive one (+1). The association may be read from this range. For forecast, if linear regression is not appropriate, then consider non-linear curve fitting.

NON-LINEAR CURVE FITTING

For non-linear data dispersion, it is necessary to look beyond linear modeling. One needs non-linear curve fitting. One approach to curve fitting is to increase the order of a polynomial equation. Basic definition of a polynomial follows: (1) the line equation y = a + bX + c is a polynomial of the first order. It is a best fit equation for two points; hence, the shortest distance between two points is a straight line; (2) the equation y = aX^2 + bX + c is a second order polynomial; it is the best fit of a curve that is bounded by three points; and (3) the equation y = aX^3 + bX^2 + cx + d is a polynomial of the third order; it is a best fit polynomial connecting four points. Each point referred hitherto is known as a constraint. Each constraint may be a point, an angle or a curvature.

ATTACHED FILE

A file is attached for exercise in nonlinear curve fitting.

https://drive.google.com/file/d/0B772RxSNNsoVa1FFazBYdmZDNzg/edit?usp=sharing

Technically, linear regression (with OLS or ML) will be most efficient if its assumptions are satisfied - which you can (partially) test for by the standard specification-test toolbox. However, it might be fruitful to start with some exploratory non-parametric data analysis in order to transform the data adequately and to establish a linear approximation of the model/the conditional expectation.

Of course, if you have in mind a non-linear model which can't be linearized, you could try to estimate it directly by nonlinear regression...

MISUSE OF LINEAR MODELING

Linear regression has often been misused to be the holly grail of proving relationship forecast. There is always a built-in tendency to conclude that Y changes at the rate of “b” for every unit change in X, reading from the linear equation: Y + a + bX + c. Actually, this equation is used for a rudimentary forecast given an array of X: (x1, x2, …,xn) and Y: (y1, y2, …, yn). Some financial or economic data may not be linear. For non-linear data, linear modeling may not be appropriate.

CORRELATION COEFFICIENT

When working with financial information, some data may not behave in a straight line fashion, especially cost functions or productivity. Instead of using linear equation as a starting point, one may look at correlation coefficient as a tool to measure the association between two data arrays. In so doing, use the following as a guide:

X quantitative: Y quantitative = Pearson correlation

X quantitative: Y ordinal = Biserial correlation

X quantitative: Y nominal = Point biserial correlation

X ordinal: Y quantitative = Biserial correlation

X ordinal: Y ordinal = Spearman rho or tetrachoric correlation

X ordinal: Y nominal = Rank biserial correlation

X nominal: Y quantitative = Point biserial correlation

X nominal: Y ordinal = Rank biserial correlation

X nominal: Y nominal = Phi, L, C & Lambda

The correlation coefficient interval runs between negative one (-1) and positive one (+1). The association may be read from this range. For forecast, if linear regression is not appropriate, then consider non-linear curve fitting.

NON-LINEAR CURVE FITTING

For non-linear data dispersion, it is necessary to look beyond linear modeling. One needs non-linear curve fitting. One approach to curve fitting is to increase the order of a polynomial equation. Basic definition of a polynomial follows: (1) the line equation y = a + bX + c is a polynomial of the first order. It is a best fit equation for two points; hence, the shortest distance between two points is a straight line; (2) the equation y = aX^2 + bX + c is a second order polynomial; it is the best fit of a curve that is bounded by three points; and (3) the equation y = aX^3 + bX^2 + cx + d is a polynomial of the third order; it is a best fit polynomial connecting four points. Each point referred hitherto is known as a constraint. Each constraint may be a point, an angle or a curvature.

ATTACHED FILE

A file is attached for exercise in nonlinear curve fitting.

https://drive.google.com/file/d/0B772RxSNNsoVa1FFazBYdmZDNzg/edit?usp=sharing

Highly appreciate your answers. But how to come to an understanding of whether linear model will be more suitable or non linear model? And what about structural equation modelling approach, can it be used instead of regression?

How many firms and independent variables per firm are there in your data set?

Dear Utkrsh, If your independent variables has correlation or cause-effect relations, use Path analysis (assume the relations are all linear, or if not, transformed).

Most parametric alternatives discussed so far appear to assume a large data sets (N >30 cases). However, if this researcher's data set is in fact relatively small, i.e, total N

I have more than 200 firms but i may have to group them by industry which may result in less than 30 firms per industry. So if I want to apply the technique separately for different industry, I will have less sample size

There are many new and improved methods. See

Wilcox (2012) Introduction the Robust Estimation and Hypothesis Testing, Elsevier

What is the dependent measure? Its scale of measurement.? As for industry type, I would consider it, at the moment, as another covariate for each firm, another independent variable.. Knowing more about the type of measure used to build your dependent variable can help serve as a guide to additional types of analyses.

An alternative approach that may be of interest to you is Data Envelopment Analysis.

As far as I know DEA is used for efficiency analysis. isn't it?

Edwin, the dependent variable is an efficiency measure between 0-1. independent variable are macroeconomic and firm specific financials. what do you suggest?

Regression splines

How is time represented? Are there multiple years/quarters of data represented for each firm? How many?

Structural equation modeling is best used in social science and medical studies for understanding the interrelationship between different variable with dependent and independent variables. Actually SEM is developed on the grounds of multivariate regression but serves in a better way than multiple regression. It can study the cause and effect of these variables simultaneously and well demonstrates in a easy way by using measurement models and path analysis diagrams.

You can use also Principal Component Analysis, that can return some information

I would have to agree with the SEM comment, but I would first suggest that you determine the line/curve that best fits your data. Just be aware that the path portion of SEM is really just a regression in disguise.

On a different note you may also consider looking at multilevel modeling that way you can next individual company data into the industry data and account for the grouping effect.

Utkarsh,

The best technique depends on the purpose of the analysis. Are you interested in differences between firms? Or you are only interested in the effects of the independent variables on the dependent variable?

Out of interest, how have you "calculated" your dependent variable...? Is it composite of entities (some combination of conceptually distinct entities), or is the DV a single (unidimensional) entity?

Try genetic algorithm instead, mainly for multivariate "type" regression. But the cost as you said in the avoiding of analytic functions thus the missing of thorough statistical analysis, that should be done besides and with your own experience for this. Best regards.

@guillermo I am more interested in effect of independent variables on dependent

@ john DV is a single unidimensional entity

Utkarsh,

If you don't have repeated measures in the independent variables, then the regressions proposed in previous comments are fine. But if you do, then there is a problem. For example, if you have as IV the profit of each company in each of the last 10 years, these 10 data points in each company are not independent. One option is to calculate the average of these 10 data points and run the typical regression analysis or other alternatives.

This is what I have been doing for a long time until I discover multilevel analysis (also known as hierarchical analysis or mixed effects models [note that this is NOT multinomial regression and is NOT hierarchical regression]). This analysis takes into account the levels of the data. In your case level 1 would be the observations, and level 2 the firms. You can use all your data points, and it does not matter if there are missing values.

Since you are not interested in the differences between firms, you should use firms as a random effect. The analysis will estimate the variance between firms (i.e., only one parameter).

In this link there are many tutorials of excellent quality: http://www.cmm.bris.ac.uk/lemma/

I hope this helps.

Happy New Year

Guillermo

@Guillermo it seems an interesting concept ....can u point me some more text related to this concept of multilevel analysis?

We don't yet know whether multi levels are well suited without information about the representation of time/observations per firm, and how such possible multiple observations might be structured... So far all anyone knows is that there are 200 firms, and one dependent measure, and unknown multiple observations, if any... Tell us more about what has been designed with the data collection.

Also we need info on the kinds of variables in the theory you have constructed. For instance the DV might be nominal or ordinal... or be constrained in terms of minimum or maximum values...

We have a dependent variable which has been separately calculated and refers to efficiency (can be more than one as there will be some super efficient). The independent variables are firm specific data like age of firm and some firm firm financial ratios of firms. All are yearly data. The independent variables will also have some macro economic data like inflation, GDP etc ( yearly data gain). Now I want to check efficiency's relationship with these variables. How should I progress?

Stochastic Frontier Analysis may be an alternative

If you have multiple yearly representations of 200+ firms and their corresponding financial outcomes, I think Guillermo's suggestion to use multi level/hierarchical anaysis would be best for your design. Best of luck.

Linear regression uses two restrictive assumptions: linearity and a causal ordering from covariates ("independent") to the response. Relaxing the first assumption leads to nonlinear procedures, relaxing the second one to canonical correlation analysis. Relaxing both assumptions simultaneously may make modeling difficult.

Linear regression is a traditional case of statistical learning. A larger family of models is described in the book 'Introduction to Statistical Learning with applications in R' and the theory in 'Elements of Statistical Learning'. Both books can be freely downloaded from Hastie's website http://www.stanford.edu/~hastie/pub.htm. Some models such as random forests are meant for high precision prediction rather than explanation. It comes down to finding a function of the input variables that optimally predicts the output variable(s). I guess that is your interest too.

Hi Utkarsh.

For mixed models a great book is "Data analysis using regression an multilevel/hierarchical Models" by Andrew Gelman.

Andrew has a blog site too. http://andrewgelman.com/ . I like MRP (Multilevel regression and poststratification) technique. It help me in many cases

Can any one please explain that how rolling regression can be applied in eviews to the variables of inflation, growth and financial depth.

I would suggest to try Panel Cointegration model...read

what is the alternate option for multipile linear regression analysis in a cross sectional study , where to  finding out the  Associations between the independent and outcome variable ?