I have a scenario where I get negative R2 values for a linear fit. My algorithm relies on a goodness of fit measure to improve the model. But in the specific cases that I am working on, the R2 values are never becoming positive, let alone close to 1. What are the other measures by which I can quantify the goodness of fit?
PS: If it helps, then I am getting negative values because the data is non-stationary. I am trying to break the data to stationary chunks by looking at R2 values. But they seem to never become positive.
.
negative R2 maybe the result of a constant term missing in the model
http://web.maths.unsw.edu.au/~adelle/Garvan/Assays/GoodnessOfFit.html
.
How many independent variables do you have? Do you see pairwise linear fit between your dependent variable and each independent variable? Why do you think a linear fit would best describe your data?
ISSUE: How to resolve negative R2?
BASIC DEFINITIONS: The following definition applies in R2 calculation for time series:
(1) SST = Sum(yt - y*)2
... where t = 0 to T; T = sample size; yt = observed value at period t; y* = mean value. No possibility of negative number here because of the square.
(2) SSE = Sum(yt - yt^)2
... where yt^ = expected value. No possibility for case of negative value here either.
(3) MSE = [(Sum(yt - yt^)2] / T - k or SSE / T - k
... where T = sample size; and k = number of parameters. There is a possibility to have negative value when the sample is too small and there are many parameters, i.e. T < k. The problem may come from the denominator term: (T - k).
(4) R2 = 1 - (SSE / SST)
Here, we see that R2 would be negative if (SSE / SST) is larger than 1. This problem come from two possibilities: (i) SSE is positive and large, or (ii) using MSE instead of SSE for equation (4) and MSE is negative due to sample size smaller than number of parameters: T - k if k > T. SST and SSE cannot be a negative because they are results of squaring the difference: (yt - y*)2 and (yt - y^t)2.
SOLUTION BY AMEMIYA ADJUSTED R2: The adjusted R2 proposed by Amemiya is given by:
(5) R2adjust = 1 - [[(T - 1) / (T - k)](1 - R2)]
Assume that your do not have a problem of "unusually small sample" so that T < k, then the problem should be solved. Suppose that the current problem of negative R2 is, say R2 = -0.80 (negative 0.8). Plug this into Amemiya's adjusted R2 equation; assume T = 20 observations and k = 5 parameters, then ...
R2adjust = 1 - [[(20 - 1) / (20 - 5)](1 - (-0.80)]
= 1 - (19 / 15)(1.8)
= (1 - 1.27)(1.8)
=0.27(1.8)
= 0.48 ....... a positive number.
Notice that under this adjustment, the sign changes and the level of R2 also changes.
Kolmogorov-Smirnov Test finds out the “fit” of an observed pattern of frequencies of ordinal data to an expected pattern.
Chi-square goodness-of-fit test finds out the “fit” of an observed pattern of frequencies of nominal data to an expected pattern.
- Badges
- Science topic
- Similar topics
- Geoscience
- Cartography
- Mapping