I want to develop speed of vehicle using multiple linear regression model. For instance, I have 457 of sampling. I used 300 of sampling for model development and keep 157 sampling to check model accuracy. Based on proposed model, I compute predicted speed. Then which test can I perform to compare predicted and observed speed?
You could use an indicator like RMSE to evaluate accuracy.
Good question, one of my favorite topics :-) Basically I know of two options:
- single indicators (RMSE, as mentioned above, R2, etc., see any textbook)
- performance indicator such as CURE technique of Ezra Hauer (see http://www.oocities.org/[email protected]/Pubs/ICTCT97TwoTools.pdf)
It would be nice if you provided more information about your study conditions, design, results...
Chea -
You might want to look into "cross-validation" techniques.
I think I'd like to take the difference between observed and predicted, for those 157 cases, and find the estimated mean of that distribution (hopefully near zero), and from the population standard deviation approximated by those data, estimate a standard error for that mean, and a confidence interval around it, possibly one assuming normality or t, and another using Chebyshev's Inequality. The mean would represent the bias, where hopefully no appreciable bias would exist.
You could estimate the variances of the prediction errors. That can be used anytime, and though its construction assumes no bias, in practice, bias would impact it and inflate the resulting estimate in the correct direction. Software may do this for you, but be sure the software is doing what you think it is doing. One example would be SDSI in SAS PROC REG, where SDSI is the square root of the prediction error.
Well, perhaps you'll get some better suggestions from some people with more experience with this, but if I understand what you want, then I think the above might be useful. But I would stay away from hypothesis tests and stick with confidence intervals and prediction intervals, as p-values do not stand alone, and require a type II error analysis or some other sensitivity analysis. Prediction intervals and confidence intervals are more practically interpretable. All are impacted by sample size, but the impact on the p-value is often mistakenly overlooked. That is why many people use a threshold of 0.05 everywhere, which is a very bad idea.
See below please:
Press release for the American Statistical Association:
http://www.amstat.org/newsroom/pressreleases/P-ValueStatement.pdf
My letter in The American Statistician:
https://www.researchgate.net/publication/262971440_Practical_Interpretation_of_Hypothesis_Tests_-_letter_to_the_editor_-_TAS
Cheers - Jim
Article Practical Interpretation of Hypothesis Tests - letter to the...
To check the accuracy in multiple linear regression , Use the variance ,If variance score is near about the 1 is perfect prediction
print('Variance score: %.2f' % regressor.score(X_test, y_test))
Detail example of the linear regression
# -*- coding: utf-8 -*-
"""
Created on Thu May 18 16:26:16 2017
@author: Neeraj Verma
"""
# Simple Linear Regression
# Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
# Importing the dataset
dataset = pd.read_csv('Salary_Data.csv')
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, 1].values
# Splitting the dataset into the Training set and Test set
from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 1/3, random_state = 42)
# Fitting Simple Linear Regression to the Training set
from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)
# Predicting the Test set results
y_pred = regressor.predict(X_test)
print('Coefficients: \n', regressor.coef_)
# The mean squared error
print("Mean squared error: %.2f" % np.mean((regressor.predict(X_test) - y_test) ** 2))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % regressor.score(X_test, y_test))
There is an impressive tool to do that easily, you can do the k-fold validation or training/testing validation also:
https://www.solver.com/xlminer-data-mining
https://towardsdatascience.com/linear-regression-using-python-b136c91bf0a2
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis