I first want to know how to compute the standard error of binary logistic regression's coefficient for a independent variable.
In linear regression, the text books explain how to compute the standard error of regression's coefficient for simple linear regression. By using matrix form, the standard error of regression's coefficient for multiple regression are computed.
However, I cannot find any source explaining the procedure to compute standard error of regression's coefficient with one independent variable for binary logistic regression. They only show to how to compute standard error of regression's coefficient for multiple logistic regression model.
You have to learn something about likelihood theory. In a sense, the standard error measures the curvature of the log-likelihood function its maximum. It is defined as the square-root of the reciprocal of the Fisher-Information (evaluated at the maximum likelihood). The Fisher-Information, in turn, is the negative Hessian of the log-Likelihood (i.e. the matrix of the second derivatives).
I like to recommend this book:
Yudi Pawitan: In All Likelihood: Statistical Modelling And Inference Using Likelihood.
ISBN:0199671222
If your dataset is not too large, then you could use a jackknife or bootstrap procedure to estimate the standard error of any coefficient. Let's assume that you have an unbiased sample from the population you are studying. You can take many samples (with replacement) from your sample (consider it a population for the purposes of this evaluation) For each sample calculate the parameter estimate you wish to study. Do this a few hundred times and the resulting distribution of estimates of your parameter will be an approximate sampling distribution of that statistic. The mean of that distribution will be it's expected value and the standard error will be approximated by the standard deviation of that estimated sampling distribution.
A similar strategy involves sampling without replacement. Take out the first observation and calculate your analysis using the remaining N-1 observations. Record your statistic (parameter) of interest. Replace the observation you removed and take out the next observation and repeat the analysis. This will result in N analyses, each based on N-1 observations. The distribution of parameter estimates for the entire group of analyses will be an approximation of the underlying sampling distribution. The standard deviation of that approximate sampling distribution will be (a slightly biased) estimate of the standard error of that statistic.
If you are writing your own computer program or have access to intermediate technical calculations, and the problem is mathematically tractable, then the analytically derived solutions work well. When the problem is not tractable, such as many things in the robust statistics arena, then the trusty jackknife simulation may be a viable alternative. Some programs build in jackknife or bootstrap methods. Even when they don't, it is at least possible to do it entirely manually.
For your specific case I would consult with your local statistician to see if you could find a computer program that produces exactly what you want. Check SAS or R or SYSTAT. With a little work, you can probably find what you want and need. I believe that each has both the log-linear regression and bootstrap/jackknife or other resampling methods built in.
See:
https://en.wikipedia.org/wiki/Resampling_(statistics)
On the other hand, if your interest is in understanding how and where the standard error comes from, then Professor Wilhelm's suggestion is an excellent one.
Here are a couple of references that you might find useful in defining estimated standard errors for binary regression. The first is a relatively advanced text and the second is an intermediate level text. These are both classics and if your library does not have them, ask your librarian to get them for you. Used copies should also be available at reasonable prices.
Bishop, Feinberg & Holland. (1975). Discrete Multivariate Analysis. Boston: MIT Press.
Hosmer & Lemeshiow. (1989). Applied Logistic Regression. New York: Wiley (Wiley Series in Probability and Mathematical Statistics).
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA