I have data set consist of x and y variable where I wan to perform maximum likelihood estimation (MLE) to fit a mean function and astandard deviation function o the data. I am estimating the the beta and alpha parameters using Maximum likelihood function in order to observe the mean and sigma trend in my data. And finally performing the optimization of the likelihood. Every time I encounter the issue of obtaining different values for the mean and sigma when passing different initial values, I find that my model becomes sensitive to the initial parameters. Is there any way to address this situation so that I can obtain the best values for the mean and sigma that automatically fit into my model?
P.S. I have applied different optimisation methods also but didn't work.
i have achieved required results by implementing it through other techniques but I want to implement it using MLE. How can I cope this issue ?
Non-linear models (from my experience) are notorious unstable for small sample sizes and marginal patterns. Initial values can be obtain by an initial "best estimate". How large is the number of observations you use and is there a clear pattern in the data? I often find no real solution rather than moving to JAGS or Stan. That being said, the standard 3 parameter sigmoidal equation (y=asymptote/(1+e(midpoint−x)/scale))) you describe is often most stable. Could you give more specifics?
Wim Kaijser I have clear sigmoid S shape pattern in my data.
Data set consist of almost 700 rows.
The goal is to estimate the best parameters for beta and alpha that I can observe the mean and sigma values for different values of X. For mean I have used 3 parameters function and for sigma I have used 2 parameters function.
After that I am computing likelihood and passing it to the minimize function by giving initial parameters as 1,1,1,1,1.
The issue rises that my model is very sensitive to initial parameters every time I pass different initial paramets I get different values for beta,alpha,mean and sigma which ultimate effects on PDF values
Does params[3:] mean parameters from index (3+1) up to index k (the total number of parameters), or from 3 to k?
1st three parameters will be assigned for beta and rest of 2 after 3 will be assigned to alpha
Sorry for the belayed response, would it work to fit the model with the nlme package? in theory you could first estimate some initial starting values for more complex models. But in the following example this is not needed.
#Parameter values
b0
By sigma trend, do you mean you are also fitting the standard deviation of the error term as a function of X? If so, could you try first with fixed variance for the error term: models with non-fixed variance may be more difficult to fit.
You may also check that your likelihood is realistic. For instance, sigmoid models are often used to data contrained between 0 and 1 (to a rescaling eventually), which is not compatible for a Gaussian random variable. So this may interfere with the MLE fit.
Last, if initial value has a significant impact, and if you really think you are not overfitting the data (one of the most common cause for unstability), you may use a grid search on the initial values to find the global minimum... You may also explore the shape of the MLE obtained as a function of the initial values to see if something appears...
Good luck,
Thank you all so much for your valuable suggestions. I have implemented grid search, and it worked very well.