Have a look at robust regression methods.

You will definitely want to review statistical methods that are used in epidemiology.

EDITED: If your dependent variable is disease:yes-or-no, then a common approach is to use Poisson regression, and use the log of the population of each district as an "offset".

You shouldn't be using common linear regression methods for this kind of count data.

Hi, Robust logistic regression may be what you need. See the attached paper. The program is now in the CRAN library. Best wishes, David Booth

Of course, you first have to determine what it is you are trying to figure out.

If you are using R, you might take a look at the epi and epitools packages.  Epidemiology is really not my field, so I don't know what analyses you would want to do. 

There is a free book online by Aragón that covers epidemiology using R.  It seems to cover some basic topics, so might be worth a look even if you don't use R.

I agree with Salvatore. What do You want to research with Your data? 

May be some factors, may be annual changes?

Do You know population size at each district ?  

Dear Salvatore and Olga,

I have computed cumulative incidence rates of each county using the population of them through 15 continuous years, now I am looking for a method to model the combination of several climatic factors to predict and estimate the impact of each independent parameter on the disease. I am able to use R, STATA and SPSS softwares. by taking all these into the account, let me know if there is any suggestion.

moreover, I have used ordered Probit-Logit model, but I don't know if there is any assumption for this analysis or not.

Dear Mohsen, 

for that kind of data I use  JoinPoint regression software https://surveillance.cancer.gov/joinpoint/

It is free.

The file should be managed the special way. Examples and guidance are at the site.

If You tells about cumulative incidence rates - survival analysis, but I am not sure that You follow-up each observation.

I think for the appropriate selection of regression model, test the probability distribution of the response variable.

Dear Muhammad,

My problem exactly is the non-normal distribution of the respond variable.

It is badly non-normal and unfortunately no normalizing method can make it normal.

I have about 400 counties which about 270 of them has 0 value and 3-6 of them are have much greater values like 300-400, and the rest of counties have values in the range of 2-8.

*Note that none of my values should be omitted.

Then for your case, Robust or non-parametric regression may give better results.

Dear Mohsen,

From my experience, the Generalized Additive Modeling is a very good algorithm for modeling with/or without outliers. It also work with data of non-normal response distribution.

You can implement it in R through gam and mgcv packages.

Combination of both  Residual Analysis and Multiple Regression analysis

Find Outliers That Matter

Robust regression techniques will reduce the effects of the so-called outliers but ignore the issue of the outliers. A deeper analysis may be valuable.

Outliers result from the following: The data are wrong, the model is wrong or both. Both sources of the problem should be investigated. In your case the response variable may be poorly measured. This should be considered.

The term "model" here is general and applies to the subject matter context of the problem as well as the mathematical form of the model.

Finally investigating outliers takes time and in the end may not be resolved for many reasons. It is important to find "outliers that matter". Not all outliers matter in the sense that no matter whether the outliers are in or out of that analysis the overall results and conclusions do not change. If the conclusions don't change then the suspected outliers don't matter.

You could also try something like Random Forests or Boosted Regression Trees which are a form of machine learning using binary partitioning techniques. They are not dependent on normality assumptions.

In a regression, the errors must be normal, not the response variable.

Follow.