Give us a dummy example  ..... The story doesn't have to be true if it correctly describes the problem. It should help us visualize the problem, and goals.

I don't like discarding data unless there is some obvious problem. An ice storm damaged part of the greenhouse and part of the greenhouse was frozen. Ok, discard those plants.

Systematically discarding is equivalent to biasing. I don't recommend that path unless there are literature sources backing up that approach in situations like yours (whatever that might be).

Are you trying to figure out which factors are most important? Maybe something like stepwise discriminant analysis, where you are looking for the variable(s) that best identify a success or a failure. Note: depending on how your independent variables are correlated your answer may depend a bit on how you proceed. If you use all 15 variables you get one model, but if you remove a variable you can end up with a different model that may be as good or better than the first try. However, I am not sure that I really understand the problem other than you have a large number of zeros in your data.

Timothy, here's the problem in more detail:

The parts produced are classified as 1 (no fault) and 0 (faulty), hence my response variable is either 0 or 1. These parts are produced under different process conditions (factors and covariates). These conditions may be classified as follows:

1. Most of my factors (~10 of all variables) are cardinal variables, such raw material types (0, 1, 2, ..6), direction of a manual processing system (L or R), color of the raw material (0, 1, 2), ...

2. The remaining process variables (~5 of all variables)  are cardinal (continuous) variables, such as mass of the raw material, speed of the rotor, etc. 

I would like to model the response w.r.t. nominal and continuous variables, i.e. how the probability of success changes w.r.t different process conditions. I am using a logit link (though a loglog link seems to fit better, so I may change the link later). To determine which main factors and interaction terms should be included, I am using deviance (or Pearson's chi square) in addition to Wald statistics. Since deviance statistics is not chi-square distributed if expected number of outcomes is less than five, I cannot use continuous variables directly (and my experience has shown me not to trust Hosmer and Lemeshow test too much). Hence I have discretized the continuous variables, such as mass flow rate is divided into four ordinal variables (0,1,2,3), each representing a different mean flow rate. Hence I've ended up with 15 factors, at two to five levels. For each of the combination of these levels&factors, I have determined the number of parts produced, and the non-faulty ones among these produced.

So, here's the problem. I have approximately 7500 observations, but since 15 factors are highly correlated (experimental results are not obtained from a factorial design, but a real process), each cell corresponding to each level of 15 factors does not contain, or contains only a number of parts. For example, at a flow rate of level 0, machine direction left, color 1, etc, there are only 2 observations. If, at the end of the logit model construction, success probability prediction for that cell (combination of factors) comes out to be 0.5, then the expected number of non-faulty parts would be 1. Now, to evaluate the performance the model, I would like to examine the deviance statistics, but the deviance statistics is over all the cells, including those for which expected number of successes is smaller than five. At this point, what should be statistically valid strategy? That's what I'm trying to ask. I can think of two solutions:

1. During modeling, neglect combinations of factors, for which there are only a very small number of observations.

2. Include all observations in the model, but during deviance calculations, neglect combinations of factors, for which there are only a very small number of observations. This one seems to me a better choice, but I'm not sure whether I'm violoting something related with Deviance statistics.

I cannot think of an elegant answer to the problem. I understand the problem better than before, but not perfectly.

You mentioned the Hosmer-Lemeshow test, so you are interested in knowing if the observed failure rates match your expected failure rates. If you can construct a 15-dimentional matrix with expected failure rates, then you could try Mantel's test (Mantel's matrix randomization test). Alternatively, you could use Mantel's test to see if the observed failure rate was significantly different from zero (or any constant). In this latter approach do Mantel's test using two copies of the observed values. The expected outcome is, of course, zero. Cells that are empty are treated as missing data. A good book is "Randomization, Bootstrap and Monte Carlo methods in Biology" by Bryan F. J. Manly (Chapman Hall).

Another approach would be to weight all observations by the number of values. Divide the sample in each cell by 7500, and use this as the weight for that cell.

Thanks Timothy for your contribution.

Weighting the observations is a good idea, I'll try that. I'm not familiar with Mantel's test, and I'll check out that method, too. 

However, these still do not solve my problem of dealing with the deviance statistics. :) Anyway, thanks!

I think you need to reduce your number of factors. If factors are correlated, examine which ones are highly correlated with each other and eliminate one or several of the pack, because correlated factors can REPRESENT each other. Think of this as a group of VECTORS. Try to ORTHOGONIZE them, that is, make a group that cannot be represented by each other. This way you'll be able to reduce your number of cells with too few representatives and your model will work better.

Thank you Gloriam for your answer. Actually, I'm currently working on a nominal version of PCA, so that I can represent the collective nominal variables with a few latent vectors.

I am currently using correspondence analysis (CA), which similar to PCA, but may be applied on nominal variables. After PCA (I mean CA), I will use oblique rotation, as you have suggested, but I believe that there is not going to be a drastic improvement on the interpretation of the results.

On the other hand, I realize that you recommend using oblique rotation INSTEAD OF PCA, but, to be frank, I do not know how to apply it to nominal data.

Paul, thank you for your help!! I definitely will apply this analysis.

Adding to what Paul is saying, look for FACTOR ANALYSIS. Usually used in the social sciences, which have many factors highly correlated with each other, it is a procedure that uses correlations to identify variables that are highly correlated to each other, identifying major factors (the base vectors, so to speak). From engineering it would be equivalent to DIMENSIONAL ANALYSIS. This way you reduce the number of variables and reduce the number of the cells.

Thanks Gloriam.

Actually, it is quite surprising, at least to me, that there are many techniques invented again again (with minor changes) by different branches of science. Being a chemical engineer, all my previous research has been based on continuous (or to be more accurate, cardinal) variables using PCA, ICA, Isomap, Local  Linear Embedding, etc. Hence, at first, I did not think that dimension reduction would be possible for nominal variables. Now I am even more surprised to see that people in marketing, economy, psychology, etc. have already encountered and solved this problem. 

I think internet makes interdisciplinary communication much more easier, faster and efficient nowadays. 

As you might have noticed my bachelor's is in mechanical engineering and my master's in psychology. When I was describing what I had to do for my psych thesis to a fellow engineer (this one computer engineer) which involved measuring how reaction time was affected by induced emotion, his comment was priceless: "But that is ENGINEERING, Vanine." ... And so it was.