Suppose a mouse study is being designed which aims to address the question of which dose response shape (e.g. linear, quadratic, exponential, etc) best describes the response to a particular exposure. The different dose response shapes will be compared by Akaike information criterion. How should sample size be estimated for such a study?
I have not found yet an answer to your interesting question, but at first I think that you should consider three aspects of the matter:
1) The size of the sample needed depends on the variability of the variable.
2) It is convenient to have replications for each value of the doses because they enable to do the tests of lack of fit for the differents models considered.
3) When we use AIC we are estimating the lost of information because of using a model different of the actual function that describes the problem, but this estimate is asymtotic (http://en.wikipedia.org/wiki/Akaike_information_criterion) then we should use a sample size as large as we can. Other posibility is to use the method AICc (with correction for small samples).
In the mouse problem, you may use as a first block, a first essay, with two or more mouses for tried dose. If the data lets you select a model using AIC, and you may validate your model using the lack of fit test and studing the residuals then you are ready. If the first block do not lets you to reach a fundamented conclusion about which model is the best, you may repeat the essay as a second block, ... till you could select the best model for the problem.
I have not found yet an answer to your interesting question, but at first I think that you should consider three aspects of the matter:
1) The size of the sample needed depends on the variability of the variable.
2) It is convenient to have replications for each value of the doses because they enable to do the tests of lack of fit for the differents models considered.
3) When we use AIC we are estimating the lost of information because of using a model different of the actual function that describes the problem, but this estimate is asymtotic (http://en.wikipedia.org/wiki/Akaike_information_criterion) then we should use a sample size as large as we can. Other posibility is to use the method AICc (with correction for small samples).
In the mouse problem, you may use as a first block, a first essay, with two or more mouses for tried dose. If the data lets you select a model using AIC, and you may validate your model using the lack of fit test and studing the residuals then you are ready. If the first block do not lets you to reach a fundamented conclusion about which model is the best, you may repeat the essay as a second block, ... till you could select the best model for the problem.
Dear Guillermo and anyone else interested,
Indeed, using AICc is certainly planned, rather than AIC. As a rough approximation, I found statements in the literature that sample size should be >10*K, where K is the number of parameters in the largest model (the most complicated dose response function in this case). But I am looking for a more rigorous method to estimate and justify sample size.
Dear Igor
The sequential strategy that I mentioned above may be a good aproximation while you may conserve the essays in similar conditions.
How many chemicals do you want to look at? In most toxicology studies, the researcher only looks at one chemical at a time. Unfortunately, a One-Factor-at-a-Time (OFAT) approach does not have much power. So, you need to use a lot of replicates/subjects. In the world of designed experiments, we joke that the OFAT method gives you Oh-FAT chance of getting valid results!
The true number of subjects you need to use is based upon the power you want to achieve. The power of a test is essentially the probability that you make the correct decision, i.e. you don't make any type I or type II errors. (No false positives and no false negatives). The power is based upon the variability of your subjects, the alpha value you desire, usually 0.05, the beta you desire. This is probability of a false negative.
If you want to test multiple chemicals at the same time, you can use an Optimal Response Surface. This design will chose the best dose levels for each chemical. Because you are testing multiple chemicals at the same time, you get a lot greater power. This allows you to use fewer subjects. As a bonus, you also get a lot better results, you see interactions between chemicals. So, you end up working smart, not hard. and you get rewarded for it!
Dear Andrew,
The number of factors will not be very large - no more than 3 for this study.
Perhaps you can suggest some literature on the Optimal Response Surface?
In general, however, if I understand correctly, AICc is not a statistical test, but a relative goodness of fit index. So, there is no "power" associated with AICc - the selection between alternative models is based on AICc differences (translated into evidence ratios) and/or probabilities of the given model being selected as the best (Akaike weights). For these reasons I am asking the question of how to estimate the sample size for a study where AICc is intended to be used to select between different dose response models.
As a potential answer to the question, I thought of the following approach to estimate the sample size for a study where AICc is intended to be used to select between different dose response models - would appreciate comments from anyone interested!
1. Monte Carlo simulate the data (say tumor or no tumor in each mouse at each dose level/exposure type, assuming a certain number of mice) multiple (say 1000) times.
2. In each simulated data set, perform model selection by AICc using several plausible dose response shapes (e.g. linear, quadratic, exponential, etc). As a selection criterion, choose some lower threshold for the Akaike weight (e.g. 0.8 or 0.9) - the model with an Akaike weight above this value will be chosen as the best in the set. If no model meets this standard, then the comparison is inconclusive.
3. For each model, calculate the probability of being selected the best (i.e. the number of times this model met the above criterion divided by the number of simulations). Set another selection criterion (say 0.9) for this probability - i.e. if one model has the probability of being selected the best of >0.9 out of all the simulations, then one assumes that the selected sample size (number of mice) was sufficient to select the best dose response shape.
4. Repeat steps 1-3 for several other potential sample sizes (mouse numbers) to find the minimum sample size which satisfies the two selection criteria,
Once again, I would appreciate advice on whether or not this approach makes sense.
Optimal Response Surfaces are fairly well regarded in industry for making proprietary products. There isn't much info on how they used these types of designs. A book that I like a lot is Optimal Design of Experiments: A Case Study Approach By Peter Goos and Brad Jones. Most university libraries have electronic access to the book.
If you only have 3 chemicals to test, you can choose the number of dose levels to use and what those dose levels will be. The software will choose the best combinations of dose levels and chemicals. If you choose to, the design will have 12-15 different dose level/chemical (treatment) combinations. With this design, you can create just about any type of model you want. I would think that replicating each treatment combination 5-10 times would be a good start.
The design itself is so flexible that you can include male and female subjects as variables in the design study. Suppose that you are limited to housing a total say 8 mice at a time. The Optimal Response Surface will break down your design of 150 mice into groups of 8, or just about any other number you can think of. Each group is called a block. This type of design will test combinations of dose levels where each is at its highest levels.
An alternate approach is to use a Mixture Design. For this type of experiment, you will again use 12-15 different treatment combinations. I would use a 4-component design. # chemicals of interest and one benign chemical (water). This design will test different proportions of each chemical. With this design, you will test each chemical at 100% maximum dose by itself, then in different combinations, say 50% water, 50% Chem A, 33% Chem A, B and C. This type of design is a little better suited for tox studies of unknown or poorly known toxicities.
If you are interested, I can create a couple basic designs and post them later on.
Dear Andrew,
Thank you for your answer. I will look up the book. And of course I am interested to look at some sample designs.
Hi Igor,
In the attached designs, I made several different designs and included power calculations for each term in the model. Each design and power calculation assumes the same conditions. The power calculations assume that the Std DEv is 1.00 and the difference between low and high dose levels is 0.50, 1.00 and 2.00. In the excel sheet, they show up as 0.5 Std Dev, 1 Std Dev and 2 Std Dev. I also coded the dose levels from 0 to 100. I assume 0 is a control level and 100 is 100% of the highest dose level you test. I assume that you are only interested in 3 chemicals. I can add in a term(s) for gender of subject. I can also break these designs down into smaller blocks too.
When you get into a logistic regression with larger variability, the same patters will hold true. The power calcs will not. I assume that you have a total of 18 or 19 subjects per design/replicate.
I include the results from 2 different optimal response surfaces. One uses equally spaced dose levels. The other uses a "log' based dose levels.
I also have 2 different types of Mixture Designs. One is a simplex Lattice. The other is Simplex Centroid. With the Mixture Designs, I used 4 components. Chem A to Chem C and Water. This is the benign substance I was talking about.
I also included the OFAT, one chemical at a time method with equally spaced and "log" spaced dose levels. In these designs, you test 6 subjects for each chemical. This constitutes one replication per chemical.
What you will find is that the OFAT method delivers the least amount of information for the most amount of effort. The Mixture Designs test 6-7 dose levels per replication. The Optimal Response Surfaces have the greatest power. Mixture Designs and Optimal Response Surfaces can determine the effects of a few extra terms than I show. These are various 3-way interactions. This includes A*B*C, A^2*B, and so on. The Mixture Designs and Optimal Response Surfaces also contain design points that can act like model adequacy/verification test points.
Very useful article:
Article How to calculate sample size in animal studies?
- Badges
- Science topic