My recommendation is not to control for multiplicity at all ... unless your only focus is on p-values and no replication is planned (which in this case stepwise FWE or FDR procedures are probably best). If your focus is on effect sizes, confidence intervals, and other methods of estimating and understanding the effects then multiplicity control becomes unnecessary. The attached paper discusses my perspective on the issue.

Hello Rob.  Interesting paper.  This statement in particular caught my eye: 

• "When the focus is on estimation, rather than statistical significance, multiplicity has no effect on the results or conclusions."

Given that statement, I'm wondering what you think about confidence intervals for multinomial proportions, for example.  Specifically, are methods that produce simultaneous CIs for those proportions preferred over simply computing a binomial CI for each category in turn?   (Of course, the simultaneous CIs will be wider.)  To me, this example seems to be addressing exactly the same multiplicity problem, but in the context of estimation rather than testing.  What do you think? 

Finally, I'd like to add that the two Lancet articles by Schulz & Grimes (links below) are among the clearest and most sensible things I've read on this very tricky problem. 

Cheers,

Bruce

p.s. - It's great to see that someone other than me remembers that great Huberty & Morris (1989) article.  ;-)

http://blogs.sas.com/content/iml/2017/02/15/confidence-intervals-multinomial-proportions.html

http://www.thelancet.com/journals/lancet/article/PIIS0140673605664616/abstract

http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(05)66516-6/abstract

 HI Bruce,

Great to chat with someone from my alma mater. I am not very familiar with simultaneous CIs, however in my limited understanding the issues are very similar to most multiplicity issues (i.e., we want to control an "overall" error rate). If the intervals are used exclusively for NHST, then a Bonferroni (or some other adjustment is necessary). However, if the goal is to try and estimate the parameters as precisely as possible and replication is expected, I don't see a need for multiplicity control. 

BTW ... thanks for the attachments, I was not familiar with the Schultz & Grimes articles.

Rob