Hi Jari

I would argue that the reason is a mix of trends and manageability. Walsh is a good example you use. 300 methods may have seemed impressive back in the early 60s -  but researchers are probably more realistic nowadays  - and higher degree research students will often be looking for the most manageable options. In many cases, that will equate to the most used and tested methods.

Jari -

I think that even though John Tukey noted “The best thing about being a statistician is that you get to play in everyone else’s backyard," statisticians are often not good at making our work's relevance known to our 'hosts.'  Further, our hosts are often not utmost in our minds.  I have evaluated dissertations, theses, and papers where many "new" estimators are produced, and it is obvious that this was only done to satisfy a requirement for "new"/original work, with no intention of anyone using it for anything.  A connection to application often seems not in the authors minds at all. 

You note many methods that might be helpful, of which users are generally unaware.  Again, I'd say that is due to a weak relationship between those interested in theory and those interested in practice.  In my career, I'd  always found that it was most interesting to problem solve the real problems, using and developing methodology to that end.  Some expressed greatful surprise at that.  In most of my career, however, there were managers in charge of a statistical agency who did not trust statistics, and did not want to listen.

Many have trusted p-values to be useful as a stand-alone statistic.  I warned about that decades ago.   "Significance" is a misnomer.  But statisticians let the scientific community down by not edifying researchers on the dangers if its use, while they (statisticians) published many papers on statistical properties of "tests" that were of little practical importance. I'm all for developing theory, but there should be attention to application and to good communication with those in whose 'backyard' we can play.  "Significance" represents a huge failure in this regard. Yet we even have a magazine (in which I have published a letter), with that name: Significance!) 

I worked a great deal on official statistics, concentrating on total survey error, and practical application in a resource restricted and messy world, which meant using and developing methodology not used in many more traditional applications.  Results tested and performed extremely well in many thousands of cases, for decades of US Official Statistics, yet many academicians would rather suggest doing something more traditional, based on their nearly zero experience. 

In the 1990s, a well-known statistician, in discussing a seminar I gave, suggested that what I was promoting should be a method to be found in every statistician's "toolbox."  (Yes, he used the same word as found in this question.)  It was with regard to the coefficient of heteroscedasticity, for weighted least squares regression.  It is still not in enough such toolboxes. 

Any statistician should warn that anecdotal evidence can be misleading.  But in my experience, which includes often paying attention to other people's experience, and the history of statistics, it seems likely that your question deals with part of the disconnect between academic statisticians who are not communicating well with non-statistician researchers, and users who are not comfortable with statistics.  A partnership in problem solving between these groups is lacking.  In the case of the methods you mention, more attention to applying them is needed, but that requires a willingness to increase meaningful communication. 

People aren't like that, unfortunately.  Brilliant things and silly things might be mixed together.  Sometimes, I think,  an otherwise great method is based on what might be a silly premise for some applications, such as building on OLS, when WLS is obviously needed, for example. 

So, there is an important niche that a statistical company could exploit: practical problem solving, while communicating with the client and not either getting lost going off on an academic tangent, nor doing something silly, just because your client told you too.  But there, money gets in the way.   As a govt statistician, management ended up in my way, and contractors did whatever silly thing they were paid to do, even when they likely knew better.  Now, in the US, truth seems a huge casualty of an authoritarian-minded executive branch, so we cannot expect improvement with regard to official statistics.   In the past, they have been fairly good, though they could have been better (more accurate and better presented and understood), and at least, I think, they were basically honest.  But now it seems that those useful data may be surpressed when inconvenient.  Let's hope that can be prevented.  Many members of Congress have tried to do that before.  They have more chance of doing more of that now.

With regard to the methods you mention, what can I say?  It's a messy world. 

Best wishes - Jim. 

Dear Jari

Good question indeed!

Part of the answer has been covered by the Prof Knaub and Dean. True, it is due to theoretical perspective in Statistics (based on only mathematical formulation; if this is the case what will be the solution); second lack of interaction between users and statistician. The lack of interaction is due to variety of reasons including non-interaction; apathy and neglect, isolation. 

To me, the method should be simple and must reflect the truth, which a real researcher can sense or observed. This practice may really work most of the time. 

Regards

Hi all,

Good responses so far - but one fundamental flaw. You all assume that method is quantitative and statistically-driven. Have you considered qualitative and/or mixed methods?

Personally, I did not exclude the qualitative toolbox. However, it seems that the development in quantitative statistics or precedures is somewhat five times greater/wider/faster than in the area qualitative research and hence, startistical/quantitative terminology may dominate the discussion. Why FIVE times? Some time ago I made a systematic analysis of methodological keywords used in the publications in ERIC database with more than 1 Million cases. About 80% of the keywords found came from the area of "quantitative" research (such as "correlation" or "Factor analysis" or "Regression analysis" and round 20% from strictly qualitative research (like "case study", "ethnography" or "content analysis"). The reason is obvious: of the possible journals, huge majority favors quantitative or statistical results. Seems so.

I appended a gray ms. of the matter. kind of white paper. a bit old now.

Hi Jari,

A good rebuttal here. It does also depend on discipline. In one of my fields (nursing and midwifery) - the majority of published studies (although nowhere near as much as around a decade ago) are qualitative.

Discipline is inportant, yes.  I started in nursing in 1980s. At that time it was practically impossible to publish anything statistical in the domain (at least in Finland) - the works were more or less theoretical ones. Late in 1980s I shifted to education. At that time, the researchers with the qualitative methodology needed to fight to get respect. Now, most of the students select qualitative methodology in their master's thesis. It is quite easy to do a qualitative study/report up to the doctoral level. After that, it seems to be more difficult to convince the reviewers.