Comparisons of stats packages often quickly degenerate into arguments consisting of little more than taunts such as, "My package is better than your package!".  This is fueled by the fact that quite often, there are very few people who are sufficiently well-versed in all of the packages being compared to provide fair and balanced commentary.  I think that the folks at UCLA's Computing Services have been a nice exception to that general rule:  People like Michael Mitchell actually do have a fair bit of experience with all of the packages being compared (first link below). 

But as Ariel noted in his reply, I don't think the comparisons linked below necessarily address your specific question about run-time adequately. (And of course, they are becoming a little bit dated now.)

HTH.

http://www2.jura.uni-hamburg.de/instkrim/kriminologie/Mitarbeiter/Enzmann/Lehre/StatIIKrim/Mitchell_2007.pdf

http://www.jstor.org/stable/3600259?seq=1#page_scan_tab_contents

Hi Matt, I am a Stata user for exactly the same reasons you list here (in addition to the fact that there is an avid user-contributed program community that is enormous and approachable). I regularly analyze datasets that have several million rows of data (typically health plan data), and I never run into problems. Certainly any program will grind forever with such large datasets, but I don't find Stata to be more limiting than other programs.

As for comparisons, I remember seeing recently a comparison of results generated between the various programs using specific commands, but that is not the same as comparison of run-time. I'll look around for something.

On another note, I asked some colleagues about your other question pertaining to nomenclature for public assistance programs, and they didn't have anything more to add than what we posted...

Comparisons of stats packages often quickly degenerate into arguments consisting of little more than taunts such as, "My package is better than your package!".  This is fueled by the fact that quite often, there are very few people who are sufficiently well-versed in all of the packages being compared to provide fair and balanced commentary.  I think that the folks at UCLA's Computing Services have been a nice exception to that general rule:  People like Michael Mitchell actually do have a fair bit of experience with all of the packages being compared (first link below). 

But as Ariel noted in his reply, I don't think the comparisons linked below necessarily address your specific question about run-time adequately. (And of course, they are becoming a little bit dated now.)

HTH.

http://www2.jura.uni-hamburg.de/instkrim/kriminologie/Mitarbeiter/Enzmann/Lehre/StatIIKrim/Mitchell_2007.pdf

http://www.jstor.org/stable/3600259?seq=1#page_scan_tab_contents

Ariel, thanks for the feedback (and for asking around about the public assistance question...another study to do someday :) ). Glad to hear you don't run into problems with Stata with large files. 

Building on your opening sentence, Bruce, I've found the same, which sent me looking for hard benchmarks. Another thing that happens is that you get advice from people that aren't up to date with the software or have had one two bad/limiting experiences (I fall into that category with SPSS at this point, since I haven't touched it for 5 years as a result of bad/limiting experiences). Mitchell's a great reference and I have 4 of his books on my shelf right now (and use the ATS/IDRE stats consulting at UCLA, his former employer, weekly...they had no benchmarks on-hand). Brady West is another author who tends to compare programs. His Applied Linear Mixed Models book has code and output for multiple programs, and his Applied Survey Data Analysis book and Websiite (with Heeringa and Berglund) has the same approach. 

Thanks for the additional links!

Bruce is right in that the providers of the benchmarks should be familiar enough with the packages compared to provide efficient ways of handling problems (e.g., vectorized vs. for-cycle processing in R). Some of the comparisons I've seen looked rather funny to me: "Look at how these 5 lines of this Stata code translate to nice and easily readable 20 lines of R code" -- as far as I remember, it had to do with -bysort- in Stata vs. some apply() functions in R.

Back to back SAS to Stata comparisons that I've seen (I am coming from Stata side):

1. Billy Buchanan's conversion of SAS large scale tabulation project into Stata: https://ideas.repec.org/p/boc/scon14/6.html -- the workflow efficiency gains seem to be immense, instead of a spaghetti of ~100 SAS scripts he has, basically, 5 Stata programs, and instead of 1 week to run the whole thing, he runs it in half a day. (This probably speaks more about how good programmer he is, and how poor the original product was, although I am not particularly surprised: my projects producing a hundred or so survey tabs rarely run more than 20 minutes or so, and you have to work with something complicated like model selection over mixed models to have Stata running for a week).

2. Lesa Hoffman provided multilevel examples in SAS and Stata (which she calls STATA) in her book http://www.pilesofvariance.com/index.html and in training materials: http://www.lesahoffman.com/Workshops/ICPSR.html. This is the ONLY source that I am aware that really puts the multilevel/mixed models back-to-back for SAS and Stata, although to get identical results, you have to downgrade from the more computationally efficient defaults in either package.

HTH.

Great response, Stas! I saw your email before I saw this, which explains why I thought you were psychic :-). Thanks for the leads.

More to come as I attempt to transpose my 5M records file (w/ 20 some vars and up to 120 records within my grouping id). Not necessarily with any practical goal in mind, but it's easy to run. Any predictions?

My prediction is that it won't work: http://www.stata.com/help.cgi?limits. You can't have more than 32k variables. You can of  course take that to Mata and then transposed it there, as Mata does not care as much about the dimension of the matrices:

mata : st_view(data=.,.,.)

mata : data'

:)