The main reasons for discrepancies are that the different packages might use slightly different ways of handling things like unbalanced data. By and large, the mainstream packages do provide consistent answers, but it is up to the user to understand the nuances and methods used. The real problems occur when untrained users simply press buttons without a good understanding of what they are doing.  

This is very interesting question. Even I observed when you do some econometric tests ( with same data set) in different packages, you will get different results. To my surprise very less attention payed in academic circles. 

The main reasons for discrepancies are that the different packages might use slightly different ways of handling things like unbalanced data. By and large, the mainstream packages do provide consistent answers, but it is up to the user to understand the nuances and methods used. The real problems occur when untrained users simply press buttons without a good understanding of what they are doing.  

We have done again this discussion, so I copy:

I do not know any top journal results about comparing R & SPSS, but I know that R has better accuracy in calculating simple mathematical functions like sin(x).Look at a comparison of packages for computing the sin(x):

(1)SPSS:

Unable to compute sin(x) when x>3*10^15. see the log below:

===============================================

COMPUTE sinx=SIN(x).

EXECUTE.

>Warning # 625

>The absolute value of the argument to the SIN function is too large. The

>largest permissible value is about 3 * 10**15. The result has been set to the

>system-missing value.

>Command line: 24 Current case: 10 Current splitfile group: 1

...

1,00E+001 -,5440211108893698

1,00E+002 -,5063656411097588

1,00E+003 ,8268795405320025

1,00E+004 -,3056143888882522

1,00E+005 ,0357487979720165

1,00E+006 -,3499935021712929

1,00E+007 ,4205477931907825

1,00E+008 ,9316390271097260

1,00E+009 ,5458434494486996

1,00E+010

1,00E+011

1,00E+012

[It is accurate until x=10^9.]

===============================================

(2)R:

===============================================

It computes everything but it looses accuracy after x>10^16:

x sin(x)

1,00E+011 .9286936605443354419975

1,00E+012 -.6112387013579698713528

1,00E+013 -.2888852824922840678568

1,00E+014 -.2094084333834997091461

1,00E+015 .8582721324763733505847

1,00E+016 .7796799451610669784429

1,00E+017 -.4646441089357642995061

1,00E+018 -.9928161040530034675555

1,00E+019 -.8556574595436564623085

1,00E+020 -.7469218912594929316029

> sin(10^15)

[1] 0.8582721

(the true value is 0.8582728)

> sin(10^16)

[1] 0.7796799

(the true value is 0.779688)

> sin(10^18)

[1] -0.9928161

(the true value is -0.9929693)

=======================

So, it depends on your data, your specific research field and desired accuracy.

Statistical software packages will get different point values for a given statistic. These differences can be from roundoff, truncation, or the algorithm. The algorithm may specify single or double precision and may differ in the calculation method. Nevertheless, the differences are usually very small. In general, statistical point-value results have no meaning beyond the second decimal place (if even that far). There is no reason to require precision beyond the the third or fourth decimal place. The US NIST tests software out to the tenth decimal place.

Major result differences between packages should be examined to determine if different definitions or formulae are the cause. It is important to ascertain from the package documentation what is meant by the particular statistic.

Major package errors should be reported to the vendor and in the literature.

Regulation seems unnecessary. This is a case of buyer beware.

Kindly read the following paper

Keelinga, B.K., Pavurb R.J., 2007. A comparative study of the reliability of nine statistical software packages. Computational Statistics & Data Analysis 51, 3811 – 3831.

Results revealed that there are methods by which you can see the performance of the software. Beautifully the author compared SAS SPSS STATA R Excel and some other software and showed that SAS and R are good for non-linear regression whereas not showing good performance for ANOVA. 

My concern is that if a regulatory body exists then they can say to software developer that you have to remove particular method which is existing in the software  with low performance... 

As Joseph points out, the algorithms the software use are different. If you load the same software package on different computers, you should get different results too. These results should be at a level that beyond being significant.

Suppose that you take a data set with a mean of 1.00 and a Std Dev of 0.10. The software packages should give you a mean of 1.00 +/- 0.0001 or better.

If in doubt, use SAS. SAS is the standard for data analysis and in many cases the only software one can use for different types of data reporting.(at least in the USA) STATA and SPSS would be strong #2 contenders.  R is written by volunteers. So some of the packages will only work/work properly on computers with certain software packages already installed. 

One of my profs insisted that we use only Statigraphics and only access it through the DOS window. Otherwise the results we got would be "off".

To my opinion, 

Statistical Computations are first converted to proper algorithms with a view to accuracy of digital computers. There are no separate statistical methods for same problem. Hence there can not be two answers of same method. The conversion of efficient algorithm is a matter of skill and challenge to Statistician. 

To interested reader I suggest "Statistical Computing" by Kennedy and Gentle

This is an important issue, and shows the need to fully understand the complexities of today's statistical analyses. However, with increasing complexity the reliance of the average researcher on statistical tools such as SPSS and R increases. 

Yet I wouldn't call for a regulatory body to intervene directly, although official tests may be a good idea, and can be used for recommendations. Note though that these tests cannot capture all potential issues that could potentially come up, hence providing a false sense of security. I think we as researchers need to get used to the idea that there could be flaws in any kind of analysis, and if in doubt double check our results, e.g. with other software.

Also, in particular with regard to R incomplete packages may provide incentive to fix it once bug reports have been written. Indeed, the R version tested by Keelinga and Parvurb has already been out of date by the time of publication, and R's reliability has improved by then, too [1]. It would be interesting to see the results of the latest R-version.

Packages are written by scientists, for scientists. For some kinds of analyses (e.g. GLMMs), the code is cutting edge and reflects the current state of art in statistics. The likelihood of finding bugs or imprecise analyses is comparatively high. Most people using these tests should be aware of this. 

Finally, the advantage of open source software is that potentially issues with reliability can be traced back to code and the version. Due to access to source code, old versions in repositories and public change logs it is potentially much easier to find out which papers have been affected if any kind of reliability issues are found. 

Concluding, I don't think a central authority would be a good idea, particularly since it's not clear who would run it (Scientists? Trade office?) and how bad software would be sanctioned. (What about free software? What if it's outside the jurisdiction of such agency? etc)

Instead I think that scientists should pay more attention to these issues, and there should be more reviews of statistics packages. Instead of appealing to authority, we should raise awareness and provide information. This should be part of a general introduction to statistics, in line with many other common problems of statistical testing, e.g. multiple testing, incomplete analyses reported in papers, significance fishing, etc. 

[1] http://r.789695.n4.nabble.com/software-comparison-td822867.html

You raised a very good point but I believe that most of these software packages perform your task for you with minor numerical or decimal difference. R is free and there online resources to guide you through most computational methods. If you have access to SPSS, SAS or STATA, just stick to one.

The results obtained from these software differ slightly (not significantly - due to the algorithm used for calculation). This doesn't mean that they are not reliable.

I would give your question a definite: it depends.

Algorithms for many statistical tests are under debate (especially when it comes to calculating confidence intervals) an thus I prefer R a lot to commercial software such as SPSS, because you can always check yourself which algorithms underlie the calculations plus often (as Jospeh mentioned) different packages are available and hence one may judge by oneself whether there are any differences between results and which results to trust most.

However, I would worry less about the reliability of the programs and far more about the reliability of the one using it (absolutely including myself), because a singular person is way less likely to detect his/her (unintentional) faults than a community or team of people.

Since the early development of statistical software, this topic has been recurrently addressed  by several user communities (biometrics, econometrics, chemometrics, spatial analysis, stochastic modelling in finance and insurance,  etc .), especially with the broader use of simulations and non-linear models, where random number generators and optimization techniques are involved.

For a recent introduction to these issues, you can read the following reference

Odeh, Featherstone, and Bergtold (2010) Reliability of statistical software, Amer.J.Agr. Econ., 92(5): 1472-1489

which is focusing mainly on the packages used by econometricians, but it can give you at least some useful information about  two of the packages you mentionned (namely, R and SAS):

Some years ago, McCullough, a well-known econometrician, has provided a methodological framework in:

McCullough, B. D. (1998), Assessing the Reliability of Statistical Software: Part I,The American Statistician, 52, 358-366.

and some conclusions about SAS, SPSS and S-plus :

McCullough, B. D. (1999) Assessing the Reliability of Statistical Software: Part II, The American Statistician, 53, n°2, 149-159

and, eventually, on more specialized econometric softwares in:

McCullough, B. D. (1999) "Econometric Software Reliability: EViews, LIMDEP, SHAZAM and TSP", Journal of Applied Econometrics, Vol. 14, No. 2, 1999, pp. 191-202.

Best regards

The some known analysis u can performed on different statistical software SAS, SPSS etc and u feel very easily that  which softawere is good  for which aspect's  analysis.  

Thank you Dominique! I have read quite a few articles of McCullough. In series of the same kind of research I found one very interesting paper (Keelinga, B.K., Pavurb R.J., 2007. A comparative study of the reliability of nine statistical software packages. Computational Statistics & Data Analysis 51, 3811 – 3831.) 

It revealed that if one method is correct in one software, it is not correct in others like MSExcel is calculating negative Sum of Squares for ANOVA but you will get the p-value???

I have to say that if software is reliable its all methods should be validated and if any of the method is not correct it should not be included in that particular software. 

Dear Gayatri,

You're welcome.

I think a possible explanation could be found in the following Mc Cullough's paper :

Microsoft Excel's 'Not The Wichmann-Hill' random number generators

Computational Statistics & Data Analysis

Volume 52 Issue 10, June, 2008

Pages 4587-4593

doi>10.1016/j.csda.2008.03.006

quoting the abstract :

"Microsoft attempted to implement the Wichmann-Hill RNG in Excel 2003 and failed; it did not just produce numbers between zero and unity, it would also produce negative numbers. Microsoft issued a patch that allegedly fixed the problem so that the patched Excel 2003 and Excel 2007 now implement the Wichmann-Hill RNG, as least according to Microsoft. We show that whatever RNG it is that Microsoft has implemented in these versions of Excel, it is not the Wichmann-Hill RNG. Microsoft has now failed twice to implement the dozen lines of code that define the Wichmann-Hill RNG. "

But I am not an expert in this field.

Best regards

I use the these software a lot, R, SPSS, MINITAB, etc. they are all fine and beautiful, but you have to be very careful in using each of them. Some of them are developed for a particular kind of analyses, you should be able to identify the one best in analysing categorical data and continuous data. what of importing or entering of data. some of them use only codes and commands. Just try and understand how to use few of them very well. They all give the same results, except in few cases where you have negligible differences in values after decimal points.

There are negligible differences between statistical software because they use different methods to calculate the same thing (depending on what are you looking for). The difference between them is negligible!

I have a lot of problem importing dirty data from excel files, and R was the only one capable to import such file in a meaningful way. I strongly recommend R, because it is free, it has a lot of capabilities, interact nicely with the underlying operative system (you can play in the command line/Terminal and do whatever you like, as far as you are authorized to do so) and even uses the same default methods used by SAS or STATA if it is order to do so (without paying a penny, a pence or a rupee).

Nobody speaks about EpiInfo, but that is the only software capable by default to create forms and run from a USB stick. It is made by the US government (CDC specifically) and like R, it is also free.

Check out these data sets. http://www.itl.nist.gov/div898/strd/general/dataarchive.html

This is the link to the Data Sets Joseph was talking about. Try these out in your software and see which ones work best.

If a statistical software is reliable or not, is a also depending on what you mean with reliable.  As been pointed out by others the algorithms is the critical part, but also the program in combination with the computer itself can be a problem.  An illustrating example is to try the following example using both a statistical software and using Excel:

First let the software calculate the ratio 1.4/3.5

Then let the computer check if the result is lower than, equal or greater than  0.4 typed from the keybord,

The theoretical result is of course equal, and you are likely to get different result between your statistical program and Excel, but can you from this state that one is mor reliable?