Yo may wish to look at my long answer here

https://www.researchgate.net/post/What_is_the_practical_rationale_of_Bayesian_statsitics_to_improve_estimation

where I discuss how Bayes based MCMC estimation can tackle models that are difficult for max likelihood. 

Ideally Bayesian analysis should get the same results but the two search processes are quite different and the tree criteria are different.

Maximum likelihood analysis searches specifically for the globally optimal (highest likelihood) tree estimate. Bayesian analysis yields a consensus tree of all the sampled trees after the burnin period. Usually (if the priors are well chosen) the set of Bayes trees will include the ML tree but not always.

@Paul Chiou - Thank you for your detailed response giving an example with the binomial distribution. In essence, it is as if we need to specify a prior, i.e. if there is some prior information known about the parameter. If that information is vague, we might get the same or different results from ML, depending on the choice of prior.

@Kelvyn Jones - Thank you for your explanation given in the linked question. Under the Bayesian paradigm, the parameters seem to be treated as random variables, whereas in ML they are treated as fixed. I suppose then that with the Bayesian method, we are trying to express the uncertainty in the parameters? Also, we can use prior information about the parameter to our advantage by specifying an informative prior. I wonder, does this somehow reduce the uncertainty in the parameter?

@Patrice Showers Corneli - Thank you for giving an interesting perspective using Bayes trees. It seems that Bayesian will use sampling to come up with one consensus tree, while ML will simply pick the tree with the highest likelihood, without doing any sampling steps.

@Rohit P Ojha - Thank you for your explanation to add on to the answer from Paul. Indeed, it seems that ML uses some Bayesian concepts, whether knowingly or unknowingly. Certainly, even a non-informative prior does contain some little information, which may affect the posterior summary measures. Essentially, the choice of prior seems to be one of the main things of interest.

In general, what would be the conclusion if the Bayesian approach yields a different conclusion from the ML approach? Which of the two conclusions should we accept?

"Also, we can use prior information about the parameter to our advantage by specifying an informative prior. I wonder, does this somehow reduce the uncertainty in the parameter?"

I find it rather strange that in general people are still hooked up on inference when I see that Bayesian - inspired MCMC analysis brings improved estimation that allows improved inference - they are often fighting old battles and not seeing what can be done now - that is my opinion.

Collinearity between predictor variables brings increased un-certainty- specifying an informative prior can help in the estimation and reduce uncertainty

see

https://www.researchgate.net/publication/258073418_Bayesian_Informative_Priors_with_Yang_and_Land%27s_Hierarchical_Age-Period-Cohort_model

Using priors can be regarded as a sensitivity analysis - to what extent are the results changed by plausible changes in the prior distribution.   

Article Bayesian Informative Priors with Yang and Land’s Hierarchica...

if you have infinite (or large number) of data points you can asymptotically get similar estimations   

Before you do any Bayesian Analysis be sure to read cautions about intelligently choosing priors and the consequences of not doing so in Zheng Yang's marvelous book Molecular Evolution: A Statistical Approach 2014. There also read about the myth of uninformed priors. At the very lest read the attached.

I think that, with a large enough sample size and assuming a non-informative prior, the posterior pdf would be Gaussian in shape, centered around the MLE. This implies that the posterior mean would be approximately equal to the maximum likelihood estimate +/- 1sigma (the corresponding spread).

Thank you, Erwin Alhassan , for your informative response! I think this is a valid point.

Would you mind kindly sharing any references that can help explain this further?

It depends on the nature of the parameter being estimated. Regression intercepts and slopes (i.e. means) might behave like that, but this may not be the case for a variance or a correlation coefficient. Thus for a variance the estimates are bounded by zero and can be positively skew, giving rise to asymmetric Bayesian Credible intervals that can be derived from a non -Gaussian posterior. See the simple examples in the Bayesian chapter of this

Book Developing multilevel models for analysing contextuality, he...

Thanks Kelvyn for the book, it looks like I already had it in my repository :).

My opinion on the case of an asymmetric distribution is that, MAP (maximum a priori) estimate (which is the mode of the posterior distribution), should be preferred i.e. if one is just interested in a point estimate. Incidentally, MLE is equivalent to MAP in the case of a uniform prior of the parameters.

In the link below, the authors concludes that, for a Weibull distribution (which is asymmetric), the mode (MAP) produced a less biased estimate of the parameters compared to the mean and the median. And maybe, this is just important for model calibration purposes.

Article Improving maximum likelihood estimation with prior probabili...