Marc Suchard and colleagues have worked with parallel computation for a number of years.  In the following, they give a characterization of when you can use parallel methods and closely related GPU methods.

Understanding GPU programming for statistical computation: Studies in massively parallel massive mixturesMA Suchard, Q Wang, C Chan, J Frelinger, A Cron, M West

Journal of Computational and Graphical Statistics 19 (2), 419-438

In general you cannot use iterative-fitting-optimization methods because they must make multiple passes over all of the data.  Hadoop Map Reduce are also not good for certain search-retrieval algorithms.