This is a very nice question, and the answer depends on how you define "big data", since there is no perfect agreement on this (it is more of a "buzzword" if you want).

If you define it as "data that cannot be processed by a single computer", then by definition learning from big data requires a distributed solution. In this context you have a lot of research on distributed training algorithms (e.g. distributed SVM [1]) or distributed execution frameworks (e.g. MapReduce, GraphLab, etc. [2-3]).

However, big data can be defined in a huge variety of ways, e.g., you can define it as a set of data that cannot, under any circumstance, be loaded all at once in the memory. In this case, you can face several problems which do not concern anything distributed.

For example, online learning algorithms overcome the limitation by processing a handful of examples at a time, and here the research is on how to efficiently handle large amounts of data without worsening performance. For instance, in online kernel-learning there is the problem of the linear growth of the model [4]. Another example is active learning, where you want to select the most useful patterns from a huge database of patterns that cannot be processed all together. Or you can have "randomized" approximations to make the optimization problem simpler and possible to solve even in the presence of big data [5].

So as you see, "distributed ML" and "big data ML" are related but actually distinct concepts.

[1] Navia-Vázquez, Angel, et al. "Distributed support vector machines." Neural Networks, IEEE Transactions on 17.4 (2006): 1091-1097.

[2] Chu, Cheng-Tao, et al. "Map-reduce for machine learning on multicore." NIPS. Vol. 6. 2006.

[3] Low, Yucheng, et al. "Graphlab: A new framework for parallel machine learning." arXiv preprint arXiv:1006.4990 (2010).

[4] Singh, Abhishek, Narendra Ahuja, and Pierre Moulin. "Online learning with kernels: Overcoming the growing sum problem." Machine Learning for Signal Processing (MLSP), 2012 IEEE International Workshop on. IEEE, 2012.

[5] Rahimi, Ali, and Benjamin Recht. "Random Features for Large-Scale Kernel Machines." NIPS. Vol. 3. No. 4. 2007.

This is a very nice question, and the answer depends on how you define "big data", since there is no perfect agreement on this (it is more of a "buzzword" if you want).

If you define it as "data that cannot be processed by a single computer", then by definition learning from big data requires a distributed solution. In this context you have a lot of research on distributed training algorithms (e.g. distributed SVM [1]) or distributed execution frameworks (e.g. MapReduce, GraphLab, etc. [2-3]).

However, big data can be defined in a huge variety of ways, e.g., you can define it as a set of data that cannot, under any circumstance, be loaded all at once in the memory. In this case, you can face several problems which do not concern anything distributed.

For example, online learning algorithms overcome the limitation by processing a handful of examples at a time, and here the research is on how to efficiently handle large amounts of data without worsening performance. For instance, in online kernel-learning there is the problem of the linear growth of the model [4]. Another example is active learning, where you want to select the most useful patterns from a huge database of patterns that cannot be processed all together. Or you can have "randomized" approximations to make the optimization problem simpler and possible to solve even in the presence of big data [5].

So as you see, "distributed ML" and "big data ML" are related but actually distinct concepts.

[1] Navia-Vázquez, Angel, et al. "Distributed support vector machines." Neural Networks, IEEE Transactions on 17.4 (2006): 1091-1097.

[2] Chu, Cheng-Tao, et al. "Map-reduce for machine learning on multicore." NIPS. Vol. 6. 2006.

[3] Low, Yucheng, et al. "Graphlab: A new framework for parallel machine learning." arXiv preprint arXiv:1006.4990 (2010).

[4] Singh, Abhishek, Narendra Ahuja, and Pierre Moulin. "Online learning with kernels: Overcoming the growing sum problem." Machine Learning for Signal Processing (MLSP), 2012 IEEE International Workshop on. IEEE, 2012.

[5] Rahimi, Ali, and Benjamin Recht. "Random Features for Large-Scale Kernel Machines." NIPS. Vol. 3. No. 4. 2007.

Distributed Machine Learning can be used in big data or in any distributed system not specific for big data

Ba eradat be hamvatane aziz.

For large datasets, distributed computing will become essential, due to both speed and memory constraints.

Latouche, Pierre. "Distributed Machine Learning." (2007).

I also agree with Symone.