Here's the article: homes.cs.washington.edu/~pedrod/papers/cacm12.pdf

The algorithms cannot be that "dumb" as well as the annalyst who uses them. Just the first Figure there explains a lot: algorithms can check for the cases where variability is decreasing (whatever ML approach it is) - and here big data help a lot. But they get stuck or become misleading in case of systematic bias (in bioinformatics eg there are plenty of method-related artifacts) - and then one has to use human brain too :)

His section "More data beats a cleverer algorithm" follows the previous section "Feature engineering is the key". In this context he is probably right, but with this priority: the representation "is the most important factor" (his quote) for the success of the project..

I think what he means is that IF you got the 'right' representation and the appropriate amount of data you are OK without the fancy algorithms. However, what he should have stated more explicitly is that if you don't have a 'good' representation---which is almost always the case---no amount of data or clever algorithms will produce satisfactory results.

What he and almost everyone else do not realize is that there are fundamentally new representations which could radically change the usual rules of the game.

I'm quite sure that the PR / ML obsession with better algorithms can be explained by the lack of good representations: if you don't have access to a 'good' data representation and you like statistics, you start inventing "cleverer" algorithms. ;--)

Let's look at it from a different point of view. What if a 'dumb algorithm' and a 'clever algorithm' both end up with the same result. Which one do you prefer? Given the facts that dumb algorithms mostly require less energy and memory resources, I'd go for the dumb one.

Now, what if the outcome of both algorithms is unknown? Then I'd still say that the 'dumber' your algorithm is when it obtains the required result, the more efficient your solution is.

It's no doubt that big data has a much higher chance at covering much relevant infor. Then, recall does matter. However, a ‘dumb algorithm' can help to reduce resources for tracking helpful contents, it focus on higher precision. Big data or smart algorithm, it depends.

We now have tens of thousands of classes and hundreds of millions of instances in our Monk system for retrieval of handwritten word patterns in historical collections. We indeed find that the performance of methods is converging when the number of examples for a class is large. That is long known: cf. Duda & Hart on the error of nearest-neighbour when N approaches infinity: It will not be worse than twice the minimal Bayes error.

Under these conditions efficiency and convenience (no human interference needed) become the major criterion for classifier choice.

Also see: https://www.researchgate.net/publication/230901036_Separability_versus_Prototypicality_in_Handwritten_Word_Retrieval Figures 5 and 7.

Conference Paper Separability versus Prototypicality in Handwritten Word Retrieval

ML can greatly benefit by a little bit of infusion of background knowledge . This can improve quality for a given training size, or allow you to work with smaller training set.

Examples of easily created/available knowledge: (a) knowledge in the forms of community created dictionary, (b) structured information harvested from community created knowledge such as Wikipedia, (c) community maintained high quality structure knowledge such as MusicBrainz that already exist, (d) manually and easily created schema knowledge, (e) use of text/structure mining tools...

Dear Prof. Ryles,

If you have larger data and if you are estimating some parameter by the Law of Larger numbers the expected value of sample error should approach zero as the sample size grows. We do not know whether the converse is true i.e. whether the expected value of the error increases if you do random sampling with growing size of population by keeping the sample size the same. We have proved in Mallick (1993, 2015) based on Radner & Rothschild (1975) that this happens in psitive probabilities. This we have String Theoretically developed into spacextime integrability of Information Systems (Mallick (2014), Mallick, Hamburger, Mallick (2016, 2017)) and provided as a possible solution of the P vs. NP Millenium Mathematics problem (Mallick, Hamburger, Mallick (2016, 2017)) by developing the ECommerce Field using Higgs-Englert-Bosonic particles in a Tychonoff product type Systems Classified and Systems Integrated field using our developed Mallick-Hamburger-Mallick Functor Algebra Calculus (Mallick, Hamburger, Mallick (2016, 2017)) including Genetic Algorithms for solution. Hence, I think the answer to your question is not all that simple. Thanks for posing the question.

Soumitra K. Mallick

for Soumitra K. Mallick, Nick Hamburger, Sandipan Mallick