You might find this paper interesting: "A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models": Article A systematic review shows no performance benefit of machine ...

I think the best strategy varies from field to field, data to data. When image processing is involved, convolution neural networks makes sense because of the large amount of data and sophisticated algorithms available. In the case of this paper (clinical prediction models), data is much more limited and difficult to come by since you're limited to available clinical patient data, and extracting genomic data (for example) is not cheap . It's also very difficult to generate/prove negative examples due to the complexity of biology and how much unknown there is to it. So different approaches are more suitable to certain fields.

Hi, Peter! I am currently working on what I believe to be a unique approach to machine classification. Although based on a neural network model, it uses a different paradigm. You can explore the current version, programmed in python, here: http://holder66.dyndns.org:8080/nn. I am busy recoding this in Elixir, and I plan to take it open source: https://github.com/holder66/ham_nn. The readme on github has some links to additional information, including test results, with this paradigm.

I'm looking for collaborators, if you're interested!

Peter,

To properly address your question it might be better to narrow it down by domain such as machine vision, speech processing, language processing, etc. Basically machine learning is pattern recognition with the current specific focus on generalization.

Lets take for example speech recognition, pure neural network models have not been able to outperform(as far as I know) mixed NN-GMM-HMM models and up to a few years ago it was the kingdom of HMM's. Once the deep learning euphoria dissipates, it is my expectation, that we will see more HMM research again.

In machine vision there are numerous alternatives to NNs and most techniques as in speech recognition are used in mixed mode NN's+'choose your image proc. technique as pre-processing'.

In NLP, it has been my experience that for the tasks that I do NN's simply do not work because of my dependence on graphs. In this area where researchers are swearing on NN models, I have found them lacking because of how other researchers have framed the problem. Again, as my post The danger in stating which AI method will be the leader in next years is to think that every problem is a nail and we just have a hammer.'[1], the problem is not only that but given that we just think in terms of that hammer we reshape and limit the problem so that it becomes the nail.

In all the fields I think that research can be done without NN's it is just the deep learning euphoria that has taken over.

[1]https://www.researchgate.net/post/which_artificial_intelligence_method_do_you_think_will_be_the_leader_in_next_years

I'm working to generalize the isotonic regression, for any kind of learning situation, isotonic regression have the advantage to underfitt the model.

Deep Learning approaches are the best when you can sample as many times you need in the distribution of data. It corresponds to lot of interesting applications in image processing, sound processing, text processing, video games, Go, Chess...

However, there are lot of problems where it is not possible to sample as many times you need in the distribution of data: cognitive radio, optimization of energy ressources in IoT, SON management, optimization of marketing campaigns... Here we need sample efficient algorithms that interact with a dynamical environment. You take a look to bandit algorithms for instance.