Paul,

the approaches are in the philosophical sense coming from different communities.

Best regards,

Volker

Dear Paul,

Thanks a lot for your statements. To keep it clear, my question was not intended to compare russian vs non-russian science in a political or whatever sense. My intention is to take again a deeper look in these papers because Aizerman's research was one of the basic inputs to Vapnik's research which ends, so to speak, with empirical risk functions, SVMs, transductive learning, etc. Another community which used Aizerman's approach is a part of the Fuzzy Set Theory community. The researchers who have worked and work on Fuzzy-Pattern-Classifiers. 

Therefore, I assume it's wothwise to take a deeper look in the 'old' papers.

Best regards,

Volker

This is a good question, especially with the clarification.   There are many possible answers.

First, there is Mercer's theorem (1909): If a problem is nonlinear, one can map the problem from the input space to a new higher-dimensional space (called the feature space) and then use a linear model in the feature space.    More precisely, Mercer found the following: 

Mercer Theorem. Any continuous, symmetric, positive semi-definite kernel function K(x,y) can be expressed as a dot product in a higher-dimensional space:

J. Mercer, Functions of positive and negative type and their connection with the theory of integral equations, Philos. Trans. of the Royal Society of London, Series A, Containing papers of a mathematical or physical character 209, 1909, 415-466.

Next, the use of Mercer's theorem for interpreting kernels as inner products in a feature space in machine learning comes from Aizerman, Braverman and Rozonoer (1964)--55 years after Mercer introduced his result.    

M.A. Aizerman, E.M. Braverman, R.I. Rozoner, Theoretical foundations of potential function method in pattern recognition learning, Automation and Remote Control 25, 1964, no. 6, 821-837.

For more about this, see

M. Sewell, Kernel methods, University College London, 2009:

http://www.svms.org/kernels/kernel-methods.pdf

For a detailed representation of Aizerman's result in defining a kernel mapping, see pp. 2-3 in

K. Kovacs, Various kernel methods with applications, Ph.D. thesis, University of Szeged, 2008:

http://www.math.u-szeged.hu/phd/dreposit/phdtheses/kovacs-kornel-etezis.pdf

More to the point, kernel functions inspired by Aizerman's basic idea are considered in great detail in terms of solving classification problems in

K. Riesen, Classification and Clustering of Vector Space Embedded Graphs, Ph.D. thesis, Universitat Berlin, 2009:

http://www.iam.unibe.ch/fki/staff/PhD-Thesis.pdf

See Section 5.3, starting on page 104, on Feature Map vs. Kernel Trick.

Dear James,

thanks a lot for your input which completes my view on this topic. We just have established a Workshop on this topic together with our colleagues of Chemnitz University, Germany who "invented" Fuzzy-Pattern-Classification based on Aizerman's functions in the '80th.

Best,

Volker

It looks interesting and indeed quite different from SVMs.

Does anyone manage to get his hands on the original paper:

"Theoretical foundations of the potential function method in pattern recognition learning" ?

I think I found it, but it is in Russian:

http://www.mathnet.ru/links/e4ba3eb5362a24068fccabec42c77302/at11677.pdf

Application of Potential function method/ M.A. Aizerman, E.M. Braverman, R.I. Rozoner /:

Utility function evaluation and preferences representation based on von Neumann axiomatic and Raiffa's lottery approach:

https://www.amazon.com/Decision-Control-Management-Support-Adaptive/dp/1466629673

Thanks Yuri!

I think we should. I think it can be seen as a Hopfield network and the modern form of it 'dense associative memory' which does use a kernel.

Definitely, the book “book was published “Braverman Readings in Machine Learning. Key Ideas from Inception to Current State” has chapter about it