Under the above assumption ALL data can be used for training. Testing is useless since the NN obtained is known to correctly classify all the data. Also, if new data are provided that the old NN mismatches, all the data can be put together to obtain a new NN that correctly recognizes all data, old and new. Learning is instantaneous. So, why not ---whenever required--- instantly update the NN to fit all the data. If mistakes are made the NN can immediately be reset so that the mistake will not happen again. Summing up, instant learning implies a new artificial intelligence paradigm.
http://www.matematica.ciens.ucv.ve/dcrespin/Pub/Crespin.zip
Hello Sarge and Joachim.
Thanks for your messages. I will refer simultaneously to both comments. Following Joachim timid initiative I will try to supplement my arguments with symbols. Use will be made of the standard LATEX language for mathematics.
The question of the data being noisy is very important.
First of all, the data MUST be assumed to be noisy. A neural network can deal with noisy data if it RECOGNIZES THE DATA WITH MARGINS. See the precise definition below.
Let the training [or learning] data be a pair of non-empty, finite, disjoint subsets $A, B$ of vectors in Euclidean space, $A\subset \R^n$, $B\subset \R^n$, and assume that the neural network is of the following kind:
Feedforward, classical, discontinuous, $n$-input, binary valued, multilayer (with $k$-layers, $k>0$) perceptron neural network [PNN].
These PNNs in general transform a data $n$-vector, $x\in \R^n$, into a binary value, $P(x)\in \{0,1\}$, and therefore correspond to functions
$P:\R^n \to \{0,1\}$
Any such function $P$ splits the data space $\R^n$ into two disjoint complementary sets, $X_1, X_0$, one being the inverse image of $1$, the other being of $0$. In LATEX symbols
$X_1\cap X_0=\emptyset$ $X_1\cup X_0=\R^n$ $X_1=P^{-1}(1)$ $X_0=P^{-1}(0)$
For any PNN the sets $X_1$ and $X_0$ are polyhedra [a polyhedron is a finite union of convex cells, with each convex cell being a finite intersection of linear half spaces].
A PNN $P$ COGNIZES $A$, if $A$ is contained in the first partition member, equivalently if $P(a)=1$ for all $a\in A$; and $P$ UNCOGNIZES $B$ if $B$ is contained in the second one, or $P(b)=0$ for all $b\in B$:
$A\subset X_1$ $B\subset X_0$
When both conditions are met one says that $P$ (PERFECTLY) RECOGNIZES the pair $A,B$ (of data sets).
The recognition of $A$ and $B$ by $P$ is STABLE if $A$ is contained in the (topological) INTERIOR of $X_1$ and $B$ is contained in the INTERIOR of $X_0$. But metric conditions are more revealing.
Let $r_A$, $r_B$ be numbers, each to be called a MARGIN. By definition $P$ (PERFECTLY) RECOGNIZES $A$ AND $B$ WITH MARGINS $r_A$ AND $r_B$ if any $a\in A$ is at distance at least $r_A$ from the boundary of $X_1$ and any $b\n B$ is at distance at least $r_B$ from the boundary of $X_0$. Note that if this happens then the sum of the margins is a lower bound for the distance from $A$ to $B$
$r_A+r_B < d(a,b)$ for all $a\in A$ and all $b\in B$
Recall that, by traditional definition, the distance $D$ from set $A$ to set $B$ (these being non-empty and finite) is the smallest of the numbers $d(a,b)$. The inequality then says that the distance from $A$ to $B$ is larger than the sum of the margins.
Instantaneous learning can now be stated as the following
Theorem: For data sets $A, B$ in $\R^n$, at distance $D>0$ from each other, and for any pair $r_A, r_B$ of margins such that $r_A+r_B < D$, a 3-layer PNN $P:\R^n \to \{0,1\}$ can be obtained that perfectly recognizes $A,B$ with margins $r_A, r_B$.
The theorem implies that all data vectors $x\in \R^n$ at distance less than $r_A$ from some $a\in A$ are known by $P$:
if $d(x,a)
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence