Interesting question. ANN can be used for function approximation, you develop your model (equation), and then feed the ANN with data corresponding to observations of instances of the model. The ANN learns the optimal weight for the model from the data feed to the ANN. Know, if you have a model, and you are providing specific weight, the ANN will not need to be trained; will be fixed in advance. IMHO, this is not anymore a machine learning algorithm (does not learn), but a simple classification (or regression) algorithm implemented using a ANN like function.

This answer refers to feed-forward neural networks but similar considerations hold for other kinds of ANNs.

Feed-forward neural networks are organized into groups of units called layers. Most neural network architectures arrange these layers in a chain structure, with each layer being a function of the layer that preceded it. In this structure, the first layer is given by

h1 = g1(W1*x+b1) ; where g1 is the activation function, W1 are the weights and b1 is the bias term

the second layer is given by

h2 = g2(W2*h1+b2)

and so on ... You can easily get the related close form y = f(x) from above equations, e.g. in case you have just 3 layers

y = g3(W3*h2+b3)

hence,

y = g3(W3*g2(W2*g1(W1*x))) + [b3+...]

Notice that typically the activation functions are not linear, hence f(.) is typically not linear. Also, you'll need to find a vectorized implementation of this equation if you want to use it in real datasets.

An interesting related aspect is the representational power of ANNs. The universal approximation theorem (Hornik et al., 1989; Cybenko, 1989) states that a feedforward network with a linear output layer and at least one hidden layer with any “squashing” activation function (such as the logistic sigmoid activation function) can approximate any Borel measurable function from one finite-dimensional space to another with any desired nonzero amount of error, provided that the network is given enough hidden units.

The universal approximation theorem means that regardless of what function we are trying to learn, we know that a large MLP will be able to represent this function. We are not guaranteed, however, that the training algorithm will be able to learn that function. Even if the MLP is able to represent the function, learning can fail for two different reasons. First, the optimization algorithm used for training may not be able to find the value of the parameters that corresponds to the desired function. Second, the training algorithm might choose the wrong function as a result of overfitting.

You can find more details on Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016

Use Perceptron Model...........

You should start with the simplest model neuron. A binary neuron, the state could be either active (S=1) or passive (S=0). State value is solely determined to be whether or not its input exceeds a threshold.

For example a neuron, which determines whether or not the mug is hot and we should remove the hand or not. Let's say 122F (50C) is hot, we will use this value as our threshold. In other words, if the temperature of the mug is more than 122F then remove the hand. The neuron math representation is input is temperature, threshold = 122. So if input < 122 ? 0 : 1.

In the context of neural networks, a perceptron is an artificial neuron. Perceptron - is a binary classifier, that can decide whether an input represented by a vector of numbers, belongs to some specific class or not. We did it in our example.

Here is example with three input variables. In our example, this may be the temperature of mug, outdoor temperature and does person wearing a hand glove. Because given it’s completely independent values, we introduce weights. We know wearing a glove person can hold mug any temperature. According to this glove weight is big. In opposite outside temperature does not give a big difference. Our formula will be:

[mug temperature] * 0.7 + [outside temperature] * 0.6 + [glove] * 1.4

The human brain has about 100 – 1 000 trillion neurons. In opposite artificial neural network has about 1–100 billion. Human brains have ~10 000 computational power than computer brains.

Dear Gafel

The following paper is well described the relation between weights and biases of the ANN:

"Real time prediction of drilling fluid rheological properties using Artificial Neural Networks visible mathematical model (white box)"

Very much interested in this question, Gafel Kareem have you found anymore useful resource? I would be glad if you could share.