I am asking the community here to get a brief overview if there exist a method or algorithm, that enables me to transform a traditional perceptron based feed forward artificial neural network into its open equation form?
The closed for of an equation for a linear system for example is
y = kx + d and its open form is R = y - kx - d
Any idea or hint is appreciated.
Kind regards and thank you in advance.
Wolfram,
If I have understood you question properly, you need a method to express the predictive power of ANN in terms of equations instead of using ANN without knowing what is going on inside it.
ANNs are generally considered 'black box' and it is not possible to explain the interaction of its inputs in terms of equations. However, there has been methods developed to deal with this problem. For instance, you can have a look at this paper:
Millie, David F., et al. "Modeling microalgal abundance with artificial neural networks: demonstration of a heuristic ‘Grey-Box’to deconvolve and quantify environmental influences." Environmental Modelling & Software 38 (2012): 27-39.
The authors have used a technique to convert an ANN into an equation for obtaining the governing equations in an ecological context.
I hope this helped.
This paper may also be helpful:
Young II, William A., and Gary R. Weckman. "Using a heuristic approach to derive a grey-box model through an artificial neural network knowledge extraction technique." Neural computing and applications 19.3 (2010): 353-366.
Hi Ehsan,
thank you for the feedback. I was not aware of the papers you mentioned. Yes I do know that ANN's are some sort of "black-boxes". About 2 decardes ago I have delevoped an algorithm, which is able todo just this transformation, but i have never published this work, because it was part of a propriatary framework. But I am planning to publish this work in the near future, but I am surprised that not much was happening in that area of research.
Thank you very much for your feedback, I will have a look to your mentioned papers.
The input-output function is fairly straightforward to describe:
y = f( W z + b) where y is the vector of outputs, z the vector of values (= activations) of the hidden units, W is the weight matrix, b a bias vector and f(.) a fixed non-linear function (e.g. a sigmoid).
In turn, and assuming a single hidden layer, z is defined from the input vector x using the same equation form (but not necessarily the same values for W and b, nor f possibly):
z = f(W' x + b')
Plugging z into the first equation gives you the input-output function.
The same reasoning applies if you want to consider more hidden layers.
So, once the ANN is estimated, there is no big mystery in what it is computing (although such a close form is not necessarily straightforward to interpret).
The "black-box" nature of such a model is more likely referring to the optimization problem which needs to be solved to estimate the parameters (W + b values, for each layer). This problem is non-convex, the objective function is highly non-linear and exhibits many local optima. The training algorithm also typically includes many meta-parameters. So, it is difficult to characterize, for a given training dataset, to which parameter values the learning algorithm will converge. Nevertheless, once the learning is done and hence the parameters fixed, the input-output relationship is well known (see above).
Hi Rok, thanks for your feedback. I will look into this papers, that you refered to.
Dear Colin,
thank you for this very interesting comment. Unfortunatly I am not able to participate in that conference but I consider doing so for next year. I will follow your work as it looks very interesting for decision making processes.
Regards, Wolfram
Wolfram,
Yes, you can represent mathematically what is going on inside the black box. A series of equations or a combined equation can be used to represent the neural system. The idea is simply coming from (output= summation of (weight x inputs)+bias).
I did it for one of my works but its not published yet... if u have the model, please send it to me, I can construct the mathematical representation for you. Also, I can built an algorithm for this purpose.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training