How can we verify deep neural networks in terms of reachability, adversarial sample, etc. or from a statistical insight?
Hi Tianle Zhang
For the major understanding of the deep neural network performance, it is always recommended to apply it, first. Then, evaluate it with regards to accuracy, confusion matrix, etc.
NB: at the evaluation stage, the holdout method, for instance, can be used.
HTH.
Dr. Samer Sarsam
Dear Tianle Zhang
I think that the following link can be useful:
https://machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/
Also, in my opinion, the selection of the best DNN configuration depends on your data and specific application. I suggest incorporating the complexity criterion in the optimal structure selection process as well.
Dear Dr. Samer Sarsam
Thanks for your help.
There are lots of statistical criteria to evaluate the performance of deep neural networks w.r.t applications. However, are such models with better indicators absolutely safe and trustworthy? In other words, how can we evaluate and quantify models’ reliability and trustworthiness against adversarial examples?
Samer Sarsam
Sajjad Afrakhteh
Dear Dr. Sajjad Afrakhteh
Thanks for your help.
DNN models with the optimal structure and parameters are able to achieve the best performance, but they may not be more robust. Recent researches have shown DNN models are likely to output a worry result met with adversarial examples in application stage. I would like to know how to evaluate the reliability and trustworthiness of DNN models and quantify their robustness against perturbations.
Dear Tianle Zhang
You are welcome.
We recommend using a K-fold cross-validation approach to assess the overall learning process of NN. For valid assessment, this has to be applied N times for the K cross-validation. Then, statistical tests like paired t-test can be applied to evaluate the significance of the algorithm against other algorithms with regards to accuracy, confusion matrix, and so on.
Cheers,
Dr. Samer Sarsam
Hi Tianle Zhang
To mention a paradox and make visualisation easier when training a Neural Network, you can imagine a multi-dimensional environment with unknown variante, layer interacting in parallel simultaneously and your best approach is to use a model that you will have pre-trained in feedforwarding network for supervised learning. You can also think about the cloud architecture when referring to a Neural Network using multiple instances like a private cloud and a controlled environment withing those you will have countless of "Shards" and "cluster". Using backpropagation or (BP) you can calculate the gradient of the Loss function, gradient descent or variants such as stochastic gradient descent is widely used but could be much better.
With Artificial Intelligence, a well trained Neural Network, should be trained before, during and after, while gathering results, repeating steps and measure accuracy reach between your compiled plotted data.
There is an increasing number of researchers working in the problem of formal verification of Machine Learning algorithms and, specifically, of Neural Network models. As far as I understand it is still a very incipient research topic.
Some research projects that I am aware of and you might be interested in:
- George J. Pappas is working in formally verifying hybrid systems with neural network controllers. In this work they transform a neural network into a hybrid system in order to apply well known hybrid verification techquines: Conference Paper Verisig: verifying safety properties of hybrid systems with ...
-- Guy Katz is working in techniques to verify properties of DNN. In this work they propose a SMT solving technique that applies to DNN with Rectified Linear Unit (ReLU) activation function. Conference Paper Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
- Marta Kwiatkowska (the leader of PRISM model checker project) has been also working on the topic. For instance, in this paper they work on Bayesian neural networks.
Conference Paper Statistical Guarantees for the Robustness of Bayesian Neural Networks
- Badges
- Science topic