There are increasing numbers of examples of deep learning devices comprising many layers of neuronal nets. These can be trained to carry out intelligent functions such as play Go better than human players. Such devices are now connected to systems like Twitter and Facebook, to learn about the particular actions of groups of individuals. I think it is time that we had algorithms or whatever which could scan these networks and find out exactly what they have learned, about what or whom, and what they intend to do with the information.
A weak element is that knowing what a neural network knows needs human interpretation ...
You’ll need to run the net on all possible inputs, or train another net on other neural networks as inputs.
Thank you, gentlemen, for your answers! Joachim and Imre, simplification is only possible on totally specified neural nets where the owner can perform that process. Then we have to take the owner's word (Facebook?? You must be joking) that they have done the reduction correctly. Imre - that is precisely the point, that humans have to know what these nets are doing in terms of human socioeconomic systems. I am looking for something that can be done by an external agency to 'plumb the depths' of these nets' 'knowledge' and make sure it is not malign.
Joachim exhausted the topic of explainable AI, but nobody has addressed the last bit of your question: "(...) what they intend to do with the information".
Intent is a property of conscious minds. It is a complex state which requires the subject to be capable of several mental activities (e.g. understanding the situation, planning ahead). Artificial intelligence however is not artificial consciousness - just less or more sophisticated maths. So talking about such abilities in the context of neural networks (or any current ML/AI tools) is pointless, in my opinion. What do the owners of such complex systems intend to do with the information is a valid question we should be concerned with. Suggesting that the systems themselves are somehow able to deliberately act, however, is not; it only fuels the sensation-hungry journalists and bloggers who thrive on inciting fear of unknown.
Przemislaw, I am not as optimistic as you regards the view that current AIs lack intention. When an AI plays a game like chess or go, it must have the intent to win, or it wouldn't. Likewise an AI operating a driverless vehicle must have a number of intentions like not hitting anything, not being hit, not injuring it's passengers, not getting lost and so on. But these are trivial examples of AIs. I am more concerned about the AIs of the near future that get connected to the Internet and told to mine it for data. What sort of data can you guarantee they are mining and what is to be done with it. I don't see that intention is solely a property of 'consciousness' because until we can define consciousness rigorously you can't really say what properties it does or does not have. But thank you very much for your thoughts. It is an important topic as Joachim acknowledges!
Tony, what you're describing are not intentions but objectives and they are externally imposed, not internally emergent.
A system that has been trained to play go doesn't know anything else than playing go. Give it a board and it will play go; give it anything else and it will get confused because it only knows go and is only designed to win at go (it will actually interpret it as go and make an attempt to play it, which may or may not make sense).
A car-driving AI has been designed to avoid obstacles and not get lost - it's programming, not a choice. It may fail and hit something, but that's a problem with the system design: there is a technical reason that it did not respect a constraint. Maybe it's an error, maybe it learned to break it in some cases because it helped it optimize the objective function - figuring that out is what Joachim Pimiskern
's answer is about.My point however is that AI is fully defined by the objectives which are external to it, designed by its creator. The creator may have an intent to use the system maliciously, but the system itself - not.
Agree. The tools of the AI, neural networks included, are of the same type as the stones that our ancestors used to open walnuts. The difference is that the stones replace or modify the capabilities of our hands and the AI tools replace or modify the functional organs of the brain, where are the skills of playing Go, calculating simple logical functions, etc. Modifying oneself is one of the ways used by Human Being for evolving. And these tools are created (or searched) and used, following users/creators motives. They, by themselves, don't have motives, intentions.
Regarding explanation, most of applications, based on connectionist paradigm of AI lack of them; particularly neural networks are known by they "hidden" behaviour. On the other hand, symbolism have been more successful in providing it. Probably it is due the top down, deductive scheme, used by them.
Rey and Premislaw, thank you for your informed responses! However I am not so convinced that there is an analogy between a rock or club both with an effective range of action of a few metres and an AI that is connected to the internet. I appreciate that objectives are imposed while intentions are internally generated. What I am concerned over is the ability of the latest - and future - AIs to take objectives and convert them into intentions without human operators being aware of what they are doing. This is why I want AI proponents to come up with ways of examining the workings of the AI layers without the AI being 'aware' - in any sense of the word - that it is being done, so we aren't suddenly faced with unforeseen situations and 'out of control' AIs. I can't accept assurances that it 'can't happen'. Such situations have arisen so many times in the 'advance' of technology that this time we must be forewarned and forearmed, so it doesn't happen!
Many agrees that what NNs (and other connectionist mechanisms) learn are implicit algorithms that could indeed be exploited in various tasks. If we want to know what these mechanisms learn by supervised or unsupervised learning, then we have to be able to extract these algorithms and the associated data/patterns. However, extraction (and reuse) of implicit algorithms is still in its infancy, though the results might be very useful. Visualization still cannot lead to semantic interpretation. As Joachim hinted it, a black-box type of behavioral/performance analysis may be an alternative route, but the clumsiness and uncertainties related to this are already known. Getting out or knowing/learning self-intents of evolving systems, .... yeah, it seems to be a fairy tale in a general case, at least in my view.
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning