You're begging the question here by assuming that we already know what "super-human intelligence" is, that we'd recognize it if we saw it, and that it doesn't already exist. We don't yet understand human intelligence, so I think we're a long way off from recognizing a super-human variant.

Excellent answer Matthew, I think exactly the same. The concept of human intelligence, or even intelligence itself, is still very hazy. We don't yet understand enough of what's going in our brains to give an encompassing definition, much less to try to identify it in other systems. As exemplified with John Searle's mental experiment, the "Chinesse room" (http://en.wikipedia.org/wiki/Chinese_room), but specially with the counter arguments to it; apparently, we are not really able to know if the processes occuring inside the systems are of higher order of just a correlational algorithm of questions and answers, but more important than that, we are not really able to know if that's what is actually occuring in our brains too; that's because we can only analyze and judge an observable, such as an answer or a decision. In short, the real problem appears to be, not the ability to create that intelligence itself, but our ability to identify it. What would we judge? Processing capacity...I don't think so, we are outperformed long ago in particular tasks; and the Turing test can be missleading. What do you think?

Hi Daniel,

I definitely agree with you that we don't have a monolithic qualitative definition of intelligence. We can test characteristics of intelligence (like spatial reasoning or working memory) with things like WAIS (http://en.wikipedia.org/wiki/Wechsler_Adult_Intelligence_Scale), but these tests make clear that human intelligence isn't just one thing. Computers are already super-human when it comes to working memory and processing speed, but obviously we don't mean that when we talk about super-human intelligence.

The Chinese Room Argument argues that perfectly accurate and instantaneous responses do not require understanding to produce. And yet, I wonder - even if we could provide a technical definition for understanding (qualia), could we make a machine with super-human understanding? What would that mean? And if it understands things that no human can, I don't think we could even notice.

Once again, the problem seems to be very essential. It's not about what it is, but what we are able to identify; thats actually a key concept in the idea of a "technocological singularity", from which the unpredictability afetrwards arises. I think that the most important issue regarding a super-human intelligence is not an entity which performs orders of magnitude better than us in the same functions, but an entity which becomes able to conceive things that are just unconceivable to our minds; an entity that, for example, is able to think about thinks that are plain paradoxical to us, like the universe before the Big Bang, where there is actually not even a "before".