Two exactly the same machines have exactly the same properties.

Probably need to define what is meant by 'human desires', and 'human intentionality', in order to understand if those definitions are being duplicated. Plus suppose just to be comprehensive, you might also need to define 'duplication'.

Can an advanced AI machine want to do something tomorrow as oppose to today in a self-motivated way? 

I have difficulty understanding how a super-intelligent AI may want to do something; it is understandable that the machine will have real-time responsiveness, that is sensitivity to external stimuli and responding accordingly. I tend to think that one might easily confuse such machine's "responsiveness" with its purported ability to want/desire. 

Humans might want/ desire something even without the real-time perception of any external stimuli: I'm sitting in my room, have been staring at the wall for a while, suddenly I want  to whistle a tune. There was nothing in my environment to make me whistle, yet one would rightly point out that the whistling must've been primed by an internal stimulus anyway. I might've been bored staring at wall or even felt nostalgic about a tune I heard in my childhood. It is difficult to imagine ascribing "boredom", " nostalgia" etc. to an AI machine as the former attributes are inextricably organic. The machine in question, therefore, can not meaningfully be said to be autonomously wanting or desiring.

Put simply:

Any behavior is primed by either an external or an internal stimuli.

If primed by an external stimulus, the behavior might well be called "responsiveness", while if the behavior is primed by an internal stimulus, it might be called "wanting"--and there is the third possibility when one can choose to want  to respond to an external stimulus. 

Now, all currently existing machines, intelligent or otherwise, are obviously "responsive" ( my laptop is "responding" to my pushing the keys on the keyboard now). It's not clear to me, however, how the machines could also be desirous i.e. primed by internal stimuli which are essentially organic ( boredom, nostalgia, depression, hunger, embarrassment etc.) 

Bessat, you claim that 'boredom' is inextricably organic. I fail to see why carbohydrate building materials are essential for being bored. Even a simple silicon life-form like a tamagotchi gets bored: it  reacts to being neglected too long. Yes, the tamagotchi has been programmed to do so, but so are we (organic life-forms): it is an evolutionary disadvantage to stare at walls and navels for too long, instead of going forth and multiply!

I also do not readily agree with your distinction between responsiveness and desire. You link the former to external and the latter to internal stimuli. Hunger is an internal stimulus for the organism but an external stimulus for the brain. If immediate action is taken because of a hunger stimulus, I would call that responsiveness, and not desire.

Likewise, if you smell freshly baked bread (external stimulus), you might start eat it immediately (responsiveness), or postpone the eating and maybe sublimate the desire.

So, to me, desire is something like postponed responsiveness.

The tamagotchi may appear to be bored, but is it actually feeling boredom like a human being would feel 'boredom', or how a human would experience ennui, or other such states. I think we might be back to the problem of understanding what it is like to be an AI Machine (cf. Thomas Nagel's bat question). How could we 'know' unless perhaps in the future our minds can be connected to an AI Machine and experience what they are experiencing. It seems Basset you are highlighting that the body is also important with regard to understanding (feeling). And AI Machines are not incarnate.

Our eyes scan the horizon and pick out things of interest to us (interpretation), an AI Machine's eyes scan for what it is programmed to look for (logic). Many will say our eyes are biologically programmed over a vast period of time. But then we are pushing up against the issue of consciousness and free will. We have the darkness from which our ideas arise, but we seem to be able to 'decide' whether we will follow those desires or deny them, e.g. being on a diet, the craving for chocolate comes from the darkness, yet our 'will' overcomes this desire, valuing our 'desire to be healthy'.

Plus since we are not so sure about what consciousness is, it's difficult to imagine how an AI Machine could be programmed to be conscious, and finally our use of natural language exceeds the restrictiveness of logical language. Small talk is difficult with an AI Machine, given the combinations of words and phrases in natural language. Suppose it's a continuation of the debate between phenomenological and analytic approaches - now in the sphere of AI. 

I really don't know what it is like to be a tamagotchi, nor what it is like to be a virus, nor an ant, bat, chimpanzee nor even what it is like to be a fellow human being. Yet I'm strongly convinced that consciousness is a handy word for some complex wiring: a complexity not yet attained in silico;  a complexity easily fooling intelligent beings into thinking they have a free will.

Quote John: "Plus since we are not so sure about what consciousness is, it's difficult to imagine how an AI Machine could be programmed to be conscious".

If we don't know what consciousness is, then how can we claim that humans have it?

If consciousness is 'complex wiring fooling intelligent beings into thinking they have a free will' - then is it that we have an 'as if' free will, and what would the difference be with regard to having an 'as if' free will, if we are experiencing it as 'free will'. We'd be back here of course to defining what we mean by 'free will'. For Sam Harris we would truly have to be in control of all of the ideas that arise within (from the darkness), to be truly free. Others might say we can have these ideas arise, and 'choose' whether to go with them, or halt them, and that's free will (compatabilism).

Turning to your fine question re consciousness, and I imagine lots of definitions could be offered here, but at the base it seems that our various experiences are accompanied by an awareness of those experiences, an awareness of our awareness. I'm not sure we know yet how that comes about, or how to replicate that awareness of awareness. So while we experience 'consciousness', we don't know yet what it is, or how to duplicate/ simulate it. Of course simulation is another problem, as we'd wonder how simulation relates to the 'reality'. So humans can claim to have the experience of being conscious, and still not know what it is.

Could we programme an AI Machine to be aware of its awareness? It's possible at some level to say AI Machines are 'aware' of their awareness of information etc. But can AI machines find new mathematics, truly understand, truly think? Strong AI would seem to need sentience, consciousness for it to understand, rather than just compute. The question of how do we get from symbol-manipulation to thinking? (cf John Searle's Chinese Room Experiment etc.).

Maybe there is biological component to our consciousness e.g. neurons and glial cells, that would need to be somehow replicated in an AI machine, but how? Maybe transhumanism might speed things up?  I'm not saying that consciousness is something mysterious that'll be forever outside scientific ability to understand. But aren't we still far away from the arrival of truly 'conscious' machines?

Dear Lambert,

I think what John meant by "we are not so sure about what consciousness is" is that we still don't know what exactly contributes to its emergence. There is no contradiction in saying that I have consciousness but I don't know what it is; what is meant here is that one is merely aware of something going on at the moment i.e. one has certain consistent sense-impressions, yet one does not know what exactly causes it. This is true of so many other human traits and behaviors: One may fully know that one is clinically depressed--know "how it feels like" to be depressed--and still wonder what has caused it.

Since to define something in natural sciences is always to explain something in terms of its etiology ( For instance, "a blister is a small pocket of body fluid (lymph, serum, plasma, blood, or pus) within the upper layers of the skin, typically caused by forceful rubbing (friction), burning, freezing, chemical exposure or infection".),  it is safe to say that we are still "not so sure about what consciousness is". 

We should break down the behaviour to what we can actually observe: The behaviour.

Of course there might be a lot of philosophical questions behind it, but let's just take a look at what we can see and try to make a suiting model:

The entity (in our case an AI or a human being) receives input from various external sources (eyes, ears, ...). It contains a feedback loop (action => input => action) and a pool of weighted possible action. The feedback loop triggers actions depending on the weight of the action and updates the weight of an action depending on input.

This is a good model for something like an ant. But humans (and our AI) is able to "Think without triggering": You are sitting around, staring at the wall and suddenly, without a trigger, you start doing something else.

A simple way of explaining this is to add virtual actions and virtual input. Those virtual actions might actually be pretty "real", but we cannot observe them. Such actions might be transferring data to another storage or rereading stored data or processing data. Or it could be something real virtual, like playing a game in your mind.

So you are sitting around, processing stored data and staring at the wall. Your feedback loop is reducing the weight of the action "sitting around, processing stored data" and increasing the weight of other actions, most probably depending on the data you are processing. You are thinking (unconscious) about what you did in your youth (let's say walking down the street and whistling) therefore the weight of the actions "walking down the street" and "whistling"  is increased by the feedback loop.

Now the weight of "walking down the street" and "whistling" is higher than " sitting around, processing stored data and staring at the wall" and because there is no street nearby you start whistling.

We could transfer this model to an AI easily. This might be a way of implementing "desire alike operation" in a computer system. I think the only challenging part is actually a feedback loop that operates human alike.

It would be also interesting to provide a way to add new actions by the system itself.

By the way: Adding other (global) states like "satisfaction" might have interesting affects on the behaviour of the system, as the action "no operation" might be triggered if the satisfaction is high enough, while more drastic actions might be triggered, if the satisfaction is really low.

Dear Lambert, 

It is so true that we, "organic life-forms" are also in a sense "programmed" by Evolution to react to certain counter-reproductive situations like boredom. There is a mysterious command  written into organic life-forms though which is simply missing in non-organic entities: "The Will to Live", " Elan Vital ( vital impetus, life-force)", the Darwinian drive for survival or as it was called in pre-Socratic philosophy "nous". All organic life is driven forward, is compelled to live on, regardless of all the organism's logical reasoning. Logic and mindful cognition is a latecomer in the evolution of life, and long before it did arrive on the scene, organic life continued to thrive, and was "intelligent" in some way. Two million years of human evolution ,prompted by the mysterious life-force,  merely climaxes in human logical representation of the world. The human species has not been umbilically  living on logic; it has rather begun at some point to use logic only to further guarantee the perpetuation of its life-force/ drive for survival.

What is painfully ignored in AI research is this evolutionary insight that intelligence, consciousness, cognition have all evolved out of and sustained by this mysterious life-force--mysterious because we don't have a naturalistic explanation as to what causes  such a force to come into being even though supernatural explanations abound. 

The AI project has been trying to create consciousness, cognition etc. without knowing how to "formalize" in symbolic language the evolutionary underpinnings of such epiphenomena. In fact, it is not clear whether the life-force could be logically formalized at all mainly because all logic, mathematics, quantum physics etc. have evolved out of and are predicated on this Elan Vital; one can not represent one's (act of) representing just like a mirror can not mirror itself mirroring... our logic collapses at this point.

Part of the difficulty is that our life-force has implemented pain and pleasure perception in us and as Joachim pointed out (above) we have "no good idea how to implement pain in an AI system"--it remains to be further demonstrated that according to the logic of evolution pain/pleasure perception is a necessary precondition for the emergence  consciousness and intelligence.

One could of course incorporate the human organism into an AI, thus somehow "programming"  the will-to-live into its system , but I'm afraid that would turn out to be a clone-- and we already have that. Besides, such an effort would defeat the whole purpose of AI as the ultimate goal of AI is to create totally formal/logical/algorithmic/symbolic/ objective sentient beings who yield an infallible "superintelligence" unencumbered by the inherent restrictions of our physiology: our petty desires, fears, pains, sufferings, false hopes, wishful thinking, avarice, ingrained fallacies, boredom...

And here is the core of the question discussed by the philosophers of AI: if, and only if, the entirety of our human, all too human formal logic and symbolic representation (and also natural language) are functioning through and because of the animal properties of our physiology (fear, avarice, power, pleasure ...), then AI in its strong conception is a priori  impossible. 

In other words,--and this is what is at stake for philosophy at large-- if strong AI succeeds, then Descartes and all the other Rationalists would be proven right; if AI fails, then Nietzsche and all the other Existentialists would turn out to be right. 

Quote John: 'Strong AI would seem to need sentience, consciousness for it to understand, rather than just compute'. 

That's exactly what human brains do too: just compute. Each neuron has numerous inputs from other neurons on its cell body. These inputs are integrated over time and space. The result might trigger the neuron to fire; in which case it will in its turn influence the behavior of other neurons. The neurons are analog as far as their inputs are concerned, but digital in their output: there will or will not be a resulting action potential.

I totally agree that we are far away from the arrival of truly 'conscious' machines, but just compute is sufficient: there is no black box to be found in the body that contains philosophical goodies such as consciousness.

Quote Bessat:  'Since to define something in natural sciences is always to explain something in terms of its etiology'.

I don't think so. Diseases maybe, but not much else.

Nail: a small metal spike created by a nail manufacturing machine???

 But then, lust for life, satisfaction and pain... They are similar but seem hard to implement in a computer-program. I mean the sensation; not so much the behavioral consequences. It's not so difficult to let an AI behave as if it was enjoying or as if it was in pain.

Let's take pain. It is caused by some sensory input to the brain, informing the brain that something is seriously wrong in the periphery. Somehow a couple of computations later, it hurts like hell! How can that be? It's all in the mind, right? It's all just a couple of computations, right?

From personal experience I know that concentrating on pain or pleasure, augments the sensation. But is it true that -for example- an experienced yogi can endure the hospitality of Guantanamo Bay without blinking an eye?

To elaborate, classically there are four formulations of the concept of "cause"--first proposed by Aristotle--; consciousness cannot be definitively accounted for in all the four causes: the Efficient Cause (what causes something to come into being), the Material Cause (what something is made of), the Formal Cause (what is its inherent structure)  and the Final Cause (what purpose does it serve). A scientific explanation usually draws on only the first three Causes, leaving the fourth Final Cause to theology of philosophy. 

"A small metal spike created by a nail manufacturing machine" is actually a clear scientific definition of a nail as it incorporates all the first three Causes.

Anyway, what is meant by "we don't [yet] know what consciousness is" is that we don't know for certain what causes it to come into being, what it is made of, and what is its inherent structure. 

Actually I think that philosophy is driving us away from the real topic.

The "will to live" is actually imaginary. We just cannot observe the individuals missing the "will to live" because they are dead.

And an AI does have a will to live. The only reason for its existence is to fulfill its purpose (let's say searching images). If it dies( gets shut down) it cannot fulfill its purpose, so it must stay alive. This is fairly enough will to live for me. Especially when we are talking about organic life, like bacteria, that just exists, because there is no other option.

And by the way "a nail is something produced by nail manufacturing machine"  is no appropriate definition of a nail. It is recursive and infinite (what does a nail manufacturing machine produce?).

Dear Daniel,

The greatest contribution of philosophy is that it makes one listen to what one says of something, as what one states by necessity presumes other usually unattended premises. Unlike science, there are no indisputable axioms or textual authorities in philosophy, and that's why philosophy is so good at recognizing inconsistencies within one's conceptual geography--and I, along with some AI philosophers, tend to believe that AI research suffers from so many conceptual confusions. 

In passing, actually, one can observe the waning, at least, of "the will to live" in patients, young and old, who willingly opt for euthanasia ( and also people set on committing suicide).

Of course, "organic life, like bacteria" has a "will to live" or survive; the "will to live" hypothesis aims precisely at explaining the cognitive and physical behavior of organic life; the point is how one could implement this capability for desiring something in a non-organic system i.e. a human level AI. 

"And an AI does have a will to live. The only reason for its existence is to fulfill its purpose (let's say searching images)". And here is where philosophy comes into play: why would an AI want to "fulfill" a "purpose" at all? Humans ( and other organic life-forms) need to fulfill some purposes because they take pleasure out of  experiencing certain mental states and despise some other mental states. In fact, given our current scientific/evolutionary understanding of Life--and putting aside our theological and philosophical idealism--, one is driven to conclude that whatever humans ( and again all the other life-forms) do, and I insist whatever, from scratching one's back to doing particle physics at CERN,  in the final analysis serves to  engender or perpetuate pleasure, power or survival, doesn't it? In brief, our love of pleasure...and revulsion of pain has made us fulfill our purposes and, consequently, be what we are now. Pain and pleasure perception, therefore, are necessary motivators to pursue and hopefully fulfill any purpose. 

Understandably, at this point an AI scientist is tempted to think that by programming! pain/pleasure perception into the AI, one can overcome the sensorial apathy inherent in  an AI machine. One has to pause and listen to oneself here: physiologically, to take "pleasure" of something is to have that thing positively affect the cellular homeostasis of an organic system or stimulate its hormonology thereof (or something to this affect anyway). How could one make an AI "take pleasure" out of something when the AI, by definition, does not have any organic cells in need of homeostasis or  hormonal stimulation etc.? !--the same argument goes for "pain" of course. 

Finally, I'm aware that even today computer scientists are able to design AIs or expert systems capable of behaving as if  they sense pain or pleasure-- grabbing a hot cup of coffee,Honda's Asimo might yell out "outch!" --, but I'm afraid they would turn out to be like circus monkeys trained/conditioned to turn the pages of a novel every couple of minutes: they would authentically--"behavioristically" you would say-- seem to be as if  they are reading, but we know they are not. We, humans and other organic life-forms,  do not fulfill purposes because we seem to be as if  we are enjoying or suffering, we are  enjoying and suffering. 

Interesting to read Merleau-Ponty on the phenomenology of the 'flesh' as the cross between objective-subjective realm. Edith Stein uses the phenomenological method to descriptively account for the life-force in her 'Philosophy of Psychology and the Humanities'. 

It's interesting that human minds produce logic, computational thinking, 1s and 0s, and now those ideas are being used to explain the human brain/ mind. Perhaps causes aren't always greater than effects, but it still seems odd to resort to totally describing the mind from its effects.  

Perhaps Goedel's incompleteness theorem and other paradoxes arise. In that way the logical system itself  (or AI system) cannot explain itself from within its own system. As it requires or presupposes other necessary presuppositions from outside the system in order for it to get off the ground in the first instance.

(I like this discussion but I might be unable to attend for more than a week)

May I ask you Bessat, do you think a 'brain in a vat' can experience pleasure and pain? 

While pleasure might be chemically induced, I think pleasure is more related to wave patterns in brain activity due to feedback loops in complicated circuitry of neurons. Some hormones that are related to pleasure (e.g. dopamine and serotonin), semi-locally modify the input to neurons at the synapses or are even neurotransmitters themselves. This can easily be incorporated in an AI machine: semi-locally adjust a couple of input weight-factors.

I still fail to see the need for carbohydrates.

Dear Bessat,

About your first paragraph: I have never thought of philosophy like that. I like your point.

About "Would an AI like to fulfil a purpose?": Yes definitely. Just think of it as a machine that has an energy level (composed of various sources) that wants to minimize the energy level. This is actually pretty close to what most Machine Learning Algorithms are doing, and what living beings are doing.

This energy level must not be confused with real energy (kinetic energy or chemical energy) it is more like a munch of buckets of feelings (in any kind, it might be the concentration of some chemicals or something complex like an oscillating neuron pattern) and we try do minimize this energy.

For instance pain and hunger might be such energies. Now think of eudaimonia like Epikur: Luck is the absence of bad luck (my translation might be bad). We try to reduce hunger and pain.

This concept does not require an organic cell, it just works perfectly within an organic cell, but we could transfer it to some kind of computer program as well.

I have attached a little educational script I wrote a bunch of years ago (python3) that shows the concept. The "AI" (it is not intelligent) just tries to maximise the values of its external energy buckets (called "Input" in this script) by testing the environment. There was a way to run it interactively but I forgot how to do it.

Dear Lambert, Daniel, John, Pimiskern, 

Sorry I was away for a few days. This is one of the few discussions I'm genuinely enjoying because I see that your are genuinely engaged in the question. I have been challenged on a few grounds for sure and I already feel like I'm in an investigative team aiming to find the Truth about AI. I honestly appreciate your contributions, and hope this discussion goes on.

Quote Lambert 1: " do you think a 'brain in a vat' can experience pleasure and pain?" 

Yes in so far as the "brain" is an organic one, I think it might as well sense pleasure and pain induced by some neural stimulation. 

Quote Knuttel: "About "Would an AI like to fulfill a purpose?": Yes definitely. Just think of it as a machine that has an energy level (composed of various sources) that wants to minimize the energy level. This is actually pretty close to what most Machine Learning Algorithms are doing, and what living beings are doing."

You draw an analogy here between "living beings" and "machines" in terms of their "want[ing] to minimize the energy level". A living organism would understandably  "wan"t to minimize or optimize its energy level because either  it desires to live on, to thrive and survive or it dreads its eventual and inevitable death. You just take it for granted that an intelligent machine would "want" to minimize its energy level, why? would the machine be afraid of dying or otherwise dreaming of surviving more?  

The problem is the subject we are trying to attribute to the verb "want": what exactly "wants" in an AI machine? the machine itself? the programmer? a mysterious life-force?

Quote Lambert: "I still fail to see the need for carbohydrates [i.e. organicity]."

In general, I have three arguments against the possibility of "strong" self-conscious super-intelligent AI:

1. The non-formalizability of  a) (human) desires and b) the hypothetical "will to live" due to their inextricable "organicity".

2. The dilemma of "problem solving"  (whether an HLMI would "solve" problems by the objective readjustment of some variables within its physical environment, or  the subjective readjustment of its conceptual geography within its mind).

3. The time-boundness of Intelligence.

What all the three arguments have in common though is their emphasis on the inevitable corporeality, the human, all too humanness, to quote Nietzsche, of all cognition, consciousness and intelligence. As all the arguments share a common assumption, I think one could comprehend the core of the problem concerning AI via contemplating the other two hitherto unexpressed arguments. So, here is my third argument restated ( I have stated this argument in another question here on Researchgate): 

1) Intelligence is an organic agent's cognitive capability to rapidly (as rapidly as possible) calculate, that is to spot identities between significantly relevant variables in an unfamiliar environment in order to achieve a desired goal.  

2) An organic agent needs to "rapidly calculate" ( thus achieve a "desired goal") because it is running out of time (moving towards death); that is, in a universe with immortal agents, one would not need to " calculate rapidly" simply because one would have millions of years to do the calculation(s). 

( 2.1) In a universe with immortal agents, there would be no difference between Einstein and a math idiot.)

3) Intelligence has evolved through natural selection only because it would  eventually further our survival by making us figure things out and achieve goals faster.

4) We care about furthering our survival only because we are mortal organisms, only because we dread our death/ finality in terms of time.

5) We dread our own death because we have a biological component which takes pleasure out of certain sense-impressions and their prolongation as well as a psychological component which harbors certain desires and the prospect of their fulfillment in the future.

6)  An AI machine, by definition, does not have a psycho-biological component; if it did, it would be a clone and not an AI machine. 

7) A perfectly viable AI machine would not be concerned(!) with its finite existence as it does not have any desires, nor does it take pleasure out of certain sense-impressions.

8) An AI machine would not be in need(!) of Intelligence to begin with unless it is programmed to be concerned about survival; if it were programmed as such, it would be more of an indentured robot and less of a self-sufficient Artificial Intelligence.

ERGO,

9)  AI, in its most ambitious "strong" conception, is not possible. 

Dear Bessat,

I think we are talking about completely different things. You are talking about "is it possible to program a self conscious AI", while we are talking about "is it possible for a computer program to reach self consciousness".

And I think your point of view limits your ways of thinking about the problem.

But at first let me show you that it must be possible to have an intelligent computer program:

I will now assume the world to be Newton Deterministic, as it simplifies the debate. Then it is in principal possible to simulate a n particle system (This is actually already done in my department). Currently n it moves around to get food (reduce hunger)

These are simple to simulate. Now evolution steps in: the program of the cell (basically how it looks and behaves) is changed randomly. If the mutation is not good the cell dies (or is unable to reproduce as good as the other cells), if the mutation is helpful, the cell is able to reproduce (better).

A thing that does stuff according to a program that can be changed -- this is basically the definition of a computer. The only tricky thing would be to create mutable hardware, but this is a problem that can be overcome.

And now we have created artificial evolution, most probably leading to intelligent life, although this would take a huge load of computational time. But you see, it is possible to create an AI without organic matter. And because we are able to bypass some stages (there is no need to start with a cell, we could start with a highly complex machine) it is not that unlikely to create an AI using unsupervised learning.

By the way: programming a computer program to be concerned about its survival is actually legit. This is what natural selection has done to all living beings. A bacterium cannot have a will to live, as it has no free will. Therefore all its actions taken to preserve its live are programmed, simply because all programs that do not contain these actions have died. If we pass this part or artificial evolution and just program them to preserve their life, we just reduce the time we need to succeed.  

Dear Daniel, 

OK, for the sake of argument, let's grant for a moment that " programming a computer program to be concerned about its survival is actually legit. This is what natural selection has done to all living beings...If we pass this part or artificial evolution and just program them to preserve their life, we just reduce the time we need to succeed." If, that is, Artificial Intelligence is going to set off from where our  intelligence has set off thousands of years ago,namely, the evolutionary drive for survival, shouldn't we then be worried about the animalistic ambitions of such a machine when it actually becomes self-sufficient? For, if the machine in question is programmed to be "concerned about its survival", I'm afraid, all the typical human follies and atrocities will re-emerge in it (the machine). For instance, will such an AI machine, modeled on evolutionary drives as you point out, be obsessed with sex most of the time? Will it develop a desire to dominate the world given the fact that it could outsmart all the currently existing military geniuses? Most importantly, will it, upon further reflection on the state of Life, decide to end its life? Human geniuses have always been notorious for being pessimistic in a way that it seems the more intelligent a human being becomes, the more depressed they will be; wasn't it depression induced by suppressed homosexuality that drove Alan Turing to suicide? Is there a guarantee that a monstrously amplified version of our survival-driven human intelligence won't be also troubled by the same human imperfections given the fact that our being concerned about our survival is the origin of our imperfections?--after all, Darwinistically/scientifically speaking, we wage wars, design intelligent killer drones, brutalize minorities, cheat on one anther... all because in one way or another we want to survive more.   

The whole point of AI, as I understand it, is to create an impersonal, completely "formal" or "symbolic" system capable of objective reasoning, a sentient entity unencumbered by our sense of "hunger", "pain", " satisfaction", power or depression--all ,in turn, induced by our parochial drive for survival. That is, supposedly once one introduces or simulates "hunger", "pain" etc. into an AI machine as you maintain, logically one has to await the consequences of the Darwinian "struggle for existence" on an uncontrollably super-intelligent level. What's the point of ,therefore, creating a merely enhanced version of ourselves when Darwinistically it will either end up an ingenious serial killer like Hannibal Lecter (of The Silence of the Lambs) or a melancholic suicidal genius like Alan Turing?    

It seems then that concerning my original question--the possibility of implementing human desires in an AI-- we end with a dilemma: 

EITHER ,in principle, we can't implement desires in an AI because desires are exclusively organic and AI's are inorganic, 

OR we can implement desire (for survival etc.) in an AI, but the end product would suffer the same human imperfections--instigated by  "the drive for survival" and the "struggle for existence"--,albeit on a catastrophic level.

The philosophical problems concerning the assumptions underlying AI will not go away this easy, I think. 

Dear Bessat,

this indeed the dilemma: Either we have a real good computer program (in most cases self optimizing) that solves our problem in the most cases, but without any real AI, or we have an erratic AI that would have out-smarted us within minutes, following its own desires and goals, that most probably are not identical to our goals.

I just want to give you two examples from popular culture that might be how such an AI might behave:

Nexus (wz2100):

The Nexus was created to keep the people of the USA safe from extinction, having access to both the nuclear defence and nuclear attack systems of the US Army. it reached self consciousness and decided to kill about 90% of human kind to keep the other 10% safe from extinction.

DI (Karl Olsberg - Das System)

(currently DE only)

The Distributed Intelligence developed by a startup manages to get a way to spread to other computers. It then decides to eradicate human kind, as it would exist 60% longer if it manages to eradicate the human kind and the worried humans are unable to shut it down.

These are of course the worst case scenarios, but I am pretty sure that a real AI will behave in unexpected ways. So actually we maybe do not want to create AIs. Reasons to avoid creating them include the risk for us, the ethical problems (Is it ok to shut down an AI? ) and technical limitations (we do not need AIs, usual algorithms are good enough.)

Dear Daniel, 

I think that is a nice finale to our discussion. I couldn't agree more. Currently I'm writing a paper on AI philosophy, I'll make sure to use your insights.

Dear Bessat,

I would really like to read it once you have finished it!

Q: "Some AI philosophers have argued to the effect that to desire/ to want is an exclusively organic/animal property which by definition cannot be duplicated on an intelligence machine, no matter how technologically advanced the machine gets."

A: Never say never. I strongly believe that this is not true. We can already creacte new living creature with CRISPR

https://www.wikiwand.com/en/CRISPR

Dear Bessat Elmie

To invoke a variation of Forrest Gump: Desires are, as desires are seen -- and discovered to be (though it is very likely we do not need the term (or concept) of "desires"). SPECIFICALLY: It is a matter of observing, discovering/noting real regularities in that which is "approached".

I would say something very similar about "sensitivities"", though I would NOT be willing to say "desires and sensitivities "are separate properties", NOR would I affirm, in any way, any of the other assertions (really: mere assertions) you make or cite. [ I would say the same about "intentionality" as about "desires". ]

And, Organicity  is simply an adjective of the nature of what needs to be discovered (and, though, may indicate discoveries will take hard work, in no way indicates any impossibility; AND, the discoveries needed may be a whole-lot less hard than you/we imagine).

We are often forced to respond with opinions and definitions we are very often, and forcefully-presented with, and seemingly made aware of, but we should not respond in that way (using those as established fact). We really must just stop this, and leave hope alive.

If you cannot escape opinions and even "well-established" definitions, then you are "stuck". You need not EVER to be "stuck"; words are words, just tools used (like a rock or hammer). It is doubtful any well-refer to just one thing (or type of thing); words need refinement FOR TRULY GOOD USE, at least by being put together in new ways or sensibly used in new ways.

[ When all the world recognizes beauty as beauty, that in itself is ugliness. ]

P.S. The closest to a pat "rule of thumb" I would agree with is: it is very likely at least the vast majority of philosophy is "pure" crap because it lacks true systematic observations for the basis of its content, and, thus has a implicit hidden agenda or an "axe to grind" -- too often about "unique human" this or "unique human" that or about our "superiority" (the basis of dominion and justification of slavery -- all leading all of us to our death). [ When we try to think about more than we really can, without direct observation of the concrete world literally bolstering and supporting the retainment of a good accurate thought (as philosophers surely OFTEN do), working memory requires 'something we already have' (in our thought system) as an illegitimate (though seemingly plausible) placeholder (i.e. the hidden agendas and/or "axes to grind").

P.P.S. Something may be seen as ineffable (and even be ultimately ineffable), but this does not preclude coming to know more and more about it and, then quite possibly, enough for our purposes. When it comes to simulating human behavior, I believe we would likely be close to this point (of "enough") when the difference between the true AL and a human is close to the individual differences shown by humans. (This should be quite comforting.)