In his book [1] Bostrom's orthogonality thesis explanation state that it does not exclude irrational preferences. But in his instrumental thesis he states preferences that he qualifies as likely. What justification is possible for choosing some preferences over the other if irrationality is allowed? In other words, how can it be justified that there is a high probability of these preferences with a specific selection criteria that is highly reasonable given that in principle we have an irrational agent?
At least I would modify it to possible instead of likely
[1] Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies.[Internet]. Superintelligence: Paths, dangers, strategies.
Hi.,
Kindly check this link:
http://www.math.chalmers.se/~olleh/ChallengesOBframeworkDeanonymized.pdf
C K Gomathy ,
Thanks for the reference, since it is relevant to the discussion. I did come across it before I wrote this post, but I took a look at it again to see if it completely addressed the point I am making.
The paper attacks the Bostrom orthogonality and instrumental thesis granting him that these will lead to a super intelligence. It also attacks some of the premises which Bostrom assumes as given.
What I am proposing is that its flaws are more fundamental than thatand I cannot even grant a super AI from it. Even granting prima facie All of Bostrom's assumption (which I agree with the criticism of the paper on some of the attacks to those assumptions) the two theories are inconsistent with one another and the theory is invalid (much less lead to a Super AI and then get into trouble).
My main point: How do you discriminate final goals from instrumental goals in the theory when you allow the agent to be irrational. While in 3.the authors point to the problem their criticism is based on self referentiallity, my point goes to the discrimination criteria between final and instrumental goals in a rational agent. The discrimination needs rational criteria of evaluation and choosing which ones are instrumental and which ones are final goals.
Supporting Point : All the instrumental goals given by Bostrom if followed by the AI to achieve final goals (therefore one can say that the AI must be committed to pursuing those instrumental goals to the end of his computational cycle), then, they are also final goals in themselves(therefore the argument is inconsistent).
_______________________________
Now, since you brought the paper that grants that a super AI may evolve from these (it is afterwards for them where the problem begins), you introduced one of my core arguments in other posts "HOW will this AI be considered super intelligece?" . Bostrom's super AI seems to me more like a "Dumb AI" to say the least!
I think the point of inconsistency merits a paper of its own
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence