One of the most important challenges is still the registration problem. Since the virtual information should precisely supplement the real world, it is crucial that both information matches precisely in their position. While this problem seems to be solved by using markers/fiducials, this solution cannot be used in many real-life applications for several reasons (harsh environments / cultural heritages / esthetical design).

Moreover, it is not just the object that needs to be precisely identified and located, it also the user who is moving around in the real world. Also his position needs to be tracked, and there is still some effort e.g. to further develop the SLAM tracking technology.

The most important challenge is design of economical AR headsets and user awareness about the AR technology.

Augmented reality has several fields, we cannot define a more difficult one. There are several accessories and each one has its difficulty.

I think the most difficult is the philosophy of augmented reality, it is the confidence of all in accepting certain technologies.

One of the most important challenges is still the registration problem. Since the virtual information should precisely supplement the real world, it is crucial that both information matches precisely in their position. While this problem seems to be solved by using markers/fiducials, this solution cannot be used in many real-life applications for several reasons (harsh environments / cultural heritages / esthetical design).

Moreover, it is not just the object that needs to be precisely identified and located, it also the user who is moving around in the real world. Also his position needs to be tracked, and there is still some effort e.g. to further develop the SLAM tracking technology.

I think the registration and recognition problems are the most crucial technical issues at the moment. But the more challenging issue is how to integrate AR with the human eye, and how to have a stable, reliable brain computer connection, to relay information to the battleground, which could be corrupted, hacked or manipulated. Retinal implants with a block-chain system connection could be a solution, but we are in the early phase of AR innovation.

See also this

https://www.upwork.com/hiring/for-clients/biggest-challenges-augmented-reality/

Optics is considered to be one of the most important challenges in AR.

AR is waiting for breakthrough in Optics: performance of optical systems in smartglasses is an important gating factor for the adoption of augmented reality, says Chris Curran.

Optical see-through (OST) display and Video see-through (VST) are likely to be outdated when new engineered materials replace the traditional lenses in AR systems, Flat lenses combined with light-field are shaping the future of AR.

The major challenges are the FoV, weight, image quality, and power.

I would say multi modal interactions

I agree with Vinh T Nguyes that interaction in multimodal AR is a big challenge. It will be a bigger challenge in the future, amy be five-10 years down in the line. Meanwhile other lesser challenges need to be addressed before we get there.