Augmenting humans with extended reality (XR) systems gives them improved situational awareness, new information and abilities and also enables co-operation through remote presence. Machine learning (ML) based context understanding and human state estimation enable an artificial intelligence (AI) to intervene in stressful situations and ensure optimal performance by adapting the multimodal XR capabilities. Environmental sensing will be implemented by a combination of sensors. The cognitive state of the human will be estimated by wearable biosensors and eye-trackers in conjunction with XR devices. The results of the project will enable optimal use of XR systems for co-operation of humans and AI technologies. We will measure the cognitive responses, task performance, and user experience of humans interacting with this AI-powered XR technology that combines cognitive state modelling, detection, personalization and prediction with task modelling, context awareness and HCI adaptation.

2023

2027