The initial codes and thematic clusters of AI-XR for multimodal Human-AI Experience

156 consolidated codes across 5 themes

New interaction paradigms for spatial computing Future of interactions Natural interaction Multimodal context and input No preferences in multimodality New inputs and UI placement Matching input with the task Explicit and implicit interactions AI-based hand and gesture tracking AI-based voice control AI-based body language AI-based haptics and tactile feedback Gaze control No controllers for XR Embodied interaction Personalization of inputs Spatial reasoning Wearables for XR AI-XR communication and telepresence Switching perspectives for communication Human communication-inspired interaction Better UI Visual overlaying Mobile/on-the-go interaction Flexible interaction patterns Feeling depth in XR Audio is underused Vocalization will not be enough Lack of freedom for interaction Gorilla arm / interaction fatigue Midas touch problem “No interface” / zero-UI paradigm Constraints can aid efficiency 2D metaphors are poor fits for 3D

AI as a context-aware co-creator and mediator Agentic AI AI for understanding context/user Inherent personalization of AI Personalized XR through AI AI lacking personal/user context AI for content creation Communicating/decoding intent AI for disambiguation Immediate interactive context/content Interactive environment Directing attention Attention as a scarce resource Digital twinning Semantic graphs from visual scenes Environmental sensing and physical-world understanding Multimodal data collection New data for learning in XR AI for tracking body and behavior Humanlike behavior of AI in 3D Control of environment AI-XR guidance and explanation XR for navigation Always-on/ambient AI-XR Continuous AI-XR experience Proactive/predictive AI-XR Ubiquitous/situated computing in XR Situated/physical-world computing and augmentation Diminished reality AI-XR augmenting people Merging physical spaces Rule-based systems and predetermined patterns limit AI Incomplete AI models of digital/physical context

Feasibility, adoption, and ecosystem readiness Technological and hardware barriers Technological availability Computational requirements Insufficient bandwidth Cost barriers New technical challenges New infrastructure for AI Interaction precision and resolution limits Calibration vs. quality trade-off (eye tracking) Responsiveness of input and system Trade-offs of AI-XR XR proving its value XR not improving AI Barriers in personal performance and user experience Ergonomics Intrusive wearables Aesthetics of XR Social acceptability XR for public settings XR UI for long sessions Frustrating experience Initial friction of adoption Easy step-in/step-out from XR workflows Fast restart Workflow transition/switching friction AI-assisted workflow transitions 3D metaphors 3D is not always better than 2D Text is bad outside desktop Text-centered ecosystem XR UI for tech is the same as general XR UI Lack of specificity of AI for technical work SE in XR is niche SE is more demanding for interaction Lack of human connection Interoperability and shared XR standards Content creation intertwined with coding

XR as a situated, persistent workspace Extended workspace for coding and engineering Mobile workstations XR as primary computing/display platform Personalized organization of the XR workspace XR/AI-XR visualization for data and complex systems XR for robotics/teleoperation Teaching and understanding 3D XR for teaching and training AI-XR prototyping and 3D prototyping AI-XR sketching, drawing, and design AI for general productivity AI for deep research AI as counterfactual / devil’s advocate AI-XR collaboration XR for human-AI interaction (HAX) AI-XR multi-tasking XR avatars and digital self-representation AI-XR video creation/editing End-to-end creation in XR Iterative creation in XR AI-XR for creating variations Tools for 3D engineering Tools for technology creation Tech creation when no more applications Writing code is not the core concern anymore VR for 3D printing XR for game development/gaming 4D videography Immersive tourism Deep immersion

Privacy concerns / convenience trade-off Security in XR (next-level phishing) Ethical concerns Governmental regulations AI regulations do not work in advance Rights, authorship, and ownership AI transparency and black-box concerns XR for explaining AI and transparency Alignment Preserving human-in-the-loop Technology overreliance Overuse of AI features (e.g., beautify) Potential exploitation through XR Commercialization of AR/XR environments Societal changes Social concerns Changing emotional landscape 24/7 technology use Constantly-on devices Digital copies, twinning, and identity risks