Deep Learning-Based Object Tracking for Augmented Reality: A System-Level Survey of Methods, Constraints, and Challenges

Abstract

The immersiveness and usability of augmented reality (AR) systems rely on accurate, temporally stable, and computationally efficient object tracking. Recent advances in deep learning have reshaped visual tracking and enabled increasingly complex AR applications on mobile and edge platforms. As AR progresses toward large-scale consumer and industrial deployment, tracking has become a system-critical perception module that directly affects visual stability, interaction latency, and user trust. This survey reviews deep learning based object tracking for AR from 2018 to 2025, focusing on algorithmic paradigms and system-level constraints. We analyze AR-specific requirements such as tight latency budgets, limited energy, long-term operation, and perceptual stability, and examine four representative paradigms (Siamese networks, deep discriminative correlation filters, Transformer based models, and long-term frameworks) with their design rationales and deployment challenges. We further discuss lightweight architectures, state-space temporal models, and diffusion-based approaches, along with integration strategies involving efficiency optimization, hardware-aware design, 6-DoF pose tracking, SLAM coupling, neural scene representations, and multimodal fusion. Representative datasets and evaluation protocols are analyzed from an AR deployment viewpoint, and open challenges and future research directions are identified. We argue that AR-oriented tracking constitutes a distinct research domain where algorithmic accuracy, perceptual stability, and system efficiency must be jointly optimized to support trustworthy and immersive next-generation AR experiences.