Abstract: A remote collaborative assembly and maintenance system was proposed based on mixed reality technology. HoloLens’s spatial perception capability of the depth camera and RGB camera was employed to map the interaction from the 2D gestures on the expert side to the 3D animation markers on the operator side, where the gestures were recognized using an improved method. These simple and vivid 3D markers were located with spatial anchors to provide the operator with clear directions about the next steps for expert-operator-mode collaborative tasks, freeing the operator’s hands from interacting with the computer by gestures. The experiments verify the feasibility of such user-friendly guide interfaces for remote collaboration of assembly and maintenance tasks. 

Key words: remote collaboration, mixed reality, human-computer interaction, assembly and maintenance, HoloLens