Abstract: To solve the problem of human-computer natural interaction in the virtual training of
astronauts, a multi-somatosensory driven method is proposed based on posture / gesture recognition
and human motion characteristics. With the the advantages of Kinect device which can completely
recognize human posture characteristics and LeapMotion device which can accurately identify
gestures, the method of data transfer based on judgement is put forward. Hand joints are recognized
and the related data are processed and calculated on the basis of the recognition of joints of the whole
body. These two are combined by using the multi-somatosensory driven method, and the experiment
is carried out. The results show that by using LeapMotion and Kinect to recognize hand joints, when
the gesture is within the range of LeapMotion recognition, we can add more precise gesture
recognition to the realization of human somatosensory recognition. This method has successfully
realized the combination of human posture recognition and precise gesture recognition, and can be
applied to the human-computer natural interaction in the virtual training of astronauts.

Key words: astronaut, virtual training, somatosensory recognition, data fusion, interaction