Abstract:  Projector interaction has freed the interaction space from the electronic screen, which is more eye-friendly than other display modes and can produce abundant interesting interaction effects. In addition,the interaction with pen-writing is inherently natural and efficient. With the integration of the advantages of both, a projection interactive method was proposed based on common pen and paper. Users can interact with the computer through common paper and pen, while the camera was employed to collect the handwriting images, and projector was utilized to present recognition result. Moreover, with the integration of CNN and KNN, the present method can automatically adapt to users’ handwriting style,so as to increase the recognition accuracy. Based on this, two typical applications are designed, including English-word remembering and simple mathematical calculation. The experiments and user surveys verified that the proposed interaction method is more natural, convenient and practical than other interaction methods. 

Key words:  , common pen and paper, projector interaction, handwriting recognition, pen based user interface