Abstract: The traditional chat cartoon emoticon technologies are mainly based on the predefined chat cartoon emoticon library. Through the semantic description of users, the “semantic-to-visual” cross-modal retrieval is carried out to match the appropriate emoticon. However, the number of predefined emoticon samples in the library is limited and fixed. In the actual chat scenarios, the emoticon is often mismatched or there is no match at all. In view of this problem, this research focused on synthesizing new chat cartoon emoticon rather than retrieval. A new method of cross-modal chat cartoon emoticon synthesis based on knowledge meta-model was designed. According to the semantic description of users, the corresponding chat cartoon emoticons were synthesized immediately. The method established the inner semantic logic relation of chat cartoon emoticon through the knowledge meta-model, and enhanced the semantic consistency of chat cartoon emoticon synthesis. Through the multi-generator model, the corresponding partial chat cartoon emoticons were synthesized from each meta-knowledge point, and then integrated into a complete cartoon emoticon by the joint generator, which greatly reduced the training sample demand. In the test of public chat cartoon emoticon synthesis data set, the method has achieved better semantic consistency, and it is comparable with the existing methods in the quality of synthesized image.  

Key words:  , image synthesis, cross-modal learning, text to image (T2I), knowledge meta-model, emoticon pack