The multimodality of Chinese folklore images

Nemkina Nikol Igorevna Postgraduate student; Higher School of Translation and Interpreting; Lomonosov Moscow State University 22 Saratovskaya str., Moscow, 109125, Russia nikol.nemkina@mail.ru

The image of the hero of a folklore work, its conceptual core, expressed by various semantic characteristics in the text, is formed due to the peculiarities of perception of the surrounding world, based on universal, national and individual associative components. The text of the work forms the image of the hero with the help of various techniques and means used in the system of folklore. Thanks to the portrait characteristics, an external image is created. Functional characteristics allow you to describe the image of the hero through his actions and deeds. In addition, the speech characteristics of the hero, his relationship with other characters, the objects surrounding him and the place in which he is located – all this to a certain extent creates a unique image in the recipient's mind.    

For semantic analysis, it is also necessary to take into account the non-verbal (social, cultural, psychological) context of the utterance. "The extra-linguistic aspects of communication, correlating with the verbal form of their representation in the text, constitute a discourse in their entirety" ^{[1, p. 38]}. With the development of new communication channels, changes in the usual ways of communication and perception of information, the term multimodality has become increasingly used in modern linguistic science. Currently, various interpretations of this concept can be found, from a simple "text + image" to a more complex one – "a semiotic phenomenon combining in a single communicative space of signs sign complexes belonging to semiotic systems that differ in the way (de) encoding information – verbal, graphic, musical, mathematical, gestural, etc." ^{[2, p. 319]}. Thus, in the modern discursive space, multimodality is based on various methods of description using audio, video and text material.

The relevance of the research is due to the growing interest in the cultural heritage of the Chinese people, which is most vividly embodied in folklore, where the fundamental category is the folklore image.

The scientific novelty of the work is due to the fact that the images of fictional characters are described for the first time in the modern representation of Chinese folklore, taking into account the discursive factors of the modern linguistic environment, characterized by a high degree of multimodality.

The theoretical significance of this study is determined by the introduction of an integrated approach combining analysis at the linguistic and conceptual levels. The significance of this study also lies in the fact that it can complement research in the field of multimodality and comparative study of images of folklore works in different languages.

The practical significance of the research lies in the fact that its results can be used during courses on folklore, discursology, regional studies, linguoculturology and cognitive linguistics, as well as in classes of a practical course of the Chinese language and comparative study of languages and cultures of China and Russia. 

In addition to the formation of the meaning of the image due to the anthroponymic material and functioning in the text, another semantic element is the multimodal features of the image. Such features are most clearly expressed in the description of the image using multimedia (images, sounds), through expressive characteristics (gestures, posture, facial expressions and prosody), as well as through multimodal metaphors. According to some researchers, "with the advent of the focus of multimodality, the very way of perceiving the world has changed. The word that previously dominated as the main medium of information is replaced by an image that is expressed in the priority of the visual series over the verbal one" ^{[3, p. 70]}.

Thus, the main categorical feature contained in multimodality is multiplicity. This concept includes channels of perception and transmission of information, including with the help of material means and technologies that contribute to the formation of a fundamentally new structure of texts in the media space, as well as the synthesis of textual forms and extralinguistic elements – the identification of semantic fields as various codes of presentation of information. The applied aspects of the development of multimodality are extremely important, first of all, multimodal discourse analysis: combining the procedures of discourse analysis with the inclusion of non-linguistic elements of the message in the area of attention. The need to take into account the full range of meanings involved in discursive practices contributes to the dissemination of research on multimodal discourse within the framework of an interdisciplinary approach.  

The trend of referring to folklore heroes within the framework of modern discourse is gaining more and more popularity. The social and cultural-historical prerequisites for such a transdiscursive transition include the desire to embark on the path of development of culture and its features, based on the demands of modern society living in a multimodal linguistic environment. The peculiarities of this kind of interaction between people and the revised text include cultural and mental unity, which allows understanding the text at the extralinguistic level, when "communicants rely on socio-cultural, historical, scientific, educational and intellectual factors that are not always presented in text format, but are felt in the process of text generation" ^{[4, p. 550]}.

"In the secondary text, the change is aimed not only at the word, but at the system of images and the artistic world as a whole" [5: p. 159]. Linguistic and conceptual features may be subject to changes, which may manifest itself both in a slight difference in basic features and in their complete opposite. Nevertheless, replacing one or more features does not always lead to a change in the conceptual core. Reduction as one of the forms of secondary manifestation can contribute either to the complete loss of the signs of the concept, or lead to its subsequent development. Expanding to the possible limit, the concept highlights the core, the periphery and the conceptual dominant. "In addition to the traditional division of conceptology into the core, near and far periphery, it became possible to talk about the conceptual dominant and propose to introduce this concept in relation to the study of concepts with heterogeneous, multilevel content, as well as often presented in works of art" [5: p. 301]. The space provided by modern technologies, as well as the dynamic and interactive nature of language and its sensitivity to context, contribute to expanding opportunities for the development of the content of the concept and activate the processes associated with its functioning in a different discursive environment.  

In China, the first mobile game from Tencent called "King of Glory" has gained popularity not only within the country. In it, you can meet the heroes of Chinese myths and fairy tales, such as Pangu and Nuiva. In this game, Pangu is a warrior holding an axe, his virtual role and external characteristics correspond to the representation in the myth. The images of the archer Hou Yi and Chang 'e have also not undergone significant modification, Hou Yi fights with his magic bow, and Chang 'e is still beautiful and her actions in the game are accompanied by the main attributes – the hare and the moon. In addition, in the game, as in the myth, Chang Ae and Hou Yi are a married couple.

Nevertheless, another character is Nyuva, despite the fact that in the video game she has the abilities of creation and protection, her role and image have undergone almost complete modification both at the level of appearance and internal. Firstly, the classic image of Nyuva is half-woman, half-snake, she is usually depicted with a human head and a snake body, but in the mobile game she is represented in human form. Secondly, in China, Nyuva is considered a deity who gave life, she is elevated to the cult of the mother of the ancestress, in the video game Nyuva is a magician who attacks others and uses divine abilities to destroy. Some users of this video game believe that such disregard for the real features and the role of the hero can undermine people's ideas about the sacred images of Chinese mythology. 

Acting as a special communicative environment, video games contribute to the formation of meaning depending on the context of the conceived plot, as well as on the capabilities of the characters that determine their strength and role in the outcome of the game. "Modifications of special meanings are associated not only with a change in the conceptual scope of the term, ... but also with the way the meaning is constructed in the text ^{[6, p. 102]}. Consequently, due to the new conceptual characteristics caused by the plot of the video game, the volume of the concept increases, acquiring new shades of meaning.

It should be noted that in this video game, in addition to the traditional Chinese characters Pangu, Chang E, Hou Yi and Nezha, there are heroes of ancient Greek mythology, for example, the goddess of wisdom Athena. Such a mixture of heroes of not only different Chinese myths, but also heroes of ancient Greek mythology, indicates a high degree of intertextuality, which can greatly complicate the user's perception. One player named Lu Qiutong (), commenting on the transformation of the image of Nuiva in the game, wrote that analyzing all the abilities, external characteristics and function of the hero in the game, he mistakenly mistook Nuiva for the goddess of ancient Greek mythology Athena, which indicates the collision of two different conceptual spheres of images, due to their erroneous equalization in the game.

Another user of the game named Yan Saujani () said: ‘The popularization of mythological knowledge in this game is very limited. “King of Glory” can form an image in the player's mind only based on the external characteristics of the game character, but it does not contribute to the spread of spiritual values and deeper meanings contained in real myths.^[7]

Nevertheless, the creators of the game pursue other goals, in particular, commercial ones. Undoubtedly, appealing to previously created verbal texts allows the authors of a mobile game to fit their electronic work into the context of Chinese culture, thereby attracting more users, but most players during the game hardly think about misinterpreting the images of heroes of ancient myths, for them the primary task is to complete tasks and move to the next level. Thus, during the virtual game, the "formation of knowledge structures hidden from observation" takes place ^{[6, p. 101]}, which is enhanced by multimodality, since the user no longer needs to imagine an image described by verbal means with the help of imagination, the appearance of the hero has already been created graphically and presented on the phone screen.

The same goals are pursued by the creators of cartoons featuring mythical characters. One of the most frequent adaptations of the mythical plot is the legend of the white snake, films based on it are popular not only in China, but also have great interest abroad. As examples for the analysis of the image in the adapted work about the White Snake, two of the most famous and modern cartoons were selected:("Flood in Jinshan temple"), released in 2012 in China and("the White snake: the beginning"), released in 2019 in China in conjunction with the American film industry, as well as its sequel2("the White snake 2: Sorrow green snake"), released in 2021.

In the first cartoon, consisting of 52 episodes, in addition to the main characters of the legend of Bai Suzhen or the White Snake, Xiaoqing, Fahai and Xu Xiang, new characters are presented: the antagonist of the frog-faced water snake (), Bodhisattva Guanyin () and the dragon king and lord of waters Long Wang (). The addition of a new antagonist was made in order to fill the plot with contradictions of binary oppositions, when love and hate, good and evil, justice and partiality collide, that is, what corresponds to the way of thinking of most young people who perceive everything in black and former colors, and therefore are among the target audience. 

The image of the White Snake in the cartoon has not changed significantly, Bai Suzhen is still kind and beautiful, but sometimes she resorts to abusing her abilities for selfish purposes, for example, clouding the mind of a seller who wanted to get money for an umbrella purchased by Bai Suzhen. Such a minor change leads to a shift in the peripheral zones of the concept, which are replaced by greed and abuse of magical power.  According to the plot, the White Snake still seeks to repay the kind young man named Xu Xian, who saved her life, therefore, the nuclear concept of nobility is preserved. It should be noted that in this film, the multimodal description of Chinese history is expressed with the help of musical accompaniment in the classical Chinese style. During the viewing, viewers can hear the sound of Chinese musical instruments such as bamboo flute, guzheng and tanggu drum.  

Another animated film, shot in 2019, has also undergone certain adaptations. According to critics, this painting perfectly embodies the features of Chinese culture: ink painting, boating, landscape, pagoda, umbrella, hairpin, lapel and the manner of movement of the characters – all this is presented in the Chinese style. Such a "Chinese" multimodal representation allows us to evaluate this picture as a predominantly cultural work of China, however, according to one of the Russian observers, "the cartoon has clearly become infected with "Disney" with the obligatory comical companion animal and caricature villains" ^[8], which is explained by the release of the picture together with the American film company. Bright special effects and very cute character design contribute to immersion in the aesthetic world of the legend. The positive features of multimodality here include the possibility of a comprehensive perception of the image, from its external features to internal characteristics. 

The plot of the painting is far from the narrative in the legend, it is based on the story of the rescue of Bai Suzhen by Xu Xiang. Despite the fact that the cartoon shows the sacrifice of lovers, moviegoers disagree, taking the love story of Bai Suzhen and Xu Xiang for "a superficial and not the most convincing development of the relationship" ^[8].  In addition, some regard the cartoon as "not quite a children's drama, in which there is a place for cruelty, and even a love scene" ^[8]. Cruelty is mainly inherent in the White Snake, which is a new feature included in the concept of the image. Being at a crossroads, the main character masters a huge force, kills human warriors and destroys the city of Yunzhou. This concept is enhanced by multimodality due to the representation of Bai Suzhen in the image of a huge python.

The story continues in the second part of "White Snake 2: The Sorrow of the Green Snake". Viewers are also inclined to consider this adaptation only a distant resemblance to the classic plot. Despite the shift of the main role from Bai Suzhen to Xiao Qing, it is thanks to the White Snake that Xiao Qing becomes, Bai Suzhen is always there, guiding, helping, but her main task is to pick up the pieces and put Xiao Qing back on her feet. Consequently, the nuclear concept of the White Snake nobility is expressed in this cartoon by helping Xiao Qing and saving her. In addition, multimodality complements the concept of the image of Bai Suzhen by presenting a White Snake in male form. At the end of the story "White Snake 2", we learn that Bai Suzhen, after rebirth, sacrifices himself to save Xiao Qing. Being in a different guise, the White Snake is still distinguished by nobility and courage, and the relationship of once close sisters is metaphorically replaced by the image of lovers, since according to the plot Xiao Qing does not know for a long time that Bai Suzhen has been reborn into a man.  

The concept of the White Snake is expanding, a new concept revealed during the analysis is sacrifice, thanks to which the White Snake is ready to sacrifice its life to save its loved one, whether it is Xu Xian or Xiao Qing.  

Thus, the multimodality embodied in the cartoons about the White Snake can be represented as a combination of several semantic components in a single work, both at the linguistic and non-linguistic levels. This is manifested in visual content, musical accompaniment and verbal expression, as well as metaphorically. The concept of the image of the White Snake retains the nuclear attribute of nobility and adds the conceptual dominant of sacrifice. New signs of abuse of magical power and cruelty can be attributed to the conceptual periphery.  

The culture of ancient China is embodied in the genre of traditional Chinese comedy performance with a predominance of colloquial forms Xiangshen (), characterized by an abundance of puns and allusions. In this genre of performance art, multimodality is clearly expressed in the form of prosody and body language. 

Prosody is very diverse, it includes pauses, accents, tones in accents, tempo and longitude characteristics, tonal registers, degrees of reduction, laryngeal signs, volume, etc. Along with prosody, gestures, posture and facial expressions also carry certain meanings. According to Paul Ekman, facial expressions expressing basic emotions (surprise, fear, anger, disgust, sadness and happiness) are universal, and are perceived in different cultures in the same way ^{[9, pp. 251-252]}.  In all languages and cultures, words and gestures represent a universal set of semiotic means used in the moment of communication, most often spontaneous and involuntary, where conceptual understanding of words and gestures is important ^{[10, p. 9]}.

In one of Xiangsheng's humorous shows, a miniature "Mulan joins the army" was presented. It is based on the Chinese story of Hua Mulan, a brave girl dressed in men's clothes. The image of Mulan is considered a symbol of feminine strength and patriotism in Chinese culture. The miniature, presented in 2015, shows not only the transformed image of a girl, but also the peculiarities of culture, customs and symbols of China, expressed through costumes, hairstyles, decorations and music.

At the beginning of the performance, we see comic Jia Lin as Mulan, dressed in traditional Chinese attire and gnawing on a leg of fried chicken on stage. Her transformation into the image of a voracious simpleton is perceived by the viewer from the first seconds of the performance. Then the emperor's guard appears on the scene, who came to Mulan's house for a recruit. At first glance, the plot develops according to the events in the legend, but the response of the father and daughter, as well as their emotional expression, make it clear that the miniature is far from the well-known story and its development will follow a different line. Mulan enlists in the army not of her own free will, but as a result of her father's lies, who informed the guard that he had a son, not a daughter. In the service, Mulan is awarded the title of assistant general, but this is not due to her bravery and exploits, but due to accidents, interestingly played out in miniature.

Unsurprisingly, the complete opposite of the basic features of the image has led to controversial assessments by viewers. The miniature was based on the story of Mulan, which was played in such a way that the audience could enjoy the humorous subtext with the main focus on turning the image upside down. This is how one of the viewers describes the TV show: 2015?61?‘Comic Jia Lin starred in the comedy sketch "Mulan joins the Army" on a satellite television program in June 2015. In this program, Jia Ling was in the image of Hua Mulan and played her as a greedy, parent-defying, lazy and cowardly girl. Such an adaptation undermined the idea of the traditional image of the heroine Hua Mulan, which caused widespread controversy. And although there are those who believe that such an expression distorts the image of the hero, others call it a new form of representation of a well-known traditional story embodying the values of the entertainment sphere.^[5] Consequently, the destruction of habitual models of interpretation contributes to the creation of a comic effect.  

Multimodality in this TV show enhances the concept of Mulan through vivid emotional expression. Figures 1, 2, 3, 4 depict facial expressions describing various emotional states of the heroine. Since Mulan was in an environment where she had to lie, you can see her eyes downcast in the first three images. It is believed that when a person lies, he tries to avoid direct eye contact and often looks away. All facial expressions are associated with an attempt to avoid punishment or not reveal a lie, which puts the girl in an awkward position, clearly represented by body language. 

Body movements also have a certain amount of semantic material. Having received the task from the commander to run, Mulan ran in a typical way for a girl, throwing her arms in different directions, which caused confusion among the commander and comrades who did not know that Mulan was a girl. At the end of the presented miniature, the audience sees another Mulan, having passed many tests, the girl hardened her character and realized that before achieving anything, it is necessary to sacrifice something.

It should be noted that a common feature of the images viewed from different angles of multimodality is the desire to demonstrate the moment of the hero's formation as a person, experiencing difficulties and overcoming them. The reasons for the emergence of a new plot emphasis may include the desire to approach the preferences of a modern viewer living in a society where motivational discourse, represented by motivation and the desire for self-development, is an important part of the general discourse. 

Thus, multimodality affects the degree of perception of an image in the recipient's mind in different ways, on the one hand, it leads to a simplification of the understanding of meanings, on the other hand, to a complication of systems for creating these meanings. One of the manifestations of this complication is the existence of a high degree of intertextuality, which can undermine ideas about fictional characters, lead to the false assumption of some characters for others. The desire to get closer to the preferences of the modern viewer leads both to a slight difference in the basic features of the concepts and to their complete opposite, which also affects the process of perception of the image. Nevertheless, despite the significant degree of adaptation of images that cause different user assessments, the social and cultural-historical processes of a dynamically changing world require compliance with the conditions of functioning of modern discursive practices.