An information model of the data structure and an experimental technique for improving the human-computer graphical interface

Borevich Ekaterina Vladislavovna ORCID: 0000-0001-6263-3901 Assistant of the Higher School of Design and Architecture, Peter the Great St. Petersburg Polytechnic University 195251, Russia, Sankt-Peterburg oblast', g. Saint Petersburg, ul. Politekhnicheskaya, 29 plasma5210@mail.ru Other publications by this author

Yanchus Viktor Edmundasovich ORCID: 0000-0001-7220-0819 PhD in Technical Science Associate Professor, Department of Higher School of Design and Architecture, Peter the Great St. Petersburg Polytechnic University 195251, Russia, Sankt-Peterburg oblast', g. Saint Petersburg, ul. Politekhnicheskaya, 29, aud. V 2.22(2) victorimop@mail.ru

Introduction

The methods of quantum physics describe light as electromagnetic radiation perceived by the human eye. As a consequence, the concept of color does not exist without an observer (human). The perception of color by a person goes beyond physics and goes into the field of physiology and psychology, as it is associated with the activity of the human brain.

There are alternative scientific approaches to the concept of the essence of color. Physicalists consider color to be a physical property, as real as any other ^[1]. An alternative direction is relational. Colors are relational or "situational" properties ^[2]. Adverbialism is a form of relativism that attributes color to perceptual processes rather than objects or the perceiving subject. The multifunctionality of color vision fits into this theory ^[3].

The functions of color are deeper than just "decoration". Thanks to color vision, a person distinguishes the shape of an object faster, determines the distance of the object from the observer faster ^[4]. The perception of color depends on the conditions of observation. The perception of color is influenced by nearby colors, lighting conditions and the shape of objects^[5].

In studies of the physiology of oculomotor activity, various aspects are considered:the presence of a repulsive effect that really attracts attention ^[6]; effective tracking of only one color at a time by an observer ^[7]; investigation of the effect of screen brightness on the fatigue of the viewer when solving problems ^[8]; research on the work of information perception in the field of para-central vision. Within the framework of such studies, an experiment revealing the difference between male and female peripheral color vision is interesting ^[9]. It has been experimentally proven that there is a deficit of attention in places inaccessible to the eye. However, visual attention is not limited to the oculomotor range ^[10].

When perceiving a real environment, a person solves problems of identifying surrounding objects by color tone and contrast with other objects. Visual perception is a function of the meaning that we associate – through our own behavior or reasonable assumptions – with the object that we see ^[11,12]. Zheleznyakov V.N. speaks about the variables of visual perception of a movie frame ^[13]. To create conditions for immersion in cinema, in addition to artistic tasks, it is necessary to reproduce the real conditions of perception of the surrounding world. This problem is solved by recreating in the frame of the film a perception gradient corresponding to the conditions of perception of the real environment. Currently, the main work with the perceptual gradient in the frame is carried out at the post-processing stage using specialized programs for digital color correction of the video image (color correction). There are color correction algorithms based on mathematics, on the algebra of quaternions ^[14]. Also, for color correction, methods of recoloring this image or video are used by comparing this image with the reference image ^[15,16]

The aesthetic appeal of the frame consists of compositional construction and emotional tension arising from its perception. Semantic information of images is a good clue for predicting emotions ^[17].

The question of the correct change in the conditions of perception of the film frame in order to give the latter a stronger emotional load, according to the director's plan, remains unresolved and requires experimental research.

1. The purpose and objectives of the study

The purpose of the study is to assess the degree of influence of the factor of the color solution of the frame on its perception by the viewer. In accordance with this goal, research tasks were formulated, a number of experiments were prepared and conducted ^{[18, 19]}. The experiments were carried out using a device that records eye motor activity (eye tracker) ^[20], which made it possible to give an objective assessment of the influence of the color solution factor on the parameters of the stimulus study template. material.

Based on the Yuriev color perception model ^[21] and the results of experiments, a model of the human visual system for image recognition was proposed (Fig.1).

Fig. 1. The work of the human visual system on the perception of visual information

Perception of visual information occurs in three stages:

1.      The first stage consists in mechanical scanning of the image with the eye.

2.      The second stage is the recognition of the image seen.

3. The third stage is the formation of the viewer's emotional response. A person, based on his experience, forms his own impression of the image he saw.

At the first two stages, the dependence of the viewer's perception of the frame on the color solution was evaluated using an eytracker. A statistically significant effect of the color solution was revealed at the stage of pattern recognition.

At the final stage of image perception, the viewer's emotional impression depends not only on the observed object, but also on the person's life experience (education, erudition), as well as on the mood and well-being of the observer. In order to neutralize the influence of indirect factors of perception of the stimulus material, depending on the observer, on the emotional evaluation of the stimulus, it is necessary to expand the circle of subjects. In such experimental conditions, the eytracker turned out to be ineffective for two reasons:

• the need to interview a large number of subjects;

• the need to use a weighting factor of the attractiveness of the frame, which is determined by the subject himself (ranking method)^[22].

The purpose of this experiment is to reveal the influence of the color scheme of the frame on the emotional reaction of the observer.

In accordance with this goal , the following tasks were formulated:

• development of an experimental installation;

• development of a software module located on a network resource and allowing to interview subjects without involving them in the laboratory;

• processing of the received data.

The first model of the developed software module turned out to be ineffective. Interest in passing disappeared after completing the task of the experiment by the subjects already on the fifth stimulus. Then the subject worked out a certain system of responses and acted in a template. To maintain interest throughout the experiment, an improved experimental model was developed, implemented through an appropriate software module ^[23].

2.      Methods of preparation of stimulus material

Frames from films and animated films were selected to compose the visual series, stimulus material. The content of the frames met a number of requirements, such as: the presence of a semantic load and the presence of two centers of interest (Fig.2). For additional stimuli, frames from films or animated films with one center of interest or with an implicit center of interest (types of nature) were used. To set up the experiment, four different color solutions of the film frame were selected in accordance with the theory of color and color contrasts according to I. Itten and a control black-and-white image (Fig. 3) ^[24].

As a result, two arrays of targeted stimuli were created: an array of photorealistic stimuli and an array of animated stimuli. Each array of targeted stimuli contains 25 stimuli in five color solutions (150 stimuli in total). In addition, two arrays of additional stimuli (photorealism and animation) were created, for stimuli with one center of interest, 50 stimuli in four color solutions (200 stimuli), and for stimuli with an indeterminate center of interest out of 25 stimuli, three color solutions (75 stimuli).

Fig. 2. Selection of a stimulus with two centers of interest. Zone No. 1 is the object that is the center of interest. Zone No. 2 is the object in the center of attention. Zone No. 3 – background

Fig. 3. Example of stimulus material (animation frame and video frame in five

color solutions)

The frames were transformed by processing in a graphic editor. The method of preparing stimuli is similar to previous experiments and is described in detail in previous articles: Experimental study of digital color correction models and their influence on the visual fixation of video frames; Investigation of the meaning of the color solution in the process of matching a movie frame ^{[18, 19]}.

3. Information data model

Figure 4 shows a block diagram of the information model of the computational experiment data. The structure of the information model is based on the methodology of a computational experiment and performs the tasks of storing arrays of stimulus material, personal data of subjects, recording test results, pre-processing the results of a computational experiment.

Fig. 4. Block diagram of the information data model

Database tables with a list of attributes for each table:

•         Questionnaire: Personal identifier (PID), full name of the observer, gender (male-female), age (number), region, education (secondary, incomplete higher education, higher bachelor's degree, higher master's degree, academic degree), type of education (technical, humanitarian) artistic training (yes, no);

•         Results: PID, test date, Test execution time (single stimulus test), FK_anketa, FK_stimul, bww (black and white) (rank 1-5), tre (triad) (rank 1-5), dop (optional) (rank 1 to 5), onw (monochrome warm) (rank from 1 to 5), onc (monochrome cold) (rank from 1 to 5), err - the number of attempts to identify one stimulus in 5 color solutions;

•         Stimuli: PID, feature (two centers of interest, one, nature-background, i.e. none), color scheme (bww, tre, dop, onw, onc), media (photo, animation, graphics, text), container, file name;

• Color scheme (reference): PID, Color scheme (bww, tre, dop, onw, onc);

• Reference education: PID, education (secondary technical, secondary humanitarian, incomplete higher, technical (bachelor's degree), incomplete higher humanitarian (bachelor's degree), higher technical (Master's degree), higher humanitarian (Master's degree), technical degree, humanitarian degree);

• Region Directory: PID, region name, part of the world.

4.      Methodology of the experiment

When setting up an experiment, the key point is to develop a task for the subjects. On the one hand, the task should not be too difficult to exclude a long preliminary training of the subject before passing the experiment. On the other hand, it is not too simple, so that during the experiment the subject does not lose interest in the task being performed.

To implement the ranking of a targeted series of stimuli, the task for the subject in the experiment was improved. Targeted stimuli (5 stimuli) are placed in a 4*4 matrix in random order (Fig. 5). The remaining fields are filled with additional stimuli selected from the corresponding arrays in random order. The screen size is considered 1600*1200 pixels. Therefore, the size of the stimuli in the matrix will be 400*300 pixels. With a 5*5 stimulus matrix, the size of the stimulus will be too small, which will make it difficult for the subjects to identify the targeted stimuli (Fig. 3). Figure 5 shows an example of the working field of the experiment, the targeted stimuli and additional ones are located in a grid. The figure also shows information about the number of centers of interest for each of the displayed stimuli in the lower left corner.

Fig. 5. Example of a field for ranking target stimuli by the test. The number of centers of interest is indicated in the lower left corner of each stimulus

When a stimulus is activated from an additional database, an error is recorded, which is entered into the experimental results table. Thus, each stimulus is assigned a weighting factor from 1 to 5 and the number of ranking errors of one row of targeted stimuli is fixed.

5.      Statistical processing of experimental results

Table 1 shows the data obtained after passing the test.

Table 1. Example of data after passing the test.

Mathematical processing of experimental data was carried out using the method of statistical processing – analysis of variance (ANOVA) ^[25]. The stages of processing the data obtained as a result of the experiment are reflected in the block diagram (Fig. 6).

Fig. 6. Algorithm of statistical processing of experimental data

At the first stage, the procedure of averaging the weighting coefficient over all homogeneous stimuli (photorealism-animation) is performed for each subject and for each color solution:

                                                                        (1)

where X(i,s,c) is a function for calculating the average values of the weighting coefficient in the corresponding color scheme and in the corresponding frame stylization for one observer; i is the observer's PID; s is stimulus stylization (photorealism or animation);c defines variable color solutions (bww, tre, dop, onw, onc); N with the identifier isck– the value of the weight coefficient for one observer for one stimulus; n– the number of stimuli in each color scheme and stylization of the frame. Since the observer in the experiment has to solve the problem on 25 main photorealistic stimuli and 25 animation stimuli, averaging occurs over 25 stimuli. The type of data is presented in Table 2. The "Mean_err" column indicates the average number of errors when passing the test for one set of targeted stimuli.

Table 2. Example of data after averaging operation.

The second stage is a comparison of samples by factors:

• color scheme;

• frame stylization factor (photorealism or animation);

• gender attribute;

• factor of the type of education (technical or humanitarian);

• the factor of artistic training (presence or absence).

The comparison of samples is carried out using single-factor analysis of variance. Risk of making a mistake of the first kind (rejection of the true null hypothesis) corresponds to the p-value. If the p-value is less than the significance level, the null hypothesis is rejected.

The significance level is the acceptable probability of statistical error, in which the differences are considered significant, but in fact they are random. If the analyzed sample has more than 100 values, then it is advisable to take the threshold of rejection of the null hypothesis equal to 0.01. So, if the p-value is less than 0.01, it would be possible to make a decision about the presence of a connection (difference) ^[26,27].

6.      Conclusions

A method of conducting a computational experiment to study the influence of the color scheme factor on the perception of a video frame by the viewer at the final stage of image perception is proposed. The developed methodology allows to cover a large number of subjects. This factor will make it possible to level out the subjective component in the assessment of the stimulus material by the subjects and make the results of the experiment more reliable.

The developed experimental technique allows us to assess the degree of influence of the color solution on the subjective assessment of the frame by the viewer. The proposed system of arrays with data allows using the developed software module located on a network resource for conducting an experiment. The questionnaire element of the subjects allows us to assess the influence of factors of gender, age and education on the perception of staff. The task of preserving the interest of the subject throughout the experiment is solved by the experimental model using additional frames.

The results of the experiment should be used in the development of textbooks on color correction, as well as in the design of control interface elements.

The methodology is programmatically implemented and tested in the local network of Peter the Great St. Petersburg Polytechnic University. It is planned to finalize the Internet survey module and conduct a global experiment with subsequent statistical processing of the results.