Experience in analyzing a student's digital footprint in LMS Moodle.

Gushchin Aleksandr Nikolaevich ORCID: 0000-0002-3466-4038 PhD in Physics and Mathematics Associate Professor, Department of Urban Planning and Landscape Architecture, Ural State Academy of Architecture and Arts Karl Liebknecht str., 23, Sverdlovsk region, Yekaterinburg, 620075, Russia alexanderNG@yandex.ru Other publications by this author

Introduction

The transition to e-learning and the use of distance learning technologies has not only set the task of learning in a new way, but also set the task of analyzing educational results in a new way, introducing the concept of the digital footprint of the learner. The availability of new opportunities allowed us to set new tasks. So A. A. Stepanenko, A.V. Feshchenko argue that the analysis of the digital footprint makes it possible to identify "strong students", "weak" and students who have shown a high level of intellectual development and a high level of personal motivation ^[14]. T.M.Shamsutdinova uses the digital footprint as a means of creating and managing an individual learning trajectory ^[17]. M. S. Vorobyova and D. S. Lobuntsov use the digital footprint to design the student's educational results ^[6].

In this article, the author will focus on the results of analyzing the digital footprint of trainees formed during the study of a particular discipline. The need to narrow the problem down to the level of a specific discipline is due to both the lack of the necessary integrated amount of interdisciplinary data and the practice of such analysis. As a specific discipline, we will use a course on project management, taught for undergraduates of the entire stream of the specialty "Architecture", direction 07.03.04 "Urban Planning" ^[16]. Nevertheless, it will be shown that the results obtained allow for broad generalizations. In particular, they can be used to check the compliance of the competencies formed in the course with the requirements of professional and state educational standards. In the educational process, the formation of competencies is traditionally carried out "from above": the curriculum is written, the competencies that the curriculum forms, the fund of evaluation tools for assessing the formation of competencies. Then everything goes "down": to the level of a specific discipline.

Taken as an example, the discipline "Project Management" naturally implies the formation of managerial competencies. The need to study how the competencies of a manager are formed is felt more and more. Thus, the qualification directory of the characteristics of positions ^[11] defines the general requirements for managers of the appropriate profile. And the professional standard of the architect ^[10] specifies the requirements in the form of a generalized labor function "Management of the process of architectural and construction design of objects and works related to the implementation of capital construction projects". In the recently released Standard of Integrated Development of Territories ^[13], consisting of 10 books, 2 books are devoted to project management, which indicates the importance of management activities. Theoretically, the activity of an architect as a manager and project manager is comprehended in V.Etenko's monograph ^[18].

The course "Project Management" is prepared in electronic form. Students can take it either completely remotely or in a mixed learning mode. "Blended learning is a model built on the basis of integration and mutual complementation of traditional and e–learning technologies, involving the replacement of part of traditional training sessions with various types of educational interaction in an electronic environment" ^[4]. It is thanks to the fact that the course is prepared in electronic form that it has become possible to collect and process feedback from listeners – to analyze the digital footprint.

Methods

The composition of the digital footprint. "A digital footprint is data about a person's educational, professional or other activities presented in electronic form" ^[15].  The student's digital footprint during the course consists of two parts: mandatory and additional. The mandatory part includes the materials of all tasks performed in the course, the additional part includes any materials not directly required by the course didactics. And if, on the basis of mandatory materials, the teacher gives an overall grade for the course, then additional (optional) materials are not used and they are not sought to be introduced into the course.

In the case of the "Project Management" course, additional materials include feedback from students about the course and a questionnaire. Feedback from students was collected after the end of the course in a free selection mode to avoid bias in grading. During the completion of the course, students voluntarily write a general review of the course as a whole, and also necessarily fill out a questionnaire with four questions: "Acquired skills", "Acquired experience", "Useful in the future", "Should be avoided in the future". The composition of the questions is taken from the recommendations for the post-project analysis of MSF methodology projects from Microsoft ^[2]. The questions from Microsoft were chosen on the basis that, firstly, project management is a universal discipline, and secondly, the experience of this company in the field of project activities is undeniable. In fact, the questions presented are universal and suitable for any course.

Student questionnaires are also widely used: as a tool for self-assessment of the university ^[8], to assess student satisfaction ^[5], to optimize the educational process ^[9], to optimize the independent work of students ^[3]. Further, the following methods were used to analyze the collected questionnaires: content analysis of the emotional coloring of texts by S.Titcher, M.Mayer, R.Vodak, E.Vetter ^[12], frequency analysis, semantic analysis of texts by the method of "Origins-results-solutions" ^[7].

Results

Review analysis

At first, the general emotional coloring of the review texts was determined. Total number of words with negative connotation: 10%. Total number of words with positive connotation: 17%. The total number of nouns, verbs and adjectives is 33%, i.e. the reviews are balanced and confirms the unbiased nature of the assessments given by the listeners.

Next, semantic content analysis was used to analyze the feedback of listeners about the course. Since many students mentioned the e-course in their reviews as an innovative form of learning, a content analysis of the part of the reviews that concerned e-learning was carried out. Content analysis was performed by highlighting common semantic units. A total of 24 questionnaires were processed. The majority of students – 18 people – positively assess distance learning technologies. Figure 1 shows the advantages that students see in this form of training.

As you can see, for the vast majority of listeners who positively perceive remote technologies, the most attractive feature is the ability to manage their time. Which is undoubtedly one of the most important managerial competencies. The next most attractive feature is the ability to view the work of classmates, which is due to the shortcomings of the full-time design process based on individual consultations with the teacher. Here we must pay tribute to the developers of the learning management system (LMS) Moodle, who based their development on the principles of social constructionism ^[1]. Of the remaining sample, three listeners did not express their opinion, four expressed a negative attitude to remote technologies. There are two main reasons for the negative attitude: the lack of live communication, including with the teacher, the lack of practical skills that are acquired during training.

Next, a frequency analysis of the reviews was performed. The results of the frequency analysis are shown in Table 1.

Table 1. Results of frequency analysis (excluding auxiliary words and prepositions)

Next, the emotional context of the attitude to the word "course" was determined. To do this, all sentences with this word were written out and the epithets that accompanied this word were determined. At the same time, reviews with different degrees of comparison fell into the same category. For example, "liked", "really liked" and "liked in general". The category of other things includes proposals that do not contain judgments about the course itself, for example, "I also liked that the course was held simultaneously with the project that we are currently implementing."

 The next most frequently used word "training" part is used in combination with "form of training", "format of training". In general, the attitude to the forms of learning correlates with the content shown in Figure 1, i.e. the attitude is generally favorable.

Questionnaire analysis

            Next, we present the results of the analysis of questionnaires from listeners. Figure 3 shows summary data on the results of processing the questionnaire with the question "Acquired skills".

From the diagram presented, it can be seen that the answers of the listeners about the acquired skills are more diverse – there are 9 different types of answers in total. The largest number of votes was scored by the answer about personal skills, which included formulations such as: rationalism, pragmatism, skills to express your thoughts competently, etc.

            Next, we will analyze the question of the project analysis questionnaire "Acquired experience". Summary results of the questionnaire analysis are shown in Figure 4.

Figure 3 shows that the variety of responses about the acquired experience is less – 6 semantic units are allocated in total. The main experience gained is the experience of organizing work. The next most important acquired experience is the experience of planning.

             The next question for analysis is the students' answers to the question "Will it be useful in the future". Figure 5 shows the summary results of students' responses to the questionnaire question. The students' view of what will be useful to them in the future is even more unanimous: over 50% noted the experience of leadership as something that will be useful to them in the future. The rest of the responses are expressed much less intensely.

The last point of the analysis is the analysis of the answers to the questionnaire question "Should be avoided in the future". Figure 6 shows the results of the answers to the questionnaire question.

The answer about procrastination gained the most weight (the author's generalized formulation), the next most important answer is the answer about the need to avoid hasty decisions. The answers are given the following meaningful interpretation: the answers about the need to avoid proctination and hasty decisions correspond perfectly with the answers to questions about acquired skills and emphasize the understanding acquired by the listeners of the need for timely management decisions. The answer that it is necessary to avoid problems related to the installation of programs is also interesting. It is connected with the fact that during the training, students were offered to download a free demo version of the MS Project program from the original website. Apparently, during the execution of this task there were difficulties associated with a low culture of using software.

Discussion

            For the convenience of discussion, the summary results of the students' answers are given in Table 1. The answers are grouped by topic.

Table 1. Summary data on student responses 

Let's consider how the above answers of students are related to the requirements of professional standards. Professional standard 10.008 "Architect" ^[17] includes in the list of generalized labor functions the labor function "C" - "Management of the process of architectural and construction design of objects and works related to the implementation of capital construction projects". Further, the standard lists the types of work that are necessary to perform a generalized labor function: management of design and survey work, management of design work on the development of project documentation, preparation and protection of project documentation, implementation of author supervision measures, provision of consulting services, administration of project management processes, expert activities on the development of the architectural profession, employee management and operational personnel management of the creative team. As you can see, the requirements of the professional standard specify the types of work necessary to perform a generalized labor function, while the students' answers describe general professional knowledge and skills that could be useful in any kind of activity.

Earlier it was said that the digital footprint of a student can be used to check the formation of competencies required by the curriculum. For the course under consideration, the curriculum defines the following competencies: CC-3 – "Able to carry out social interaction and realize his role in the team" and the competence of CC-6 "Able to manage his time, build and implement a trajectory of self-development based on the principles of education throughout life" ^[16]. The very wording of the answers to the questionnaire questions indicates the desire of listeners to receive not specific competencies related to the management of a specific process, but higher–level competencies - interdisciplinary. For example, according to Figure 3, it can be seen that the listeners put the acquisition of personal skills in the first place, by which they understand rationalism, pragmatism, skills to express their thoughts competently, etc. At the same time, the importance of both specific experience and acquired skills is realized. Table 1 shows that students note the skills of leadership and organization of teamwork, which is directly related to the competence of the CC-3 – "Able to carry out social interaction and realize their role in the team." The formation of the following competence provided by the curriculum of the CC-6 "Is able to manage his time, build and implement a trajectory of self-development based on the principles of education throughout life" is confirmed both directly by Table 1, where acquired time management skills are noted, and indirectly, when students note the skills of strategic thinking.

Let's return to the first component of the digital footprint: reviews of the course as a whole. Earlier it was said that the feedback analysis was carried out in terms of the attitude of listeners to the electronic form of training. In general, the reviews were complementary. Comparing these conclusions with the analysis of competencies, we conclude that the electronic form of training does not prevent the acquisition of certain managerial competencies.   

Conclusion

            The article considered the analysis of the digital footprint on the example of a specific discipline. It was stated that despite the narrowing of the specifics of the problem to the level of a separate discipline, the conclusions made allow for broad generalizations. The materials of the course "Project Management" were used as a data source.

As it was shown, the analysis of the digital footprint allows us to judge the emotional impressions that the listeners received. The analysis of the digital footprint allows us to conclude about the formation of certain competencies that are provided for in the curriculum. Which, in turn, is the next step towards the creation of evidence-based pedagogy.