Pedagogical testing: from the origins to the creation of modern tests

Bochkina Elena Valeryevna ORCID: 0000-0003-1099-0047 Lecturer Moscow Financial-Industrial University "Synergy" 143530, Russia, Moscow region, Dedovsk, Voykova str., 14 bochkina.elena@gmail.com Other publications by this author

Khanchas Valeriya Nikolaevna Lecturer, Department of Foreign Languages, NEWMAN CENTER 61000-61999, Israel, Tel Aviv, Tel Aviv, a-Agan Street, 34 hava69@gmail.com

Relevance

The priority direction of modernization of secondary and higher education in Russia is to ensure high quality education. To do this, it is necessary to have high-quality assessment tools. Testing is a similar tool.

Currently, the testing method is widely used not only in Russia, but also around the world. Tests are used in sociology, psychology, medicine and pedagogy both to collect preliminary information about a person and his life history, and for in-depth study of the specific characteristics of an individual's mental development. For pedagogical science, the test is a tool for evaluating the results of the learning process, with its help, the level of assimilation of educational material by a student or a student is revealed. The use of pedagogical testing allows for timely monitoring of the education system and, based on the results obtained, to build the most harmonious system of relationships between the teacher and the student, as well as to adjust the process of organizing the educational process.

The modern stage of the formation of the system of assessment and quality control of secondary and higher education is closely related to the need to use the pedagogical test as one of the strongest diagnostic tools for educational environments. This is due to the fact that thanks to the pedagogical test, it becomes possible to objectively and reliably identify characteristics that are subject to external control and assess the level of formation of these characteristics. The pedagogical test is the main tool for quality control of higher and secondary education.

The purpose of the study is to identify the chronometric sequence of the formation of the pedagogical test as a tool for quality control of education and to determine the specifics of the preparation of a modern pedagogical test.

Research methodology

The methodological basis of the study was the work:

-       Avanesova V.S., Pashnina S.V., Paskhover I. L., which include various points of view of the process of creating a pedagogical test at the current stage of the development of the science of testology.

- Ganina V.V., reflecting the peculiarities of the formation of the pedagogical test in Russia in the period from the XIX –XX century.

The results of the study

The pedagogical test is usually understood as "a tool for rapid assessment and objective measurement of learning outcomes formed during the study of educational materials" ^{[2, p. 63]} The test is a system of standardized and ordered tasks, which are arranged in increasing order of difficulty. All tasks in the test are based on specific educational content (module) and are focused on the current level of preparedness of the group of students for whom it is intended (target group). 

In order to understand the essence of the method of pedagogical testing, it is necessary to study the history of the emergence of this method. There are four points of view regarding who was the first in the field of application of the testing method. Supporters of the first point of view (V. Avanesov, K. Ingenkamp, I. Tsaturova, etc.) believe that the history of pedagogical testing began in England in 1864 with J. Fisher's book "The Book of Scales" ("A scale book") ^[15]. The book contained essay tests, spelling, reading, and grammar assignments with multiple choice answers, and methods for quantifying the data obtained. The second point of view (E. Hanford, M. Mitford, etc.) believes that the first method of pedagogical testing was applied in 1845 by the politician, reformer of American education, editor of the journal "Educational School" X. Mann. He would later be called the "Father of American Education" for his services to the country's educational system ^{[11, p. 2]}. At the initiative of H. Mann, the first testing of secondary school students in the United States was conducted. The results were published, in the same year, in the journal where he worked as an editor. Later, based on the results obtained, he developed instructions for teaching reading, which was based on the method of words "the word method"), and not on the method of memorizing the alphabet "the alphabet method of learning letters" ^{[12, p. 3]}.

The third point of view belongs to a group of scientists (K. Breim, K. Tanner, etc.) who give the right of primacy in the application of the method of pedagogical testing to the American doctor and psychologist J. M. Rice. In 1987, he conducted a table-based test to check the level of spelling knowledge. These results reflected the generalized data obtained during the testing of 33 thousand school-age children from different districts of the United States. After familiarizing secondary school teachers with the results of children, interest in testing, as a means of mass universal control of the level of assimilation of educational programs, has increased significantly. The fourth point of view reflects the opinion of scientists (V.I. Ganin, V.I. Melnikov, etc.), who believe that R. Kettel is the forefather of the testing method. He introduced the concept of "test" into the general scientific terminology dictionary in 1890. In the same year, he applied the developed test for the first time. This test was a questionnaire with questions revealing the personal characteristics of a person, the peculiarities of his thinking (plasticity and mobility) and test protocols, through the interpretation of which it became possible to measure school achievements. The first test results using M. Kettel's 16-factor questionnaire were published in 1891. Despite the lack of consensus on who is the founder of the method of pedagogical testing, we see the process of almost simultaneous emergence of this idea among a number of unrelated people, which later led to an increase in interest among teachers and scientists in the problem of quality control and knowledge acquisition in general and the method of pedagogical testing in particular. Based on this interest, the development of research activities in this field and the beginning of the second stage of its development were stimulated.

The second stage of the formation of the method of pedagogical testing began in 1909 in the USA. At that time, a national knowledge control system using tests was created. The head of the laboratory in which this control system was developed was E.L. Thorndyke. The scientist and his colleagues compiled tests covering all the main subjects of school education. Thanks to them, school teachers have the opportunity to identify "problematic" educational areas of students in a timely manner and provide the necessary assistance. At the same time, in Russia, the "Society of Experimental Teachers" was created, headed by G.I. Rossolimov and F.E. Rybakov. They developed tests that allowed psychodiagnostics of school-age children.  In 1910, F.E. Rybakov presented a psychodiagnostic test evaluating the level of development of spatial imagination in children of primary school age "Rybakov Figures". This test quickly gained recognition among Russian scientists, and later, worldwide recognition. The rest of the works of the "Society of Experimental Teachers" were not in demand by society and the state.

The third stage of the formation of the testing method began in 1915 and lasted until 1935. At this time, this method is being actively introduced into schools in the USA and Europe. The scientists developed subject-specific pedagogical tests. So, in 1916, S.A. Kortis published an article describing the results of mass testing of school-age children (500 thousand people) for knowledge of arithmetic. Similar results were obtained by scientists in other educational subjects.

At the same time, a new pedagogical school was being born in the USSR and the government encouraged the introduction of pedagogical innovations and experimentation. A new direction is emerging – pedology. In 1926, Russian scientists developed tests based on American tests aimed at determining the level of knowledge in basic school educational subjects. In 1928, the works of S.M. Vasileisky, S. Bernstein and A.N. Boltunov were published, covering the theory and technique of conducting testological research.

The fourth period began in the mid-30s and ended in the 60s of the twentieth century. At that time, testing was banned in the USSR as a pedagogical method. In the USA, there is an accumulation and rethinking of previously accumulated experience. V.A. McCall tests were divided into psychological and pedagogical.

In 1969, tests were allowed in the USSR. This was the beginning of the fifth stage of the formation of pedagogical testing. Scientists began to defend dissertations on testing issues. In 1976, V.S. Avanesova laid the foundation for the emergence of a new science – testology, which studies the features of tests, the possibilities of their application, methods and methods of processing results in her work "Issues of objectification of evaluation of learning outcomes". In the USA and Europe, the development, improvement and implementation of diagnostic test batteries continues.

The sixth stage of the development of pedagogical testing begins in 1990. At this time, a number of significant changes in the field of testing were taking place in the USSR, and then in Russia.  The first testing laboratory in Russia, Testumros, was established at the MPGI named after. Lenin, which was headed by M.M. Miroshnikova. The tasks and goals of testing have been defined:

1."Determination of the level of preparation of school-age children in the main subjects of the natural and mathematical cycle.

2. Providing targeted assistance to students to achieve the necessary basic level of training.

3. Organization of an individual approach to students and teachers.

4. The development of a test test methodology that would contribute to the professional growth of teachers' skills.

5. Development of new measuring instruments (tests) that allow students to study, teachers to teach, institutions to see the real result and plan further work.

6. Development of fundamental search and applied research in the field of diagnostics of the level of knowledge of students in various disciplines, as well as support for capable youth" ^{[3, p. 130]}.

It should be noted that in the USA specially organized testing centers appeared 4 years later. This confirms the fact that Russian scientists, despite the period of stagnation preceding these events, were one step ahead of their foreign colleagues.

In 1997, the Testumros testing laboratory and its friendly Testing Center created unified tests to obtain cross-sectional results in all general education subjects. The first (rehearsal) testing was carried out via the Internet. In the future, these tests were finalized and, after testing on more than 2 million schoolchildren, they became centralized.

Since 2001, the seventh stage of testing development has begun. This stage continues to this day. One of its distinctive features was the creation of the Unified State Exam, which replaced the previous method of testing knowledge among students of secondary schools in Russia.

The conducted brief analysis of the process of formation of pedagogical testing as a branch of science shows us not only the features of this scientific direction, but also the specifics of the development of pedagogy and psychology in Russia and abroad as a whole.

At the moment, the most common tests in our country are the Unified State Exam (Unified State Exam), the OGE (General State Exam), the GVE (State Final Exam). These tests allow us to determine the level of assimilation of the basic and general educational program of basic general education in children with normal and developmental characteristics. It is worth noting that "on the basis of the order of the Ministry of Education and Science of the Russian Federation No. 396 dated December 26, 2008 "On approval of the procedure for admission of citizens to state and municipal educational institutions of higher professional education for the 2009/2010 academic year" the Unified State Exam receives the status of an unconditional (absolute) criterion for assessing the readiness of students. The Unified State Exam retains this status to this day.

There are several basic pedagogical tests abroad. S.V. Pashnin identified several basic pedagogical test systems – "TOEFL, SAT, GRE, GMAT, which base their activities on classical test theory and modern test theory (IRT – Item Response Theory) and developed by the American specialized organization ETS – Educational Testing Service" ^{[7, p. 34]} The contribution of this organization to the theory and practice of pedagogical testing cannot be estimated, because their developments are used all over the world.

The technology of pedagogical testing is one of the most common ways to assess learning outcomes in preschool, school-age children and adults. In addition to widespread tests, there are a number of proprietary testing methods. These tests can be divided into several categories: electronic, hardware, subject, practical and blank tests. Both categories of tests are aimed at identifying the features of learning educational material. Both categories of tests are divided into:

· introductory or introductory;

· current;

· thematic;

· intermediate;

· final results.

The above groups of tests allow you to monitor and evaluate knowledge at all stages of the educational process.  It is worth noting that modern pedagogical tests make it possible not only to assess knowledge, but also the level of organization of educational interaction between teachers and students. These tests are based on the principle of a questionnaire with multiple answers, the analysis of the results of which allows us to identify not only the cause of good or bad relationships between participants in the educational process, but also the specifics of knowledge transfer, and as a result, the causes of high academic performance or underachievement of students.

Several components are responsible for the adequacy of the results obtained during pedagogical testing: compliance with the conditions of the test and the quality of the test tools.

The analysis of the literature and our own research experience allowed us to determine the quality criteria of the pedagogical test, which, first of all, are based on compliance with the stages of its development and the passage of the standardization and verification process. In order for the tests used in the work of a teacher to be of high quality, it is necessary, in the process of their creation, to clearly follow the stages highlighted by scientists (V.S., Avanesov, D.Sh. Odinaev, I.L. Paskhover, N.A. Senognoeva, T.M. Fomenko, M.B. Chelyshkova):

1. On the basis of the Federal State Educational Standard for the relevant discipline, the scope of the test content and the purpose of testing are determined.

2. A test plan or specification is being drawn up.

3. Definition of evaluation criteria.

4. Select the type of test task.

5. Formulation of the main issues.

6. Development of answers to the main questions.

7. Conducting task analysis.

8. Measuring the difficulty of tasks.

9. Preliminary assessment of the effectiveness of the test.

10. Conducting an ethical and methodological examination of the content of the test.

11. Correcting assignments in accordance with expert comments.

12. Writing the conclusion and the current instructions.

13. Performance assessment based on statistical processing of the received data.

14. Retesting.

15. Standardization and validation of the test on a large sample of respondents (1000 people).

16. Test implementation.

Let's comment on the previously highlighted stages. The primary task of the compilers of the pedagogical test is to determine the purpose of testing. This can be a test of intermediate or final knowledge, determining the level of understanding or knowledge in a comfortable or stressful situation, etc. Next, it is necessary to draw up a plan of questions and typologize them in accordance with certain parameters under study, for example, in accordance with the section of the discipline being studied. It is also necessary to specify the target group of subjects and for whom the results of the study are intended. Next, it is necessary to determine the type of test – introductory, final, electronic or blank, etc.

The next step is to define the evaluation criteria. The authors of the pedagogical test compile a table in which the correct answers to the test questions are recorded and an assessment system is compiled on a point scale. Next, appropriate methods of mathematical and statistical data processing are selected. Usually, the "phi - correlation coefficient and kappa-coefficient of consistency of test results" are used ^{[6, p. 179]}. The results obtained allow us to assess the adequacy of the method used and avoid falsification of the results in the future. It should be noted that the success of testing directly depends on the quality of the selection of the content of test tasks, which is the next stage in the preparation of the test. V.S. Avanesov highlighted the principles of selection of the content of test tasks:

1. " The principle of congruence is the correspondence of the content of the pedagogical test to the content of the academic discipline. The test tasks should cover all important aspects of the content area in the correct proportion.

2.   The principle of significance reflects the need to include in the tests only the most important, basic knowledge expressing the essence, content, laws and patterns of the phenomena under consideration.

3. The principle of scientific reliability – all controversial points of view that are acceptable in a scientific dispute should be excluded from the test tasks.

4. The principle of matching the content of the test to the level of the current state of scientific knowledge – it is necessary to constantly improve the test by adding new test tasks to the bank.

5. The principle of consistency, complexity and balance of the test content is the selection of tasks that comprehensively reflect the main topics of the discipline.

6.      The principle of increasing difficulty of test tasks is "each learning element has some average measure of difficulty, which must be taken into account in the process of knowledge control" ^{[16, p. 210]}.

7.      The principle of the relationship between content and form – not every task content can be expressed in a test form. Also, the content of the test must meet logical and semantic requirements and norms" ^{[1, 3, 4,8]}.

In the process of drawing up tasks, the generally accepted requirements for them should be taken into account, which are as follows:

- the text of the tasks should exclude ambiguity and ambiguity of the wording;

- the main text of the assignment should consist of no more than 8 words;

- the task has a simple syntactic structure;

- all answers must have the same sentence length (constructions);

- the test should not contain tasks involving verbal association;

- there should be no repetitive words in the answers;

- there should be no irrelevant material on the research issue in the questions;

- it is necessary to exclude from the answers options that imply combining or excluding all previous answers: "All", "None", "One of the listed", etc.;

- the answer to one task should not be a clue to other tasks;

- the test tasks should exclude the possibility of making value judgments of the test participant, if this is not an end in itself of the test task;

- it is necessary to ensure the grammatical coordination of all test elements;

- if all the subjects correctly answer the test task, then this task must be excluded from the general list of questions;

- if all the subjects incorrectly answer the test task, then the educational program on the basis of which this test was compiled should be analyzed to ensure that the question corresponds to the material being studied.

Choosing the correct content of the test task contributes to the accuracy of checking the necessary knowledge and ensures the quality of test scores. It is also necessary to measure the difficulty and expediency of tasks in order for the test to act as a single research tool and identify the level of relevant knowledge. To measure the difficulty and expediency of a test assignment, it is compared with the curriculum being studied, and then a pilot study on a small sample is carried out. The pilot study allows you to find out the level of understanding of test tasks by children.

The next stage is the ethical and methodological examination of the test. This examination is carried out by an ethics committee or a laboratory (department) in the organization where the pedagogical test is being developed. It identifies the compliance of the test with ethical standards and verifies the reliability of the results. After receiving the conclusion of the ethical and methodological commission, either the tasks are corrected in accordance with the comments of experts and the test is submitted for re-discussion, or a mass approbation is carried out with subsequent validation and standardization of the test on a sample of at least 1000 people. After carrying out all the procedures, upon receipt of positive results, the pedagogical test is introduced into educational institutions.

Conclusion

This paper presents a theoretical analysis of approaches to the study of pedagogical tests, which are predetermined by foreign and domestic schools. Foreign researchers (USA) were the forefathers of the modern science of testology. Based on their work, most domestic and foreign tests were created throughout the 20th century. The unconditional merit of domestic scientists is considered to be the creation of the Testumros testing laboratory, thanks to which the first domestic tests for assessing the level of assimilation of the educational program were created – the first versions of the Unified State Exam, which were subsequently finalized and implemented throughout the country.

At the present stage of the development of testology, Russian scientists have identified the basic principles and requirements for the development of a pedagogical test, which are the basis for all pedagogical tests in our country. Teachers and psychologists have noted that following these requirements and principles makes it possible to achieve a high level of adequacy of the method used and avoid falsification of results in the future.