Database in the Study of the History of Estates in the Moscow Region: Filling, Enrichment and Analytics

Trishin Ivan Master of History, Historical Information Science Department, Moscow State University  301570, Russia, Tul'skaya oblast', pos. Volovo, ul. Aleksandrova, 4A, kv. 2 trishin_ivan@rambler.ru Other publications by this author

Modern scientific research is increasingly carried out at the intersection of disciplines, somewhat blurring the boundaries of the tools of scientific activity. Methods and technologies developed in natural science areas can be used in humanitarian research, as well as vice versa. The accumulation of knowledge and the formation of methods in different fields of research can help a completely different field of knowledge, and therefore the researcher's toolkit is expanding today due to the comprehensive development of scientific methodology and its particular examples.

The development of high technologies has allowed historical science to use large amounts of information in obtaining new knowledge. The concept of a mass historical source introduced into scientific circulation by I.D. Kovalchenko makes it possible to cover significant volumes of historical documents that were not fully studied by researchers in previous periods. The computing power of computer technology already in the 60-70s of the last century made it possible to build large information systems, working with which not only reduced the time spent on preparing for research, but also created the possibility of conducting such research, since previously manual processing of tens of thousands of forms and cards did not bring practical benefits.

Statistical studies in historical science were followed by works based on databases. The popularity of such databases in the eighties and nineties of the last century gave a new impetus to the study of mass sources, information from which was now completely entered into tabular databases and could be repeatedly used in further scientific research. A number of research works are still using databases to obtain new knowledge, the historiography of this area has many publications covering various variants of research interest^[1][2].

* * *

In studies devoted to 3D reconstructions of historical and cultural heritage monuments, a special role is played by the process of collecting and structuring data necessary for carrying out reconstructions directly. Nevertheless, the continuation of the development of the chosen direction forces the author of the work to plunge into large-volume materials again, which, as a rule, remains a "dead weight" in files and folders. Nevertheless, the availability of an information system describing the researcher's work in the historical field of activity can greatly facilitate work on the chosen topic, as well as provide additional results that can complement the main conclusions of the study.

Attempts to restore manor complexes were repeatedly made during the Soviet period of their history, most often the work was limited to fixing the current state of manor complexes, less often some restoration work was carried out. Attempts to restore such complexes most often remained in the form of plans and architectural projects, the transition to the practical restoration of monuments was not completed. In the post-Soviet period, attempts were also made to restore manor complexes with the active assistance of patrons, but most of these attempts were unsuccessful. However, interest in manor history grew, a new impetus in the development of the study of manor history gave the opportunity to build virtual reconstructions and visualization of such reconstructions in a software environment. Since 2019, the Moscow State University Faculty of History has been working on a joint project with the Central State Archive of the Moscow Region^[3] dedicated to virtual reconstructions of lost estates in the Moscow region (in which the author of this article is directly involved). Thanks to the efforts of historians and archivists, today there are virtual reconstructions of such suburban estates as Pushchino-on-Nare^[4], Petrovskoye-Alabino^[5], Olgovo^[6], Molodi^[7] and Nikolskoye-Uryupino^[8][9], as well as unpublished reconstruction of the Mikhailovskoye estate and a number of other works that are defended at the Department of Historical Informatics of Lomonosov Moscow State University.

Russian Russian Homestead Research Society (OIRU), a leading organization in the study of Russian manor culture and history, has studied the history of manor complexes in the Moscow region. The most active in the study was the reference book "Moscow estates"^[10], created by the non-profit partnership "Russian Estate" with the participation of members of the Society. This reference book made it possible to systematize knowledge about the lost complexes and identify the most promising objects for reconstruction. In the process of studying the reference book, the author decided to form a structured information system that would allow identifying the most interesting objects for reconstruction, as well as distributing preserved and lost estates in geographical and chronological terms. Information about the owners of these estates was also entered into the database, where the tables were linked by the identifiers of the estate complex. Also, the studied estates were linked to electronic copies of sources, on the basis of which an electronic reference book was formed, which was discussed in the article "The use of data management technologies in the creation of historical Internet resources" by the author of this study^[11]. Previously, the mechanisms for manually loading data into the database through the interface were described, the second stage of work on the information system involved filling in tables with information contained in the published reference book, as well as enriching this information with data from open sources. Some results of this work are summarized below, including statistical generalizations on the estate complexes of the Moscow region, and the process of enriching the available data is described^[12].

* * *

The data storage and processing system was built as follows: the visual interface of the directory was developed on the Wordpress website builder, the database itself was built in the MySQL database management system, and automatic data transfer was carried out using the Apache NiFi tool (all of these tools are free and freely distributed on the Internet). In addition to these tools, SQLite DBMS (a database file system that allows storing all tables and relationships in one physical movable document), as well as the Jupyter Notebook development environment, which allows data transformations using the Python programming language, were also used. Initially, it was planned to divide the database into four main tables: estates, personalities, sources and objects (models). This concept was slightly changed in the course of work: a table of reconstructions was added to the table of objects. The rest of the structure remained unchanged. The table of manor complexes included basic information on estates: identifier (number of the estate in the directory), name, district or city district of the Moscow region, the century of foundation, quarter of the century in which the estate was founded, the year of foundation, the identifier of the degree of preservation of the estate (author's expert assessment on a scale from 0 to 5), flag preservation of the main house of the estate, as well as comments. These signs are found in almost all directory entries. In total, the table received 629 entries according to the number of homesteads recorded in the directory. It is worth dwelling a little more on the classification of the preservation of estates, which was given among the signs.

Almost every entry in the directory contains notes about which objects have been preserved and have survived to the present day, as well as their condition. This information allowed us to identify six main levels of classification of the preservation of manor complexes (Table 1).

Table 1. The degree of preservation of manor complexes (according to the classification of the author of the work)Degree of preservation

This classification was supplemented by the flag of the preservation of the main house of the estate, as a system-forming object. The analysis of the safety of 629 complexes led to the following results:

• ·        49 estate complexes (7.8%) are in excellent condition. Among them are such estates as Arkhangelsk, Abramtsevo, Talitsy, etc.
• · 11% (70) of estate complexes have the fourth level of preservation (Pehra-Yakovlevskoye, Ogarkovo, Ashitkovo, etc.)
• · 400 estates (63.5%) have a degree of preservation of 0 and 1. Most often there are estates in which only the temple has been preserved (see Table 1).
• · In 26% of cases (163 estates), the main house has been preserved in one form or another.

Next , we will give some statistics on the periodization of the foundation of estates in the Moscow region:

• ·        The largest number of estates (165) was founded in the second half of the XVIII century.
• ·        For 35 complexes, the time of foundation is not specified.
• · The earliest examples date back to the beginning of the XVI century (boyar patrimony), but only 3 estates were found until the last quarter of this century.
• · 16 estates were founded at the beginning of the XX century . Most often, these objects were owned by industrialists and foreign craftsmen invited to work in Moscow or the Moscow region. No large complexes were found among them.
• · 92 records contain the exact (or approximate) year of foundation of the estate.

Also, in some cases, the reasons for the loss of the manor complex are indicated, as well as approximate dates of destruction. Among the reasons most often there are fires and demolition. The following date ranges are most common in this regard: 1918-30. (the period of post-revolutionary destruction of estates), 1941 (many estates suffered from German bombing), as well as 1990-2000. Nevertheless, restoration activities were carried out on a number of complexes (mainly in the 1970s and 1980s, in most cases we are talking about groups of restorers, local historians, specialists in architectural restoration workshops), the results of which were architectural measurements and restoration projects containing drawings of the current state of the estate at the time of the project.

From the point of view of geographical location, the largest number of estates is located on the territory of Odintsovo city district (42), followed by Solnechnogorsk and Dmitrov districts (38 and 34, respectively). The smallest number of complexes (2) is located in the territories of the Pavlovo-Posadsky district, as well as the urban districts of Orekhovo-Zuyevo and Yegoryevsk.

Other classifications of manor complexes require the involvement of additional materials and are most often hindered by the historical context of the development of these complexes ("noble" and "merchant" estates intersect at the moment of the sale of the estate by the noble family into the hands of successful entrepreneurs and merchants, and architectural classification is impossible due to the loss of more complexes as a whole).

An important aspect when considering the issue of a possible reliable reconstruction of the estate is the security of the object with a source base. For the scientific reconstruction of the complex, only those backed up by a wide range of information collected in archives, restoration workshops and private collections are selected. The available information on such estates as Nikolskoye-Uryupino and Yaropoletskaya Chernyshev estate (objects of research by the author of this article working on virtual reconstructions of these estates) has been added to the main table of manor complexes and supplemented with data obtained from archives or open sources. Classification by the identifier of the estate and archival business (or if it is not present – by other signs of a particular publication) allows you to quickly find a place to store digital copies of sources on a local computer, as well as find the right sheet of archival business by keywords. At the moment, more than 500 documents are described in the source table. Similarly, tables of virtual reconstructions of estates were made: data on their storage location (both individual models and projects executed in the Unreal Engine virtual environment) were entered into the fields, as well as descriptions were given to simplify keyword search. A different situation is observed with the owners of manor complexes. Identifying information about them required enriching the data with information from open sources.

***

The published directory contains an alphabetical index of the owners of estates, which indicates the number of the estate, its name and surname and initials (and sometimes only the surname) of the owner. Processing this list manually (more than three thousand entries) would take a significant amount of time, so it was decided to use the TurboScan mobile application, which allows you to digitize images and recognize text. Thus, the list of owners was transferred to CSV format (spreadsheet), but before adding such a table to the database, there was a need for additional processing.

The main problem was the records of the dynasties of the owners, for example: "Yankov A.H., N.A., H.N.". In this case, three people are listed in one record at once. In order to get a list that is correct for work, it became necessary to split such records into several new lines. In this case, the Jupyter Notebook development environment (an interface for working with Python), as well as a number of open Python libraries, helped the work. With the help of the regular expression library^[13],[14], the entire list of surnames was processed (Fig. 1) and a new table of estate owners was formed, in which the list grew from 2492 lines to 3140 (648 names were added). In the new temporary table of owners, the names of the owners were taken out separately for further data enrichment (Fig. 2).

Fig. 1. Code snippet for processing records with multiple names (screenshot of the author of the work)Fig. 2. Fragment of the updated owners table.

From number 10, the result of regular expressions is visible (screenshot of the author of the work)A separate list of surnames is important when enriching information with open sources.

For the current work, the Internet resource "Rodovod" was selected (Rodovod: Multilingual family tree [website] URL: https://ru.rodovid.org/wk/Multilingual genealogical tree), which contains the most convenient genealogical resource for processing. To manually obtain information about 3140 personalities, a huge amount of time and effort is needed, and therefore it was decided to automate this process using web scraping technology (15 top web scraping solutions of 2021 / Habr [website] URL: https://habr.com/ru/post/543760 ), which involves automatic crawling of web pages in order to obtain information.

Information about each genus on the specified information resource is available at the link "https://ru.rodovid.org/wkРод:***", where instead of asterisks, the plural surname of interest to the researcher is indicated (for example, Orlov – Orlov, Adlerberg – Adlerbergs, etc.). Further on the page there is a section of representatives of this surname, where basic information about personalities is available (full name, gender, year of birth, year of death, link to an article about the personality). Such a list is useful in current work when comparing surnames and initials with the years of birth and death (in reference books they are sometimes indicated in information about manor complexes). In order to get a unique list of surnames, all the entries received in the "surname" column were translated into the plural using the same regular expressions. To do this, a dictionary of the most frequent singular and plural endings (ov – ovs, ev – evs, berg – bergs, hiv – vichi, etc.) was compiled, after which all entries were checked using this dictionary, a list of the families of the owners of estates (about 1600 unique entries) was obtained. This list was automatically checked in the Rodovod electronic directory (in compliance with the scraping conditions specified in the site file "robots.txt "), and the data obtained were collected in a common table. Thus, 105,000 records of people belonging to these genera were obtained (Fig. 3). Using regular expressions, the full names of people were converted into surnames with initials, which can then be compared with the original published reference book.

Fig. 3. The table of the received data from the resource "Rodovod" (screenshot of the author of the work)For the test combination of information, the first 100 estate complexes were checked according to the list.

According to them, 553 names were mentioned, among which 203 contained at least one date (birth or death). According to these records, 77 people with matching dates and initials were compared, which is approximately 38% of the total number of records containing dates. Manual analysis allowed us to find 8 more people who supplemented the result to 41% of the owners found. While maintaining the ratio of the volumes of information available in the directory, there should be about 500 people on the full list, which in conditions of a limited amount of information should be considered a good working result. At the moment, the work on finding the owners of estates in the open database continues.

After the information about the owners was added to the general list, it was possible to form links between the tables and the format for uploading data to the electronic environment according to the algorithm described earlier in the article "Using data management technologies when creating historical Internet resources"^[11]. Thus, it was possible to automatically fill in the electronic directory without significant manual work.

***

Summing up, I would like to once again emphasize the complexity of the task of accumulating and systematizing data in the study of the history of manor complexes. The problem of fragmentation and haphazard storage of information forces the researcher to look for solutions in modern ways of structuring and generalizing data. To solve this problem, the author of the above study created a software system based on the published reference book, in which the researcher can search for the necessary information about estates, automate the loading and enrichment of data, thus performing a chain of actions from the primary search for information to working with the source. Within the framework of the proposed system, work with the information complex from the side of software solutions is demonstrated, the process of its enrichment by external sources is shown. Thus, the possibility of analytical work outside of the designated directory is expanded not only due to additional information, but also due to the software capabilities of the systems described above.