Experience of using 3D reconstructions of cultural heritage objects in museum practice (on the example of the Trinity Cathedral in Stavropol-on-Volga)

Lysenko Kristina Dmitrievna Master's Degree, Department of Historical Informatics, Moscow State University 119991, Russia, Moskovskaya oblast', g. Moscow, ul. Lomonosovskii Prospekt, 27 k.4 kr1stin7@yandex.ru Other publications by this author

Introduction

This work is devoted to the application of 3D modeling technologies in historical research and the creation of virtual reconstructions of cultural heritage monuments, as well as their application in museum practice. Literally every day, the problem of preserving historical and cultural heritage is becoming more acute, an increasing number of monuments are gradually being destroyed or completely lost. One such example is the Trinity Cathedral in the city of Stavropol-on-Volga (Togliatti, Samara region). On May 31, 1955, the cathedral was blown up and subsequently flooded during the construction of the Kuibyshev (since 1958 Zhigulevskaya) hydroelectric power station, as the city of Stavropol-on-Volga itself was almost completely flooded ^{[1, p.49]}.

To create a virtual reconstruction, the final appearance of the Trinity Cathedral was chosen just before its destruction. This version of the cathedral, built in 1813, was final and did not undergo significant changes until its destruction in 1955. The decision to restore this particular version of the cathedral was made on the basis of the available wide database of sources of various types: written, visual, cartographic, Internet resources, the main of which are photographs of the cathedral provided by the Togliatti Museum of Local Lore (TCM). The result of the work (virtual reconstruction of the Trinity Cathedral) is placed in the historical exposition of the Togliatti Museum of Local Lore "20th century: Stavropol – Togliatti".

The relevance of the work is due to several aspects.

Firstly, the Trinity Cathedral, built in 1738 in Stavropol, was one of the very first buildings of the city. It was the city-forming and uniting Kalmyks, accustomed to nomadism, in one place on the basis of religion ^{[2, p.13]}. The church at that time played an important role in rallying the people, especially the baptized Kalmyks, resettled in the fortress of Stavropol at the stage of its foundation in 1737, which is why its construction, as well as further restoration and the completion has always been given priority attention by the city authorities.

Secondly, for quite a long time there has been an idea of building a new cathedral in the modern city of Togliatti with the preservation of the old name. At the moment, a place has been allocated, a preliminary draft of a large-scale temple complex has been drawn up, and a memorial cross has been installed at the place where the cathedral is planned to be built. However, for various reasons, the case is not progressing ^[3].

Due to the complete loss of the building, as well as the lack of development of its virtual reconstruction at the last stage of its existence, the creation of a 3D model of the Trinity Cathedral is an urgent project that will contribute to the preservation of the common cultural heritage of the city of Togliatti, and will also probably help attract public attention and investors who can help in the restoration and preservation of this monument historical and cultural heritage.

A more detailed history of the city of Stavropol-on-Volga and the Trinity Cathedral, as well as a description of the process of developing a virtual reconstruction, are contained in a separate article by this author "Virtual reconstruction of the Trinity Cathedral in Stavropol-on-Volga in the first half of the twentieth century: sources, methods and research technologies". The purpose of this article is to describe in detail the stage of the introduction of virtual reconstruction into the museum exposition, taking into account the experience of predecessors through the analysis of foreign and domestic examples of the use of virtual reconstructions in museum practice.

Foreign and domestic experience in the use of virtual reconstructions in museum practice

Nowadays, 3D modeling technologies are being used more and more widely in historical research. At the same time, the result (visualization) of virtual reconstructions directly depends on the tasks set at the initial stage of reconstruction creation. The tasks of historical virtual reconstruction can be classified as follows:

·        Creation of guided tours, both automatic and with the possibility of self-construction of a route inside the visualization;

·        Creation of virtual tours, expositions;

·        Visualization of urban historical buildings;

·        Visualization of the urban landscape;

·        Visualization of historical and cultural objects.

In addition, these tasks can be applied both to completely lost monuments of historical and cultural heritage, and to existing ones: for example, the creation of 3D models of museum exhibits from numerous photographs and their placement in a publicly accessible format. The first option is closest to our research, when historians are engaged in virtual reconstruction of partially or completely lost monuments of historical and cultural heritage.

There are examples of successful fulfillment of these tasks in museum practice abroad and a little less in Russia. Let's get acquainted with them in more detail.

Turning to the examples of sightseeing trips, it is worth noting that basically such tasks in the application of virtual reconstructions are usually solved by posting them on the museum's website. Such expositions are not made live, which is due to the need to use expensive virtual reality or VR (Virtual Reality) technology: you need to buy glasses or virtual reality helmets in large quantities and software to organize simultaneous acquaintance with the reconstruction of many museum visitors. It is much easier to get acquainted with such a tour sitting at home, at the computer.

For example, the official website of the Louvre in Paris offers two virtual tours to choose from, thanks to which people from different parts of the world can get acquainted with the exhibits of Ancient Egypt and the history of the building from the XII century to our time ^[4].

You can appreciate the most famous chapel in the world, the Sistine Chapel, and enjoy the work of Michelangelo, right from home. This virtual tour offers two options: explore everything yourself or use the autopilot function so that the tour takes you through various points and gives comments on each part of the chapel (in English). The visualization is very high-quality, and the tour is as informative as possible ^[5].

There are similar examples in domestic museums. For example, a virtual tour gives you the opportunity to visit one of the most famous museums of Russian art in the world - the Tretyakov Gallery ^[6]. With its help, you can view outstanding works by Russian artists using Google's Street View technology.

It is necessary to mention the Pushkin State Museum of Fine Arts. On the vastness of the museum's website there is a section "Virtual Pushkin ^[7]. Here visitors can choose the exhibition they like and visit it without leaving home. The functionality of the system allows you to freely move around the exhibition halls and view paintings in a fairly close approximation.

It is also possible to walk around the territory of the Tsaritsyno Museum-Reserve ^[8], inside its palace and other buildings on the territory of the complex, without leaving home. The virtual tour provides opportunities for the researcher to independently choose the route of his walk, it is not equipped with audio or written comments.

Turning to the second type of virtual reconstructions – virtual tours and expositions, it should be noted that for the most part they represent a virtual copy of museum expositions or parts of their expositions – specific exhibitions. As a rule, art galleries fall into this category. This type of virtual reconstructions, as well as the previous one, is most often presented on museum websites, however, there are examples of the implementation of such reconstructions directly in museum halls, as part of the exposition.

In the summer of 2021, the Victoria and Albert Museum in London opened the exhibition "Curious Alice", dedicated to the origin, adaptation and reinterpretation of Lewis Carroll's classics ^[9]. Along with galleries of illustrations, films, posters, manuscripts and costumes, visitors were given the opportunity to immerse themselves in an interactive virtual environment by wearing virtual reality glasses. 

In October 2019, the Louvre in Paris launched the VR project "Mona Lisa: Behind Glass" ^[9]. Thanks to the interactive design, sound and animated images, users will learn details about the painting, such as the texture of the wooden panel and how the passage of time has changed its appearance. It was possible to try out the VR exhibition, available in five languages, within four months by booking it directly at the Louvre. At the moment, you can use the virtual tour at home by downloading it in the VR VIVEPORT, iOS and Android app store.

The National Museum of Finland in Helsinki opened a new virtual exhibition in 2018. Visitors can travel back to 1863 by studying the painting by R.V. Ekman "The Opening of the Sejm in 1863 by Alexander II". Wearing a virtual reality helmet, museum visitors find themselves inside the stage and can view the Hall of Mirrors from a three-dimensional perspective. They can even talk to the Russian emperor and other characters depicted in the painting. This is part of an exhibition detailing the life and politics of Finland in the 1860s during the Russian Empire.

Among the domestic examples, the State Tretyakov Gallery should again be singled out: on its website ^[6] you can find full-fledged virtual excursions revealing the work of Kuindzhi, Serov, Aivazovsky and other masters of Russian painting. The tour has an image zoom function that allows you to view the smallest details of the paintings and read the comments related to them.

The Hermitage is one of the ten most visited museums in the world, because the museum's collection includes more than three million works of art and monuments of world culture, from the Stone Age to our century. A large-scale virtual tour in the Hermitage ^[10] allows you to get acquainted with the map of the three floors of the building, choose a specific exhibition hall that you want to get into, or walk through all the expositions in order. Each exhibit is accompanied by comments, and many similar projects will envy the image quality.

The Victory Museum offers to view the exposition about the liberation of Europe by the Red Army. Guests are invited to "walk" through the halls of the museum using virtual reality technology. The tour "Liberation of Europe" in 360VR format is posted on the museum's YouTube channel. 360VR technology will allow visitors to "walk" around the exposition, examine photos and documents that were presented at one of the most significant exhibitions of the past year. The opening of the exhibition was timed to coincide with the 75th anniversary of the Red Army entering the USSR state border and took place on March 26, 2019.

Unfortunately, there are few examples in Russia when virtual reality technologies are presented directly in museums, and not on websites. There are virtual reality gaming clubs, but there are practically no museum examples. However, there are short-term spot exhibitions using the described technologies. For example, in June 2019, the exhibition "VR: New Laws of Art" ^[11] was held in Moscow at MAMM (Multimedia Art Museum), where the works of famous European artists were presented: Marina Abramovich, Anish Kapoor, Natalie Yurberg and Louise Clement.

* * *

Let's look at examples when virtual reconstructions are devoted to recreating historical urban development and landscape. For example, the Rome Reborn project is an international project (USA – Italy), which is a virtual reconstruction of Ancient Rome at the end of the Bronze Age. The virtual model produced in the course of this study is a digital map of the area containing 250 detailed buildings and 6,750 schematic buildings of that time, the model was subsequently imported into the Google Earth resource, so that users around the world can get acquainted with it ^{[12, p. 64]}.

Among the most famous domestic projects, it is necessary to mention the virtual reconstruction of the Moscow Passion Monastery and the adjacent historical space of Passion Square. The project initiated by the Department of Historical Informatics of the Faculty of History of Moscow State University was carried out by a team of teachers, graduate students, students, researchers, architects and restorers. The result of the project was a virtual interactive environment in which 3D models of the historical buildings of Strastnaya Square and the monastery are placed. The user can examine each building in detail and get acquainted with its history ^{[13, p. 107-115]}. The research projects of this department also include the use of virtual and augmented reality effects applied to the reconstruction of the historical development of the territory of the Holy Monastery, the White City of Moscow (with the Ivanovo Monastery). Perhaps in the near future we will see the implementation of these projects within the walls of Russian museums.

One of the most common areas of application of 3D modeling technologies in historical research is the process of virtual reconstruction of partially or completely lost monuments of historical and cultural heritage. Technologies are applied both to the restoration of the external appearance of buildings, objects, and interiors of historical premises.

For example, a vivid virtual reconstruction of the exteriors and interiors of Versailles was carried out by Google in cooperation with the museum: in the Google SketchUp program, the decoration of the palace and park complex was recreated for several time slices: 1624, 1668, 1679, 1725 and 2012 ^[14]. The source base, formed with the help of art historians and architects, as well as specialists in the interiors of the Palace of Versailles, includes an extensive complex of graphic sources: photographs, watercolors, engravings, drawings and plans. The researchers also used photogrammetry and aerial photography technologies to recreate the modern geometry of buildings and their decoration in 3D space. The results of the reconstruction are published on the project's website. The viewer has the opportunity to watch videos, virtual visits to the buildings of Versailles of different times, immersion in the recreated historical space in 360° video format.

In the UK, London's Tate Modern Gallery also adheres to the trend of using virtual reality. In 2017-2018, they created an exciting virtual exhibition. Visitors, wearing a virtual reality helmet, were able to experience a full immersion in a 3D model of the Paris studio of the artist Amedeo Modigliani ^[15]. The exhibition used the actual studio space as a template. The room itself still exists, but no longer in the form that it was during the artist's lifetime. After painstaking research, the museum created an exact copy of the artist's last studio, as it was 100 years ago.

The method of work of D.I.Zherebyatyev and the master of the Faculty of History of Moscow State University T.N. Malandina is promising – "Virtual reconstruction of the interior of the Small (Lower) office of Emperor Nicholas I in the Winter Palace in 1850-1855" ^[16]. As a result of the research, panoramas were created with a virtual reconstruction of the interior of the Small Office of Nicholas I in the Winter Palace, allowing you to view the room at 360 ° using any gadget with an accelerometer. The results of this research can be implemented in the multimedia space of the State Hermitage Museum complex and can be exhibited using augmented reality technologies and 360° video using QR codes, as well as on the YouTube platform.

As we can see, a large number of examples of the use of historical virtual reconstructions are contained in the online space: on museum websites and public hosting sites. Such projects are of great importance in order to make a modern museum more accessible. For example, they can help people with disabilities to get acquainted with exhibitions from the comfort of their own home. Now you don't have to go to another city, fly to another country to touch history and art.

A promising direction for the use of virtual reconstructions in VR and AR (augmented reality) format in museums is also emerging. Here historians and museum workers face certain difficulties: the cost of equipment that can only be used individually is quite high, which complicates access to the exhibition due to the large number of visitors. In addition, this format of expositions requires specialized software, additional personnel to ensure the functioning of the exposition in proper quality, and maintenance. The problems of accessibility of such expositions are also discussed: people with disabilities will not be able to use VR technology, as well as children under the age of 12, as well as people with medical contraindications (a tendency to dizziness). The solution to such problems is seen by some researchers in the use of AR (augmented reality). AR, unlike VR, does not create new worlds and realities, but complements what we see in the real world with new objects or data. To do this, it is enough to turn on the museum's AR application on your own smartphone and walk through the exhibition with the gadget's camera turned on. At the same time, the exhibition can be located directly on the street or in the place where the described events took place.

Nevertheless, the above examples show how virtual reality can improve the cognitive capabilities of museum expositions. With a well-thought-out implementation of 3D modeling technologies, they can really revive the collection and attract additional visitors to the museum.

Virtual reconstruction of the Trinity Cathedral in the city Museum

Returning to the topic of our research, it should be noted that the city of Stavropol-on-Volga, like the Trinity Cathedral, which was fundamental in its history, can undoubtedly be attributed to the lost monuments of historical and cultural heritage, which attracted the interest of many historians and researchers. As noted earlier, there are many studies concerning the history of the city's formation, the role of the church in the spiritual life of the city, as well as studying the impact of the construction of the Zhiguli hydroelectric power station on the further development of the city in a new location.

In addition, many researchers emphasize that it is necessary to preserve the memory of the old city for the new generation and attract modern technologies to implement this idea. For this purpose, the project of the museum complex "Heritage" in the city of Togliatti was implemented – "Local history vaccination. Walking around the city that does not exist", about which you can learn more from the article by K. Rees ^[17]. The project is the winner of the first stage of the competition of social initiatives "Chemistry of Good", held annually by PJSC "Tolyattiazot". The research is aimed at expanding knowledge about the life of the city not only for children and schoolchildren, but also for all Togliatti residents. The final product is a visual booklet and a virtual map that will be transferred to educational institutions and will help teachers of history, local history, social studies to introduce students to a city that does not exist on the map now.

A virtual map is an interactive environment in which several buildings are located according to their position set by sources (see Figure 1)^[18]. When you click on one of the buildings, a page appears containing information about the history of this building and related episodes in the life of Stavropol-on-Volga, as well as photos of this building.

Fig. 1. Virtual map of Stavropol-on-Volga. «Local history vaccination". A walk through the city that doesn't exist. [Electronic resource]

In addition, this resource has a map of the region before and after flooding: thus, it is possible to estimate the extent of the flooding of the riverbed during the construction of the Zhiguli hydroelectric power station. Among the advantages of the project, I would like to note the visibility and accessibility of the selected material and the form of its presentation, the interactive nature and innovative approach: such projects have not yet been developed in the city of Togliatti. However, there are some disadvantages: difficulties in accessing materials, since the project site does not always work correctly, in articles about buildings you want to see links to sources in order to be able to get acquainted with them, and clarifying links in articles are sometimes placed in the middle of the text, interfering with the perception of information. It is also possible to note possible points of development of the project: according to the photos posted on the portal, it is clear that the buildings have been restored using computer graphics, however, how exactly is not told, besides there is no possibility of viewing these buildings in augmented reality mode in 3D format. Perhaps in the future the project will grow into something more, for example, the developers will create the opportunity to walk through the streets of old Stavropol.

Thus, the project is evidence that the problem of historical and cultural heritage of Stavropol-on-Volga is very relevant and deserves special attention of historians and local historians.

Having considered foreign and domestic examples of the application of the 3D modeling method and the use of virtual reconstructions in museum practice, we have identified the main aspects that need to be taken into account when designing the results of the study described in more detail in our author's article "Virtual reconstruction of the Trinity Cathedral in Stavropol-on-Volga in the first half of the twentieth century: sources, methods and research technologies":

·        The results should be available to everyone who wants to get acquainted with them;

·        It is necessary to prepare several renderings in formats that meet the requirements of the museum;

·        The results of the study should be interactive with the use of modern 360° format technologies and augmented reality;

·        The results should be supported by historical information.

Preparation of research results for electronic publication and placement in the museum

Let's turn to the description of our own experience of introducing the results of virtual reconstruction into the museum exposition. After the model of the temple was created, placed in an interactive environment, and the relief and background were worked out, it was necessary to create renderers (visualizing the constructed 3D models) that meet modern realities, convert them into the necessary formats and, taking into account the experience of predecessors described in the previous section, prepare them for placement in the Tolyatti exposition the Museum of Local Lore.

In our research, we used the latest version of Twinmotion 2022.1.2: this is a new rendering engine with path tracing. Like its Unreal Engine counterpart, the path tracer is based on DXR (DirectX Ray Tracing), so it should run on current-generation GPUs from AMD and NVIDIA. All graphic work in our study was performed on an HP Omen 17-cb1xxx laptop equipped with an NVIDIA GeForce RTX 2060, 6 GB GDDR6 graphics card. The power of this device made it possible to use the functionality of the path tracer to create several high-resolution images of the cathedral. In addition, we have mounted a video "flyby" of the temple, several 360° panoramas that allow viewing the building from different viewing points, as well as high-quality 360° video.

Thus, we provided the museum visitors with the opportunity to independently examine the temple in detail, and to carry out an autopilot "flyby" around the temple and its territory. The first task was performed by panoramas, and the second was an MP4 video mounted inside Twinmotion. 360° video is a mixed type of familiarization: the camera moves along a given route, but the viewer can look around as if he himself is inside the scene.

Initially, the angles of rendering high-quality images of the temple, the points of rendering panoramas and the points of shooting video "flyby" were determined (see Figure 2) and 360° video.

Fig.2. QR code containing a video "flyby" around the temple, posted on the YouTube platform

When creating video materials, it was necessary to correctly place the main points of the "flyby": the pace should have been unhurried and smooth, without sudden jumps. This was realized by dividing the video into 8 parts, each of which had 6 shifting frames. The duration of the output file was 1 minute 20 seconds. During this time, the camera flies around the temple, allowing you to view it from all sides, as well as see the buildings on the territory of the temple, the burial site and buildings inside the fence of the cathedral.

360° video (see Figure 3) It represents several panoramas in motion: while the camera is moving, the user has the opportunity to look around 360°, wearing virtual reality glasses or using any gadget with an accelerometer. The video was recorded in 4K resolution with a frame rate of 120fps and a duration of 50 seconds, and its rendering took 29 hours of computing time.

Fig.3. QR code containing a 360° video posted on the YouTube platform

To achieve the effect of full immersion, as well as the possibility of self-study of the history of the cathedral and its appearance, we have created several panoramas. 3 indicative points were selected: on the south side of the temple (see Figure 4), from the north side (see Figure 5), as well as from the side of the bell tower (see Figure 6). Then, a stage of rendering panoramas in JPG 8K format was performed in the Twinmotion program.

Fig.4. QR code containing a panorama with a view of the southern side of the temple, posted on the YouTube platform

Fig.5. QR code containing a panorama with a view of the northern entrance to the temple, posted on the YouTube platform

Fig.6. QR code containing a panorama with a view of the temple from the bell tower, posted on the YouTube platform

The resulting videos were imported into the Adobe Premiere Pro program, after which the frame was stretched in duration to 1.5 minutes, the ambient sound was superimposed. We also saved the final file in MP4 video format, 4K resolution and H264 codec.

In order for mobile devices and tablets to play the file as a 360° video, it was necessary to add the metadata of the file. To do this, we used the Spatial Media Metadata Injector program offered by YouTube. After processing, the files were uploaded to the YouTube video hosting.

In order for users of the virtual reconstruction of the Trinity Cathedral and guides to be able to open the desired 360° video at the desired viewing point on a mobile device, it was necessary to prepare QR codes (Quick Response Code - quick response code, matrix barcode — approx. author). You can generate a QR code leading to a YouTube video using an online QR code generator ^[19]. As a result, QR codes were obtained in the form of pictures, which were placed on the information stands of the Togliatti Museum of Local Lore.

Using the gadget's camera, which supports automatic recognition of QR codes, or using a third-party application available for any gadgets, when pointing the camera at the code, the user's device opens the necessary link and launches the YouTube application, in which the 360° video is played. If there is an accelerometer in the gadget, the viewing device reacts to the rotation and also changes the viewing angle. Such a demonstration technology can be very convenient during the tour: she solves the problem of the lack of virtual reality glasses in the museum, there is no need to purchase software for her, as well as people prone to dizziness will not feel discomfort, members of the tour group will not be isolated from each other. However, viewing these panoramas is also possible with virtual reality glasses.

In addition to these visualization options, we have created several high-resolution photographs of the temple for their subsequent placement in the text of the work, at the final presentation, in the texts accompanying the exposition, and possibly in future publications.