VR and Cultural Preservation

by Cornel Hillmann

In recent years, we have seen an interesting trend to use VR and photogrammetry for cultural preservation projects. There are now a number of nonprofits focused on specifically that area: Scanning and capturing cultural artifacts, monuments, and especially objects that are expected to decay or perish, to make them available for virtual reality headsets.

The value of such undertakings cannot be underestimated. Future generations will be thankful for snapshots of our reality and its unique objects to be experienced in 3D. We can only draw parallels to the early days of photography and film, when the pioneers of that day and age went through the troublesome procedure to capture buildings, situations, people, and historic events on photosensitive media to give researchers—generations later—a glimpse of their time and reality. Without the black-and-white photos of Louis Daguerre of Paris in 1837, we would know less about that time today.

Anything captured today for VR using photogrammetry will have a similar impact on future generations, who will see the former as time capsules from the early days of VR.

It really doesn’t matter at what point VR will take the centerstage of the mainstream media environment. The undisputed fact is that it will, for sure, be a natural part of the way people will interact with information in an XR media ecology in the not-so-distant future. While it is already being used by early adopters and is expected to become more consumer friendly over the next 5 years, it will be an undeniable part of reality for the next generation. Can we even imagine, in a hundred years, what animals will be extinct by then, which neighborhoods have been gentrified, and what historic buildings will be lost through natural disasters or replaced by skyscrapers? Photogrammetry freezes a moment in time, where people in the future can step inside and experience the past.

Volumetric video, which is still in its infancy, is even capturing moving performances with all the expressions in a 3D space. Picture the VR user in a century from now. Photogrammetry and VR is on its way to revolutionizing the heritage sector.

Pioneers in this sector have been the Smithsonian VR art museum and the Kremer museum, with a VR exhibition of  17th-century Dutch and Flemish art.

Other noteworthy organisations in this sector are the non-profits cyark.org and culturalheritageimaging.org in the US and c3dc.fr the cultural cloud in France (inspired by the French archaeologist Alexandre Lenoir, 1761-1839). The photogrammetry results are very often shared as webGL 3D object on sketchfab, and on sponsored project pages such as by Google for Cyarc.

The interesting fact is that museums and their ecosystems between conservationists and scientific institution are undergoing a similar process as manufacturing industries: adapting to digitalisation that is sometimes referred to as Industry 4.0. The pressure to adapt to technological changes, such as IoT, AI, and AR/VR brings the opportunity to reinvent and transform the processes to be more efficient and impactful in the mission to serve the public. The concept of the smart museum is that of a connected, digitized, and accessible cultural heritage experience.

The Smithsonian has shown that exhibitions can be mirrored in VR. From a VR production perspective, that means: Once the the museum’s collection has been scanned and optimized as high-quality 3D assets (digital twins in Industry 4.0 speak,) it is a routine procedure to create a VR experience with tools such as the Unreal engine to mirror the actual exhibition space and its features. The advantage of such an endeavour is that visitors can later revisit the collection. It also gives people, who otherwise are unable to attend the exhibition because of distance or physical disabilities a chance to do so. Last but not least, it allows the archive of the exhibition to be recalled at a later point in time.

We can already admire the amazing work of dedicated photogrammetry professionals and enthusiasts on sketchfab, where scanned buildings, statues, and other aspects of cultural heritage are presented in remarkable surface detail in WebGL 3D. While these are very often stunning pieces, there is still a problem of missing cultural context and historical information. For that reason, the next big step in photogrammetry and VR is storytelling and context.

The real magic comes together when a photogrammetry piece or collection is optimized for VR, brought into the Unreal engine, made walkable, and supplied with pop-up information and audio hot-spots. Anyone who has ever seen a well-done photogrammetry narration brought into VR will walk out as a believer that this is the future of cultural heritage preservation, where the actual space can be experienced and individual pieces can be examined with additional information layers.

Using open-source tools such as Blender 3D and free-to-use game engines, such as the Unreal engine, opens up incredible opportunities for cultural institutions to stay relevant, give remote access to a wider audience, and be future proof by digitizing their collections.

For artists, designers, and VR creators, it is a field in which to sharpen your skills. Let’s not forget that photogrammetry is also used for commercial applications and can be part of the creative process to create incredible game environments.

In my book, Unreal for Mobile and Standalone VR, I introduce the basic techniques to create VR experiences for standalone headsets such as the Oculus Go, the Gear VR, and the Oculus Quest by going through the production steps for asset creation, device setup, locomotion, interaction, and optimization, while considering basic UX considerations, core concepts, and the larger industry context.
Likewise, creating a VR experience for a special event or an exhibition is accessible to media designers, CG Artists, and creators without any coding knowledge. The power of visual scripting using the Blueprint system in Unreal makes the intuitive toolset accessible to creators with a design-oriented background.

This is not only true for VR exhibitions and cultural heritage apps, but also for product and design presentations in VR. The opportunities are boundless!

About the Author

Cornel Hillmann is a CG artist, virtual reality developer and entrepreneur. Cornel began his career as an art director in Los Angeles after receiving a diploma in computer graphics from Platt College, Alhambra, California. He then launched the companies CNT Media GmbH in Hamburg, Germany, and Emerging Entertainment Pte. Ltd. in Singapore. Cornel also produced the DVD lineup of cgartist.com, which originally started as an artist boot camp and is now his home base for VR- and CG-related projects. Over the last years Cornel has lectured on the subjects of character animation, design and virtual reality and often contributes to interactive media-related events as a panel member or speaker. Cornel is currently leveraging his skills and experience in CG and media design to develop VR projects.

This article was contributed by Cornel Hillmann, author of Unreal for Mobile and Standalone VR.