Originally published on the Apress Blog, 06/05/19.
In
recent years, we have seen an interesting trend to use VR and
photogrammetry for cultural preservation projects. There are now a
number of nonprofits focused on specifically that area: Scanning and
capturing cultural artifacts, monuments, and especially objects that are
expected to decay or perish, to make them available for virtual reality
headsets.
The value of such undertakings cannot be
underestimated. Future generations will be thankful for snapshots of our
reality and its unique objects to be experienced in 3D. We can only
draw parallels to the early days of photography and film, when the
pioneers of that day and age went through the troublesome procedure to
capture buildings, situations, people, and historic events on
photosensitive media to give researchers—generations later—a glimpse of
their time and reality. Without the black-and-white photos of Louis
Daguerre of Paris in 1837, we would know less about that time today.
Anything
captured today for VR using photogrammetry will have a similar impact
on future generations, who will see the former as time capsules from the
early days of VR.
It really doesn’t matter at what point VR will
take the centerstage of the mainstream media environment. The
undisputed fact is that it will, for sure, be a natural part of the way
people will interact with information in an XR media ecology in the
not-so-distant future. While it is already being used by early adopters
and is expected to become more consumer friendly over the next 5 years,
it will be an undeniable part of reality for the next generation. Can we
even imagine, in a hundred years, what animals will be extinct by then,
which neighborhoods have been gentrified, and what historic buildings
will be lost through natural disasters or replaced by
skyscrapers? Photogrammetry freezes a moment in time, where people in
the future can step inside and experience the past.
Volumetric
video, which is still in its infancy, is even capturing moving
performances with all the expressions in a 3D space. Picture the VR user
in a century from now. Photogrammetry and VR is on its way to
revolutionizing the heritage sector.
Pioneers in this sector have
been the Smithsonian VR art museum and the Kremer museum, with a VR
exhibition of 17th-century Dutch and Flemish art.
Other
noteworthy organisations in this sector are the non-profits cyark.org
and culturalheritageimaging.org in the US and c3dc.fr the cultural cloud
in France (inspired by the French archaeologist Alexandre Lenoir,
1761-1839). The photogrammetry results are very often shared as webGL 3D
object on sketchfab, and on sponsored project pages such as by Google for Cyarc.
The
interesting fact is that museums and their ecosystems between
conservationists and scientific institution are undergoing a similar
process as manufacturing industries: adapting to digitalisation that is
sometimes referred to as Industry 4.0. The pressure to adapt to
technological changes, such as IoT, AI, and AR/VR brings the opportunity
to reinvent and transform the processes to be more efficient and
impactful in the mission to serve the public. The concept of the smart
museum is that of a connected, digitized, and accessible cultural
heritage experience.
The Smithsonian has shown that exhibitions
can be mirrored in VR. From a VR production perspective, that means:
Once the the museum’s collection has been scanned and optimized as
high-quality 3D assets (digital twins in Industry 4.0 speak,) it is a
routine procedure to create a VR experience with tools such as the
Unreal engine to mirror the actual exhibition space and its features.
The advantage of such an endeavour is that visitors can later revisit
the collection. It also gives people, who otherwise are unable to attend
the exhibition because of distance or physical disabilities a chance to
do so. Last but not least, it allows the archive of the exhibition to
be recalled at a later point in time.
We can already admire the amazing work of dedicated photogrammetry professionals and enthusiasts on sketchfab,
where scanned buildings, statues, and other aspects of cultural
heritage are presented in remarkable surface detail in WebGL 3D. While
these are very often stunning pieces, there is still a problem of
missing cultural context and historical information. For that reason,
the next big step in photogrammetry and VR is storytelling and context.
The
real magic comes together when a photogrammetry piece or collection is
optimized for VR, brought into the Unreal engine, made walkable, and
supplied with pop-up information and audio hot-spots. Anyone who has
ever seen a well-done photogrammetry narration brought into VR will walk
out as a believer that this is the future of cultural heritage
preservation, where the actual space can be experienced and individual
pieces can be examined with additional information layers.
Using
open-source tools such as Blender 3D and free-to-use game engines, such
as the Unreal engine, opens up incredible opportunities for cultural
institutions to stay relevant, give remote access to a wider audience,
and be future proof by digitizing their collections.
For artists,
designers, and VR creators, it is a field in which to sharpen your
skills. Let’s not forget that photogrammetry is also used for commercial
applications and can be part of the creative process to create
incredible game environments.
In my book, Unreal for Mobile and Standalone VR,
I introduce the basic techniques to create VR experiences for
standalone headsets such as the Oculus Go, the Gear VR, and the Oculus
Quest by going through the production steps for asset creation, device
setup, locomotion, interaction, and optimization, while considering
basic UX considerations, core concepts, and the larger industry context.
Likewise,
creating a VR experience for a special event or an exhibition is
accessible to media designers, CG Artists, and creators without any
coding knowledge. The power of visual scripting using the Blueprint
system in Unreal makes the intuitive toolset accessible to creators with
a design-oriented background.
This is not only true for VR
exhibitions and cultural heritage apps, but also for product and design
presentations in VR. The opportunities are boundless!
This article was contributed by Cornel Hillmann, author of Unreal for Mobile and Standalone VR.