VR conception and comparison with AR， MR， XR

Virtual Reality (VR) is a kind of immersive experience which also called computer-simulated reality. It refers to computer technology that uses realistic headsets to produce realistic sounds, images, and other sensations to replicate real environments or create imaginary worlds. Users will immerse themselves in a fully virtual world while using VR. VR will make you feel like you are there. You turn your head and the world turns with you, so the illusion created by whatever world you’re in will never go away. And nowadays, VR is widely used in 21 industries such as the automotive industry, healthcare, tourism, education, art, design, etc.

However, some people will have confused about VR with AR, MR, and XR. As for AR, which is the alive, direct, or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics, or GPS data. Compared with VR, AR is existed based on people’s real-life and allows people to use a mobile device, headset to put reality in the digital world. For MR, the mixed reality, this is easy to understand, combines reality with the virtual world. Specifically, MR allows digital objects to exist in the real world and people could have some interaction with them. The key characteristic of MR is that the synthetic content and the real-world content are able to react to each other in real time. Extended Reality (XR) is a new term added to the technical vocabulary dictionary which is known by few people. Extended reality refers to all real and virtual combined environments and human-computer interactions produced by computer technology and wearable devices. Extended reality includes all its description forms such as Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR). And in this article, VR will be mainly discussed.

VR Market Size

 From the report of worldwide consumer virtual reality hardware market size, worldwide people show increasing interest in VR. The number of global virtual reality device shipments was 13.48 million units in 2020, and it is expected to reach a volume of 112.62 million units by 2026, registering a CAGR of 33.85% over the forecast period (2021–2026) and the COVID-19 crisis impacted several VR techs companies. 

According to the geography, we could separate the VR market into 5 areas: North America, Asia Pacific, Europe, Latin America, and the Middle East, and Africa. The growing fast market is the Asia Pacific, and the largest market in North America.

VR Headset

VR headsets enable people to fully immerse themselves in 3D places.VR Headset can classify into three types：

Tethered VR Headsets mean that the headset is physically connected to a computer by cables, such as HDMI and/or USB. Tethered virtual reality headsets are currently much more immersive than other types of VR due to the high-quality experience they can deliver. These premium VR headsets require a certain amount of setup space as well as a constant cable connection to a powerful gaming PC (generally a very expensive acquisition).

Standalone VR Headsets (all-in-one HMDs). This category of headsets requires the least external interaction, charging the battery and perhaps creating an account to access certain VR platforms, standalone VR headsets don’t need anything else from the user.

Smartphone VR Headsets. Using smartphones to provide a virtual reality experience. Users must simply slide their smartphones into the headset.

The price of VR headsets range from 1000 dollars to at lowest 20 dollars. People could buy the cheapest Headset firstly to decide if they want to invest more in VR advanced equipment.

VR in Museum

With the development of VR, museums apply VR in their exhibitions. VR allows museums to provide visitors with a better immersive experience, bringing collections to life. VR allows visitors to immerse themselves in a vivid virtual world, better experience the real scene where the collection is located, and help curators to better tell the story of the collection.

Museums give the public more opportunities to contact different cultures, and VR allows people to explore cultures from afar. For example, for people with some mobility impairments, going to a museum is out of reach. But now, VR gives those people with limited mobility or travel restrictions in the COVID-19 environment a new option to use VR software to visit museums on platforms like Quest.

VR allows people to plan and arrange a day’s exhibition at their own pace, without worrying that a long-term appreciation of details will affect subsequent visitors. And VR content can be archived forever, allowing the public to view it repeatedly.

VR also gives people the opportunity to see exhibits up close, such as the Mona Lisa at the Louvre. This painting has been famous for a long time. Every year, a large number of tourists go to the Louvre to watch it. Because of the crowd, people can only watch it from a distance and take pictures alive, but VR has given people all over the world the opportunity to see this painting. We can use it on the app without going to the Louvre. view this work.

VR also allows museums to reach more untapped consumer groups, and to stimulate their interest through VR to obtain sponsorships. Several museums have their apps like Hermitage, which offers exhibits on Ancient Rome for $2.99 on their app.

Many museums applied VR in their exhibitions.

The National Museum of Finland in Helsinki opened a new VR exhibit in 2018. Visitors can head back in time to the year 1863, as they explore R. W. Ekman’s painting The Opening of the Diet 1863 by Alexander II. The VR headset allows people to feel as if they are stepping inside the painting. Visitors find themselves within the scene and can look around at the Hall of Mirrors from a 3D perspective. They can even speak with the Russian Emperor and other characters depicted in the painting. It is part of an exhibit detailing Finnish life and politics in the 1860s, under the Russian Empire.

In 2018, the Natural History Museum partnered with broadcaster Sky to develop Hold the World, an educational VR experience that puts you face to face with Sir David Attenborough. This groundbreaking experience allows people to visit parts of London’s museums from the comfort of their own homes. Sir David was shot with over 100 cameras at Microsoft’s Mixed Reality Capture Studio in Redmond (similar studios exist in London and San Francisco) to recreate it as Sky’s Hold the Conceived and Produced in London Hologram of the World Experience – Based on content studio Factory 42, a VR application for Sky. These lenses were combined with photogrammetry — overlapping photographs — to reconstruct the natural history museum. The museum’s CT scanning department then worked with museum scientists and animation teams to create accurate models and animations of animals such as blue whales, stegosaurs, trilobites, dragonflies, butterflies, and pterosaurs. With Hold the world, Sir David will guide you on a tour of the Natural History Museum, introducing you to the different exhibits, and you will also get a close-up look at the exhibits.

In addition to VR, the museum also uses AR technology. Virtual Reality offers total immersion in a different reality. However, AR shows the reality and an altered version side by side. VR replaces what the user sees with an alternate reality. AR adds to what the user can already see. This means it can be useful for annotating scenes and providing extra information. It is also used to put scenes into context and highlight contrasts with the current reality. VR requires specialist technology, such as headsets, controllers, and sensors. AR experiences only need a smartphone or tablet and are downloadable as apps.

In 2021, The National Gallery in London looked to take the collections of the National Gallery, National Portrait Gallery, Royal Academy of Arts beyond the walls of the museum with an Augmented reality experience that members of the public could access through their phones. Users used an app to activate the artworks which were marked with QR codes on busy streets in central London.

Challenges using VR in museum

Budget

Cost is the biggest challenge for museums using VR technology. First of all, designing and managing VR programs requires a sufficient budget because they are very expensive. Secondly, the museum needs to provide visitors with a VR Headset, which will also be a lot of overhead. And due to the daily flow of people, the wear and tear of the VR Headset will be very large.

Sanitation

Sanitation is another issue that needs to be faced when using VR headsets. The same VR Headset may be used by many people in one day, and the oil secreted by hair and skin may accumulate on the Headset. And in the Covid-19 environment, how to disinfect and clean the VR Headset is also a problem.

Motion Sickness

Although VR means innovation for some people, some people cannot adapt to the physiological reactions that occur during VR, such as nausea, cold sweat, dizziness, fatigue, etc.

AR Competitor

Compared with AR, VR provides a more immersive experience to the audience, but the overall cost of VR will be much higher than AR. And AR is more accessible, and people just need to download some software to experience it. But VR requires specialized equipment.

Conclusion

It has become a trend for people to invest in virtual reality, and more and more museums are also adopting VR to expand their audience and provide visitors with a better immersive viewing experience. Although the use of VR in museums requires a large budget and solves the problems of sanitation and damage of equipment, the immersive viewing experience that VR can provide is worth it.

Reference

41, North of. “What Really Is the Difference between AR / MR / VR / XR ?” Medium. Medium, March 20, 2018. https://medium.com/@northof41/what-really-is-the-difference-between-ar-mr-vr-xr-35bed1da1a4e.

“5 Benefits of Virtual Reality for Museums.” Radical VR, October 19, 2016. https://www.radicalvr.ca/5-benefits-virtual-reality-museums/.

“7 Ways VR Is Changing the Museum Landscape.” MuseumNext, April 19, 2021. https://www.museumnext.com/article/7-ways-vr-is-changing-the-museum-landscape/.

“David Attenborough Is Giving behind-the-Scenes Tours of Natural History Museum.” Microsoft News Centre UK, March 23, 2018. https://news.microsoft.com/en-gb/2018/03/23/sir-david-attenborough-is-giving-behind-the-scenes-tours-of-the-natural-history-museum-as-a-hologram/.

“How Museums Are Using Augmented Reality.” MuseumNext, January 20, 2022. https://www.museumnext.com/article/how-museums-are-using-augmented-reality/.

Koo, Jacob. “AR and VR Could Be Educational – and Profitable – Tools for Museums.” VentureBeat. VentureBeat, December 15, 2017. https://venturebeat.com/2017/12/14/av-vr-educational-profitable-museums-moca/.

Pickard, James. “The Cheapest VR Headset Deals in March 2022.” TechRadar. TechRadar, February 3, 2022. https://www.techradar.com/deals/best-cheap-vr-headset-deals-prices-sales.

Pocket-lint. “What Is Vr? Virtual Reality Explained.” Pocket, May 10, 2021. https://www.pocket-lint.com/ar-vr/news/136540-what-is-vr-virtual-reality-explained.

Thompson, Sophie. “VR Applications: 23 Industries Using Virtual Reality.” VirtualSpeech. VirtualSpeech, March 1, 2022. https://virtualspeech.com/blog/vr-applications.

“Types of VR Headsets – PC VR, Standalone VR, and Smartphone VR.” Aniwaa. Accessed March 4, 2022. https://www.aniwaa.com/guide/vr-ar/types-of-vr-headsets/.

“Virtual Reality Is a Big Trend in Museums, but What Are the Best Examples of Museums Using VR?” MuseumNext, January 20, 2022. https://www.museumnext.com/article/how-museums-are-using-virtual-reality/.

“Virtual Reality Market (2022 – 27): Size, Share, Trends.” Virtual Reality Market (2022 – 27) | Size, Share, Trends. Accessed March 4, 2022. https://www.mordorintelligence.com/industry-reports/virtual-reality-market.

Whelan, Corey. “VR Motion Sickness: Treatments and Prevention.” Healthline. Healthline Media, June 1, 2021. https://www.healthline.com/health/vr-motion-sickness.

Gamification, in the most well-known and basic definition, is “the process of adding games or game-like elements to something (such as a task) so as to encourage participation” The definition of gamification may vary between people and situations, but the core is to utilize concepts from game development to solve problems in other fields. 

Though The term gamification was created by Nick Pelling, a British computer programmer and inventor, in 2003, people started exploiting the concept of gamification during the 19th century. A grocery company called the Sperry and Hutchinson Co. motivates its customers to consume by substituting money with the S&H Green Stamps. This case is the ever first gamification approach in the world. One hundred years later, key opinions leaders such as Jesse Schell and Jane McGonigal brought gamification to the public. From then on, people started thinking about “ the potential of gamification and its use in several aspects of our life” 

Nowadays, gamification is widely used in different industries such as catering, education, e-commerce, and of course, Arts by adding minor interactive gameplay on the website or application. Organizations, no matter for-profit or non-profits, are all persistently searching best way of optimization and believe that gamification may be applicable at some aspects of their businesses.

Why is gamification such a hit in the art world?

Art professionals and organizations have always held a strong interest in gamification even before the born of the word “gamification.” Besides, compared with other industries such as healthcare, the art world seems to have a special obsession with gamification. So why does this happen? 

The first thing is that arts and games are intrinsically related and undividable. Though whether the game itself is a kind of art is a question with polarized answers, the relationship between arts and games is undoubtedly close. Some of us may hear about the classic Role Play Game Assasin’s Creed by Ubisoft was chosen as a resource to help restore the burned Notre Dame de Paris. An artist from the Ubisoft game told the media that the game development team “has spent years literally fussing over the details of the building. She pored over photos to get the architecture just right and worked with texture artists to make sure that each brick was as it should be”.

Source: https://cdn.vox-cdn.com/thumbor/MFRFH1aMCZuaC4s1fOPY9yJHZSM=/0x0:2724×1530/1200×0/filters:focal(0x0:2724×1530):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/23036671/Screen_Shot_2021_11_22_at_11.26.11_PM.png

Alongside transferring art concepts to the game, arts organizations also appreciate games. The Victoria & Albert Museum collected the pixel-style casual game “Flappy Bird” and curated an exhibition. The process of art creation is actually a process of creating Intellectual Property, and IP is a just need in the gaming industry. On the other hand, the art world is always seeking a more inclusive, interactive exhibition mediation to broaden the boundaries of artistic creation. Game, which has an infinite capacity of presenting creativities, is an ideal vehicle for arts.

Source: https://i.pcmag.com/imagery/articles/01EkIOfisgxkfodZ4JmTIXa-1..v1569489280.jpg

Furthermore, Arts managers have always been looking for a better way to engage their audiences positively, and interaction is the core driver of engagements. Non-profits are willing to influence the public to do one of the two things:  to participate in a new activity that the audience would have avoided or not known about in the past; or to make changes to daily behaviors over a long period of time. In the first scenario, gamify cryptic art pieces makes art more understandable and less static. Meanwhile, as games are convenient to spread in the digital world and popular among people from many demographics, the target audiences could be more diverse. For the second scenario, games’ additional educational features could help. Unlike traditional arts education approaches, gamification makes arts education more effective than ever. In conventional art education practices, educators are used to presenting an art piece or concept as a whole to the students, which can result in misunderstandings and boards. However, while gamifying arts, we need to break down the original art production into artistic elements and then re-assemble them, and then players will accept the information step-by-step when playing the game. This is absolutely a rather exciting process. 

Another critical point is that game development, especially casual game development, is financially affordable for nonprofit arts organizations. Several accessible game engines such as Unity and Unreal eliminate the technical barriers, and the raising promotion channels like Google Ads and Facebook Ads make the publishing process straightforward. And even Unity itself is a popular ads channel in the game world. Take mobile casual games as an example. Usually, a hyper-casual game development team only requires 1-2 game developers and one graphic designer and may cost about $20,000. The core of game development is no fancy technology. It is more about creativity. Overall, for medium- and large arts organizations, building a game development team or hiring a gamification agency is an affordable way.

On the other hand, gamification might help nonprofits explore a new way of monetization. The gaming industry has always been lucrative with minimal entry barriers. According to mobile data and analytics provider App Annie, mobile games are on track to surpass $120 billion in spending in 2021, growing 20% over $100 billion in 2020. By gamify art collections, arts organizations may have a chance to generate revenue from the game and sustain a mission for social good. The gaming industry needs creativity, and the art world always seeks living resources. It seems like they are the best fit.

Last but not least, the unavoidable Covid-19. Pandemic restricts physical experience, but raising technologies such as Metaverse, VR/AR/XR makes the digital experience more genuine and even beyond reality. Gamification is an excellent way to present arts on digital equipment. I will further discuss this in the last part of the article.  

Gamification in Digital Arts Programming: Production-based and Organization-based

When implementing gamification in the real world, arts enterprises usually have two choices. The first choice is to gamify art collections. The Palace Museum in China has made several temptations on this. In 2018, The Palace Museum collaborated with NetEase, a top-tier technology company famous for game development and publishing, published a traditional Chinese art style Puzzle Role Play Game “Ink · Mountains and Mystery”. The game is inspired by one of the ten most precious Chinese traditional paintings, the “Thousands of miles of mountains and rivers”, which is The Palace Museum’s collection. The painting is a blue-green landscape silk painting finished in the Ming dynasty and has been appreciated for its vivid, imaginary color usage. Fortunately, the game flawlessly rebuilt the magnificent images utilizing 3D engines and gave a beautiful story to the original painting. At the end of 2018, the game had acquired more than 120 thousand downloads and effectively promoted traditional Chinese painting to younger generations. From then on, the traditional Chinese art style and elements have become a new trend in Chinese society, especially on social media. From this case, we can see that gamification can make arts more interactive, more authentic, and even rejuvenate a traditional art piece.

Another surprising thing is that via this gamified digital program, both the “Thousands of miles of mountains and rivers” and the Palace Museum obtained fans from younger generations: the game has more than 1.2 million followers on TapTap, a game channel for GenZers and Millenials in China. I believe this is not the end of this story. Imagine how many kids, teenagers, and young adults would be interested in traditional Chinese art and may become a member of the art community. An excellent gamification digital arts program could do more than improve audience engagement, and it can help a niche arts genre maintain sustainable development.

Another choice would be organization-based programming. In this field, the Metropolitan Museum of Arts in the U.S. and the Center Pompidou in French would be the two best examples. In 2012, only one year after the world’s first gamification summit closed in San Francisco, The Met published a game called “Murder at the Met: An American Art Mystery” which is a “web-based application (app) that guides users of smartphones and tablets through the galleries of American paintings, sculpture, and decorative arts at The Metropolitan Museum of Art to solve a murder mystery.”

The game is still accessible on the website today. From a 2022 point of view, this game is not visually appealing or attractive in gameplay design, but the concept it used was Avant-guard at that point. The Met’s foresight of the gamification trend is so precise that it keeps inspiring other organizations today. Going back to the game, we could quickly tell the difference between the Murder at the Met and the Mountains and Mystery: the prior one represents the organization while the latter focuses on the art piece. To better understand the outcome differences of these two different kinds of gamification digital arts programs, we can just take a look at another game produced by the Centre Pompidou, the Prisme 7. 

The Prisme 7 is “designed in collaboration with digital artists Bright and Game in Society, and with the support of the Ministry of National Education”, in the game, the players can “discover seven levels of play and interact with 40 major works from the biggest collection of modern and contemporary art in Europe. It also offers a unique sound immersion designed especially for the game by Ircam Amplify. ” Unlike the Murder at The Met, Prisme 7 doesn’t have a clear storyline, which more aligns with most casual and hyper-casual games. I attribute this to the different types of the two organizations. As the most famous visual arts organization with a heavy focus on art history, The Met would like to utilize a gamified digital program to do art history education. By contrast, the Center Pompidou is more contemporary, which means their collections and exhibitions are not that easy to interpret for the public, so they tend to use the interactive and attractive nature of the casual game to present their collections vividly and generate more audience engagement. 

Compared with the Mountains and Mystery, these two games create more brand exposure for the arts organizations and are more severe in terms of arts. The Met basically presented the whole story behind that piece, and Prisme 7 tries to convey contemporary art concepts as much as possible by utilizing the play-reward mechanism in games. Also, both of them have a solid organizational identity. The Met uses one of its most identical masterpieces, Portrait of Madame X and Prisme 7 restored the special building of Center Pompidou in the digital space. However, the Mountains and Mystery only picked some elements from the original painting and tried to modify the art further to make it easier to spread. Users can not find many clues about the original art piece or the organization behind it. What they know is only the beautiful, fictional story that happened in the blue-green landscape. The differences are mainly from distinct business collaboration models. The Murder in The Met and Prisme 7 is developed and published by the organizations, so the two games are just like two digital gamification programs conducted by the organizations. They should make the programs align with the organization’s mission and focus on the academic and educational features. But the Palace Museum cooperated with NetEase, which was entirely in charge of the game development and publishing. So, It’s easy to find the commercial elements in the Mountain and Mystery.

I would not judge which kind is right and which is wrong. I would rather say that there are always pros and cons. The Met and Center Pompidou definitely took a giant step in informative and interesting arts education, but their productions are still targeting arts fans, which constrained the scope of the impact. The Palace Museum approached audiences from the outside art world, but they sacrificed a lot on artistry. If the organization doesn’t have a strong voice, the art may be threatened by over-entertained. 

A glimpse of future: Gamification, XR technology and Arts

For decades, VR/AR/XR technology has been a viral topic. Most arts organizations have also paid attention to this trend for a long time. However, we still have less idea about what’s next to XR technology and how we can use it more creatively, or at least make it beyond a digital venue. However, with the raising gamification concept, we may garner a chance to look at the future.

In the field of arts education, there are some research and experiments on the effectiveness of VR-based gamified art history education. A research team in the Department of Systems Engineering, Universidad Nacional de San Agustin de Arequipa in Peru has tried to achieve the “implementation of a virtual reality game using gamification by elements of video games to improve the teaching of Art History“ To do so, they ”simulate a virtual museum, with predefined stages that expose works from different artistic currents with their respective representations all owing interact with the characters, objects, animations and sounds presented in the application.” The result shows that navigation activities, tasks, and movements are statistically correlated in the VR environment and significantly influence the outcome of learning. And, compared with implementing VR or gamification separately,  the combination of VR and Gamification has higher efficiency in arts education. 

Another possible development direction is the Metaverse. In general, Metaverse is “an immersive virtual world that people “enter” through the use of technologies like AR and VR” 

In terms of the art world, leading arts organizations such as Sotheby have just announced the launch of Sotheby’s Metaverse, the auction house’s new platform designed specifically for digital collectors.  Another well-accepted opinion is that humans need to interact with Metaverse via gamification. The most significant supporter of this idea is Satya Nadella, the CEO of Microsoft and the key character behind that shocking acquisition between Microsoft and Activision Blizzard. He told the media that “Metaverse is essentially about creating games,” and “It is about being able to put people, places, things [in] a physics engine and then having all the people, places, things in the physics engine relate to each other.” As I stated before, the game is closely related to the arts, just as it’s related to Metaverse and XR technologies. Metaverse somehow creates a platform for arts, technologies, and gamification, but right now, we cannot predict what will happen.

Conclusion

Through previous analysis, we can conclude that gamification is and will always be a key concept for every entity in the art world. Arts enterprises recognize it as a management tool,  artists use it as a vehicle of artistic creation, and audiences enjoy spending time with it. Overall, gamification significantly helps the short-term growth and the sustainable development of art. In addition, new technologies and technological concepts give gamification more possibilities. We can tell that gamification will combine with different emerging elements and build connections between arts and the future.

Finally, the need for a new appropriate museum and archive infrastructure is shown to preserve the art of our time.

Digital Art Through the Looking Glass, Grau, Hoth and Wandl-Vogt

Alongside the constant evolving of technology, digital art has modeled itself into its best possible version. While the talents use digital tools to create more and more diverse and cutting-edge art pieces, it gradually becomes a significant form of contemporary art standing on its foot. Feeling a strong need to create a space dedicated to presenting digital art in its own context, the Museum of Contemporary Digital Art came to life.

The Museum of Contemporary Digital Art (MoCDA) is a digital-based museum that “exhibits digital artworks to document, collect and advance the position of digital art.” Their primary focus is education and the endorsement of contemporary digital art. Its exhibitions could generally be categorized into three different types: virtual exhibitions based on the museum website, virtual exhibitions generated by other virtual world platforms, and new media arts exhibited directly on the website. System Shock – 777 Exhibition is a first type, in which users enter the virtual space by clicking on a link to the web page. It is viewed from the first perspective and adapts the design language of traditional exhibitions. One can zoom in and out on any part of the exhibition space. And in some cases, users can follow the arrow on the ground to enter several rooms. The exhibition is accompanied by background music. By clicking on the chat sign on the right bottom of the page, users will see how many people are in the exhibition and chat with each other.

The exhibition, MEMENTO MINTI – if you stop contributing, you will be forgotten, was presented via Decentraland, a 3D virtual world. It is an example of virtual exhibitions generated through VR platforms powered by blockchains, in which users can purchase virtual plots of land, craft, and monetize the contents within the virtual world. This shared metaverse allowed MoCDA’s exhibition to be much more sophisticated. Before entering the virtual exhibition, users get to create their own avatars (though with quite limited options for free). From a first glance, the environment shares the simple blocky graphics that would remind people of Minecraft. The exhibition space was held in the MoCDA virtual building and consists of three stories.

Like the web-based virtual gallery, the interior of MoCDA Building follows the basic design language of the real-space exhibition.  Similar applications such as background music and the chat room can also be found in the virtual museum.  Except this time, users have a lot much more freedom in controlling the movement of avatars, and are able to pull out more complicated action commands, such as jump or  interact with the artwork. In general, the experience with MoCDA’s virtual museum in Decentraland is more like navigating in a game setting.

The third type of exhibition is based on the web page, so visitors do not have to open another tab to enter the exhibition. The exhibition Digital Bodies – concerning bodily structures showcases blockchain-based tokens. Although it appears like a regular GIF on the website, it is in fact a unique 3D model registered on the blockchain. Thus, MoCDA is both creating traditional exhibitions in a virtual space, and presenting arts made with the latest technology.

In light of MoCDA’s commitment to education, the institution holds  live talks with digital artists, curators, and art experts. The talks are live-streaming, and those recordings of talks can be revisited on the museum website.

A museum without gravity, plumbing or code regulations.

Johan van Lierop

Unlike MoCDA, which showcases works meant to be consumed through VR, the collection of the Kremer Museum “forced” the old masterpieces into a digital recreation. The museum is an extreme case of VR taking over the physical entity. It displays over 70 Seventeenth-Century Dutch and French paintings of George Kremer’s collection in exclusively virtual space. 

Each painting has been photographed between 2,500 and 3,500 times with the “Photogrammetry” technique, resulting in an ultra-high-resolution model for each work. The collection includes great masters such as Gerrit Dou and Rembrandt and less famous but exciting and worthy artists such as Abraham Janssens and Theodoor Rombouts. The idea is to make it truly accessible to all people to see the masterworks in a museum setting. Johan Van Lierop, an architect and the designer of the museum, believes VR is ” to the 21st century what Dutch Realism was for the Golden Age, allowing the observer to escape into an alternative reality or mindset.”

The paintings of the Kremer Museum can be viewed under the “Collection” section. The full-sized painting and a detailed introduction to the work will pop up by clicking on each piece. Visitors can then zoom in on any part of the painting to view it close-up. It seems like they worked really hard to recreate the unique colors of individual artworks and feel their textures, such as small cracks of pigments resulting in a long period.

The Kremer Museum can be downloaded from various app stores, including Apple, Google Play, Steam, Meta Quest, as well as Vive Port. A VR experience is also made possible through the headsets.

A hub for debate and discussion around innovation through the digital.

Lee Cavaliere

Launching in 2020, Virtual Museum of Arts (VMOA) is the world’s first entirely virtual museum. It collaborated with some of the most prestigious museums worldwide, such as the Museum of Modern Art, Chicago Institutes, and the Musée d’Orsay. It recreated the 3D reproduction of works from over 40 artists varying in time, styles, and places, such as Henri Matisse, Francisco Goya, Frida Kahlo, and Bansky.

To visit VOMA, users can simply tap on the “Enter” button from the homepage and then launch on the entrance. By clicking on the white circles, users will be able to move around in the virtual space and drag the mouse to have a 720-degree view of the space. There is the built-in sound of running water, corresponding to the waterfront surroundings. At the same time, plants are in motion with the wind to create a generic imitation of nature. Architect Emily Mann even managed to change the scenery according to different times of the day and weather.

When entering the gallery space, users can click on each piece for more information, as well as to zoom in and out. Some of the video installations will be viewed via Youtube. Map, catalogs, and the chat function are available on the bottom of the page. Users can either use the map or follow the staircases to move to other sections. Compared to the virtual museum of MoCDA, which has users to take the trampoline-like installation to jump to different floors, transferring through the staircase seems a more realistic experience. However, the limitation of only being able to move to the designated spot of the circle may undermine such realistic efforts.

There is also a cafe, which serves as a chat space for users to message other users sitting on the same table. The reading room is an online archive with materials to featured artists, and the shop sold merchandise such as the reproduction of paintings, T-shirts, and notebooks. All of those applications are closer to the usual web-browsing experience instead of a more interactive touch.

Compared to the MoCDA, which wants to create a place for digital art to be displayed in its original context, and Kremer, who challenge the physical means of old masterpieces, VOMA is less radical in its agenda. Instead, it focuses on developing a fully immersive experience with the latest technology. Stuart Semple, the creator of the VOMA, brought together a team of programmers, video game designers, and architects to build the museum from scratch. They attempt to restore all the essential components one would expect from a physical museum, including a uniquely designed architecture, external surroundings, gallery space, cafeteria, and bookstore.

What does VR mean for museums?

We see VR museums grapple with the means of digital and push the boundaries. From MoCDA’s creating a space for digital art to Kremer’s converting physical paintings to an exclusive virtual view to VMOA’s fully immersive museum. They claim the legitimacy of digital art forms in their own language as something more than just a by-product of the art world. In doing so, they challenge audiences on to what extent we would refer to creation as art and who gets the access, and what to expect as an audience of the museum. 

One thing particularly interesting is that, among the three, Kremer is the only one who decided not to attach their paintings to a virtual wall. By depriving the works of the physical substance, but replicate the exact features of the original painting in a way that you will expect to see with human eyes (or even clearer and allows more freedom to observe in detail. Are there more we are asking from the museum? Is there something fundamentally essential and satisfying about the experience of physically walking through piece by piece? The answer can vary for different people, but the existence of virtual museums opens up the dialogue for it. 

Resources

“About: Museum of Contemporary Digital Art.” Accessed February 20, 2022. https://www.mocda.org/about.

“The History of the Kremer Collection.” The Kremer Collection. Accessed February 21, 2022. https://www.thekremercollection.com/history-of-the-collection. 

“Decentraland. “Accessed February 20, 2022. https://play.decentraland.org/?island=I9xzhu&position

=-21%2C116&realm=dg&server=loki-amber. 

“Home – The Kremer Collection.” The Kremer Collection. Accessed February 21, 2022. https://www.thekremercollection.com. 

“Intelligence 0296, VOV, Maurice Benayoun: Museum of Contemporary Digital Art.” MoCDA. Accessed 

February 20, 2022. https://www.mocda.org/maurice-benayoun-intelligence.

“The Kremer Museum.” The Kremer Collection. Accessed February 21, 2022. https://www.thekremercollection.com/the-kremer-museum/.

“Live Talks and Podcast: Museum of Contemporary Digital Art.” MoCDA. Accessed February 20, 2022. https://www.mocda.org/live-talks.

Magazine, Smithsonian. “The World’s First Entirely Virtual Art Museum Is Open for Visitors.” Smithsonian Institution, September 17, 2020. https://www.smithsonianmag.com/travel/worlds-first-entirely-virtual-art-museum-is-open-for-visitors-180975759/. 

“System shock – 777 exhibition.” Accessed February 20, 2022. https://www.mocda.org/system-shock-777-exhibition.

“Virtual Online Museum of Art (VOMA) Officially Opens Online.” Blooloop, January 18, 2021. https://blooloop.com/museum/news/virtual-online-museum-art-voma-open/. 

“Virtual Reality Is a Big Trend in Museums, but What Are the Best Examples of Museums Using VR?” MuseumNext, January 20, 2022. https://www.museumnext.com/article/how-museums-are-using-virtual-reality/.

“VOMA – Virtual Online Museum of Art: A Virtual Online Museum of Art.” VOMA virtual online museum of art. Accessed February 24, 2022. https://voma.space/. 

“World’s First Virtual Museum VOMA to Launch next Month – with Your Help.” FAD Magazine, May 7, 2020. Accessed February 24, 2022.https://fadmagazine.com/2020/05/07/worlds-first-virtual-

museum-voma-to-launch-next-month-with-your-help/. 

Open Heritage is just one of several online-available recreations of cultural heritage sites. It presents 26 different sites as realistic, explorable 3D locations, including locations in “parts of the Roman city of Pompeii, Native American cliff dwellings at Mesa Verde in southern Colorado, and the 1000 year old Temple of Kukulcan in the Mayan city of Chichén Itzá.” These sites, unlike some others, look at how they look now, compared to how they may have looked when they were being actively used. Along with Google’s Open Heritage service, several other institutions have toyed with their own versions of VR recreations, including TIME and National Geographic. TIME’s work, which was actually a preview for a film by British director Christopher Nolan, used VR to transport the user into the Battle of Dunkirk during the second World War. National Geographic’s immersive virtual reality experiment looks into Viking culture, placing the user in the middle of a Viking “fighting pit”. While these two examples are action-packed, VR is also being used by arts and cultural organizations for more observational purposes, such as the aforementioned site recreations. Australian archaeologist, Simon Young, for example, created a VR recreation of several different Roman ruin sites. The VR experience allows visitors at these Roman ruins to look through their VR and see what the site once looked like centuries back. Young’s recreation can be accessed via smartphone and plugged into a portable smartphone VR headset. 

These smartphone VR headsets, and other recently popularized VR headsets, have changed the means of accessibility towards VR, making it much easier for cultural institutions to find and use immersive reality experiences in “educational, explorative, and exhibition enhancement purposes.” Virtual reality is a rapidly changing and growing technology, but the creation of these portable headsets has laid a baseline for museums and other institutions to use and work off of. According to an article by Erik Champion and Mafkereseb Kassahun Bekele, what actually engages the user “is related to the ability of the virtual environment to enable engaging experiences as a result of the combination of immersivity and intuitive interaction with the cultural context in the virtual environment.” The relevant cultural context and immersivity of VR in this application is what enhances the learning experience for the user, and for the most part has been able to be made accessible to museums that otherwise would not have any representation of these sites or their artifacts. 

The use of VR recreations for cultural heritage sites has become crucial in preserving public memory, with many being at risk of destruction. One main concern for preservationists is the destruction of war in countries such as Ukraine, Iraq, and Syria, and the loss being inflicted when it comes to the existence of heritage sites in these regions. In a statement made by Michael Danti of the Syrian Heritage Initiative at the American Schools of Oriental Research, he described the destruction of war as “the worst cultural heritage emergency since World War II…here we see how technologies can be used not only to document the making – and unmaking – of heritage, but also to rebuild it; both materially and in the imagination.” In Syria, specifically, the Islamic State “has sped the pace of looting by encouraging professional looters with heavy machinery and archeological knowledge to dig archeological sites in return for payment to [the Islamic State] of a 20% ‘tax’ on the value of what they find.” Not only does this looting bring them profit, so in turn they encourage the destruction, but it also serves as a violent reminder that there is no alternative in their world. This is where the creation of virtual realities surrounding these heritage sites is growing in importance.

As of 2016, virtual reality and cultural specialists have been working together to recreate these sites to preserve their existence in time after their destruction at the hands of militant groups. The Institute of Digital Archaeology, a joint project between Oxford, Harvard, and the Museum of the Future (Dubai) created a 3D virtual model of a Roman arch destroyed by the Islamic State in Palmyra, Syria. The recreation was based on photos from archaeologists and tourists alike before the site’s destruction, and after it’s initial translation into a VR experience was able to be carved into marble by robots programmed based on the model. Something that is also important to note with this technology, however, as highlighted by artist Morehshin Allahyari in Legal and Ethical Considerations for Digital Recreations of Cultural Heritage by Erin L. Thompson, is that we must consider power structures and “how it’s different when Westerners or tech companies save cultural things compared to someone else who actually comes from the culture.” Sites and antiquities that are recognized within these countries, but perhaps not the greater Western culture are subject to neglect in these projects. This is just one of several ethical considerations when using these technologies to recreate cultural heritage sites. 

In Legal and Ethical Considerations for Digital Recreations of Cultural Heritage, Thompson highlights further considerations similar to Allahyari, including “the political uses of representation and interpretation of cultural heritage; the accessibility or lack thereof of digital representations; the violence to the ‘authentic’ or ‘real’ that the virtual  might  inflict; the correct approach to data transparency and sharing; and the ease both of manipulation and surreptitious capture of digital images.” Concerns also often include political exploitation of the sites, that these VR experiences will be built by engineers that have had little interaction or no real connection with the physical sites themselves, and that they will overall be modeled after Westernized perspectives. These heritage sites, as well as their antiquities, “are not protectable by copyright; at thousands of years old, they are in the public domain many times over,” giving almost anyone that wants permission to form a recreation without the necessary thought to history, detail, and cultural consent. 

Virtual reality recreations, as an artistic response to these cultural heritage destructions, serves as an important tool moving forward for many nations and their unique cultures. These recreations “stand as symbolic representations of a society’s existence and have paid the price over the eons,” with many of these projects being created out of pure necessity of saving public memory of these sites that could soon be lost forever. While the creation of these VR experiences are at risk of numerous ethical considerations, including the perspective, political view, and accessibility, the correct intention and balanced collaboration can complete VR cultural heritage recreation projects that are relevant and non-exploitative. With the use of best practices, such as clearly marking VR experiences based on who made it versus presenting it as a completely neutral representation, and to provide annotations to the viewer within the experience that educated them on conflicting theories or aspects of the representation that may differ. Allowing viewers to make their own informed decisions based on these annotations could certainly be a solution for these experiences when being presented by organizations that are not necessarily deeply connected to these heritage sites. Overall, these uses of the medium with historical preservation as a driving factor are an important aspect of virtual reality creation that should not be overlooked despite the ethical considerations, with many of these concerns being able to be addressed if the works are to go to larger public audiences. With meaningful collaboration around the project, VR professionals and historians alike can build an experience that is adequately true and reflective of its home culture.

VR PRIMER

Virtual reality, or VR, creates a simulated environment where the user is immersed in a reality other than their own. They can interact with this artificial 3D environment, and in current VR experiences the environment adjusts to the user’s movement to visually change, just like perception changes in real life when looking in different directions or moving around.

Although commonly thought of as a 21st-century invention, VR can be traced back to 1838 when Sir Charles Wheatstone created a stereoscope, using mirrors to create a 3D viewing experience.

VR is most often presented with goggles the user wears to be immersed in the simulated environment. These range from cardboard models used in tandem with a smartphone, to large headsets that contain speakers for additional auditory immersion. Many are used with some kind of gloves or hand controllers so hand movements are captured in addition to head movements. The Oculus 2 headset, by Meta (formerly Facebook), was the bestselling VR headset in 2020.

VR has many applications in a variety of fields. Video games is one of the most popular applications, putting gamers in the middle of the action. Training is another use case–Walmart famously uses VR to train its HR Associates, as it provides a hands-on experience instead of learning something by reading. It has also found a place in therapy as PTSD patients use VR as a form of exposure therapy. The arts have more recently leveraged VR as a tool to enhance their practice.

PRECURSOR: IMMERSIVE THEATER

Before VR made it to the theater world, immersive theater gained popularity over the past decade with productions like Sleep No More (Punchdrunk Theater) leading the charge. Immersive theater brings the audience into the action; instead of being distanced from actors and sets by a divide between stage and seats, the production is taking place around the audience with no distance barrier. This immersive experience helps introduce more sensory experiences for the audience, with senses of “real space,” time, and movement. By having more senses activated, audiences are more engaged and have stronger ties to what they’re experiencing.

There is also an element of engaging directly with actors in the immersive theater space. In a typical theater setting, an audience member is sitting and watching a stage, akin to watching a screen. There is no direct human interaction, just human perception. When immersion is layered on and an audience member can interact with an actor, even talk to them in many cases, their human hardwiring for social interaction is activated, and the experience becomes more intimate and engaging. Instead of narrowing an audience’s world to what’s onstage, this expands their worlds by layering elements on top of it.

Immersive theater is successful in heightening audience experience without the assistance of VR. However, VR amps up the engagement and emotion to bring the benefits of immersion to the next level.

VR IMMERSION & EMOTIONAL RESPONSE

VR brings immersive theater to the next level–not only is the audience surrounded by what’s happening in the production, VR makes it so they are completely immersed into a different reality and POV. Audiences can get completely lost in the narrative because they are 100% immersed, and not experiencing distractions that could take them out of the present.

There is little to tell the audience they’re actually just in a theater venue or home on their couch. This effectively transports the audience and contributes to what’s referred to as a “place illusion,” where the audience feels like they’re actually experiencing a situation, despite knowing they put on VR goggles and aren’t really in that situation. This creates sensory perceptions and bodily memories that would happen as if the audience member was truly in the physical environment, prompting a new level of understanding and empathy for a situation other than their present reality.

Several studies have measured how audiences emotionally react to VR theatre experiences versus non-VR theater. While non-VR theater typically elicits some kind of emotional response from the audience, when the audience is experiencing theater in VR emotions, they experience more “tangible and personal” emotions from their close proximity to the production.

He et al. presented test subjects with two theater experiences: one filmed in 2D to simulate viewing a stage from a typical seat in the audience, and one of the same subject matter presented in VR. The study, which measured emotional response with surveys as well as EEG measurements, found that VR led to a higher sense of presence, higher engagement, and higher emotional response (both self-reported and physiologically-measured) versus the 2D experience. Tian et al. ran a similar study between 2D and VR stimuli and came to the same conclusion that physiologically-measured emotional arousal was higher in VR.

Interestingly, the effects of VR may be more of a subconscious impact than audiences realize – a study by Pozharliev et al. found that while test subjects did not self-report a heightened emotional response to VR advertising versus non-VR advertising, physiological measures like EEG did indicate heightened emotional arousal. So, even if audiences don’t consciously feel like a VR theater experience is impacting them differently than a non-VR experience, their brain is likely responding differently.

Notably, these studies did not present an interactive VR experience where the audience members could interact live with others; instead audiences interacted with a 360-degree pre-filmed world. Layering on direct interaction has even more implications for the audience.

VR IMMERSION WITH INTERACTIVITY

VR can make audiences part of the production itself, interacting with actors and helping to shape the show. Theater is typically a passive experience for the audience, in that they’re just watching. As a participant, they’re not only consuming the art but making it alongside actors and those behind the scenes. This enables a more communal theater experience, which is said to help contribute to the emotional catharsis that’s a large part of the goal of dramatic narrative.

The Under Presents have presented several VR theater experiences (one featured in the above video), and their actors have reported that they are able to respond to audience cues, like body language, that they would not be able to pick up on when distanced from the audience in typical theater setting. The actors adapt their acting live as a response, meaning the audience truly impacts the production. No passive viewing in site.

VR THEATER & EMPATHY

In addition to heightened emotional response, VR can enhance the empathetic response of audiences. Empathy is one of the often-touted benefits of theater–audience members can empathize with characters onstage, often seeing and understanding viewpoints and experiences other than their own. Theater can open audiences’ eyes to new perspectives.

VR can bring this to the next level, thanks to the total immersion. Being up close and personal with characters, sometimes even perceiving the production from the point of view of the character, helps form a more empathetic bond with the work being presented.

Cristofi et al. aggregated many surveys focusing on VR and highlighted the finding that VR experiences enable “perspective taking,” a psychological driver of empathy where someone perceives a situation from the perception of someone else. The studies provide support that this does enhance audiences’ empathetic responses and, in specific applications, reduces prejudice towards stigmatized groups.

In practice, this seems to be the case. At NYU’s Tisch School of the Arts’ Drama program, students are collaborating in VR. Students participating highlight their experience that as an audience member, you’re truly experiencing a situation like a character is, leading to stronger empathy than in a non-VR setting.

However, scientific findings are still emerging as VR becomes more popular. In 2021, a meta-analysis of all known studies on empathy in VR provided support that while VR enhances empathetic response, it’s limited to emotional empathy. Meaning users can feel compassion towards others’ points of view, but VR is not enabling users to truly imagine others’ perspectives (which would then elevate the experience to cognitive empathy) more so than non-VR applications do. The research points to the need of imagination to more actively construct and understand another’s experience, which can be done with active engagement, such as talking to others in the immersive experience.

LOOKING FORWARD

The use of VR is still in its early stages, especially with theater applications. VR remains a cool new technology that draws in audiences with its novelty, especially as presenters seek to upgrade the at-home theater experience thanks to COVID-19, but it can achieve more than just novelty.

We know that theater is an important tool in our world for looking at society under a lens and understanding different perspectives, and there is evidence to suggest that VR can help theater do this more effectively. Science points to the importance of active interaction to engage with the VR theater world, as is done by The Under Presents in their work, with University of Iowa as another example (featured above). Maybe it won’t achieve critical mass in the theater industry, but growing experimentation with it suggests it’s here to stay. It’s up to us to keep innovating with the new capabilities VR can provide.

References

Barrell, Sofía. “What Dreams May Come: A VR ‘Hamlet’ Puts Us in the Room.” AMERICAN THEATRE, January 25, 2019. https://www.americantheatre.org/2019/01/25/what-dreams-may-come-a-vr-hamlet-puts-us-in-the-room/.

Barn Theatre. Edward II VR | Full Performance Virtual Reality | Relative Motion & Barn Theatre, 2020. https://www.youtube.com/watch?v=HGrntK9pE8c.

Christofi, M., and Despina Michael-Grigoriou. “Virtual Reality for Inducing Empathy and Reducing Prejudice towards Stigmatized Groups: A Survey.” 2017 23rd International Conference on Virtual System & Multimedia (VSMM), 2017. https://doi.org/10.1109/VSMM.2017.8346252.

Culture Whisper. “Enter a Virtual Reality at the Young Vic.” Accessed February 22, 2022. https://www.culturewhisper.com/r/theatre/young_vic_vr_draw_me_close/13002.

He, Linjia, Hongsong Li, Tong Xue, Deyuan Sun, Shoulun Zhu, and Gangyi Ding. “Am I in the Theater? Usability Study of Live Performance Based Virtual Reality.” In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, 1–11. VRST ’18. New York, NY, USA: Association for Computing Machinery, 2018. https://doi.org/10.1145/3281505.3281508.

“History of VR – Timeline of Events and Tech Development.” Accessed February 27, 2022. https://virtualspeech.com/blog/history-of-vr.
Verywell Mind. “How Virtual Reality Exposure Therapy (VRET) Treats PTSD.” Accessed February 27, 2022. https://www.verywellmind.com/virtual-reality-exposure-therapy-vret-2797340.

Laharia, Utkarsha. “VR Theatre Doesn’t Seem So Distant Anymore.” AMERICAN THEATRE, May 18, 2020. https://www.americantheatre.org/2020/05/18/vr-theatre-doesnt-seem-so-distant-anymore/.

Martingano, Alison Jane, Fernanda Hererra, and Sara Konrath. “Virtual Reality Improves Emotional but Not Cognitive Empathy: A Meta-Analysis.” Technology, Mind, and Behavior 2, no. 1 (June 17, 2021). https://doi.org/10.1037/tmb0000034.
Odeon. “Odeon Theatrical | Home.” Accessed February 27, 2022. https://www.odeontheatrical.com.

Pike, Shane. “Virtually Relevant: AR/VR and the Theatre,” no. 17 (2020): 9.

Popat, Sita. “Missing in Action: Embodied Experience and Virtual Reality.” Theatre Journal 68, no. 3 (2016): 357–78.

Pozharliev, Rumen, Matteo De Angelis, and Dario Rossi. “The Effect of Augmented Reality versus Traditional Advertising: A Comparison between Neurophysiological and Self-Reported Measures.” Marketing Letters, May 10, 2021. https://doi.org/10.1007/s11002-021-09573-9.

Tian, Feng, Minlei Hua, Wenrui Zhang, Yingjie Li, and Xiaoli Yang. “Emotional Arousal in 2D versus 3D Virtual Reality Environments.” PLOS ONE 16, no. 9 (September 9, 2021): e0256211. https://doi.org/10.1371/journal.pone.0256211.

University of Iowa. New Theater Experience Blurs Line between Reality and Virtual Reality, 2018. https://www.youtube.com/watch?v=DfPnWR7Fg6c.

“VR Headset Unit Sales by Device Worldwide 2020 | Statista.” Accessed February 27, 2022. https://www.statista.com/statistics/987701/vr-unit-sales-brand/.

“Virtual Reality | Computer Science | Britannica.” Accessed February 27, 2022. https://www.britannica.com/technology/virtual-reality.

VRScout. VR Live Theater Has Finally Arrived – The Under Presents: Tempest, 2020. https://www.youtube.com/watch?v=WxeMq5VVgLo.

“Walmart Leverages VR For HR Associate Training.” Accessed February 27, 2022. https://www.retailitinsights.com/doc/walmart-leverages-vr-for-hr-associate-training-0001.

“What Is Immersive Theater? The Six Elements That Define It at Its Best – New York Theater.” Accessed February 18, 2022. https://newyorktheater.me/2019/10/04/what-is-immersive-theater-the-six-elements-that-define-it-at-its-best/.

“With Real Theaters Closed, Actors Are Turning to Virtual Reality – CNET.” Accessed February 21, 2022. https://www.cnet.com/news/with-real-theaters-closed-actors-are-turning-to-virtual-reality/.

Yu, Kathryn. “How Immersive Virtual Reality Theatre Pushes the Limits of Storytelling.” Medium, April 10, 2019. https://noproscenium.com/how-immersive-virtual-reality-theatre-pushes-the-limits-of-storytelling-8265b198bfc7.