“Preserving cultural heritage isn’t something that is solely nice to have or nice to do, it’s sometimes imperative to the well-being of the world’s societies.”

– Brad Smith, the President of Microsoft

According to Poonam Gandhi, a professor at Flame University, culture and heritages shape human society and provide a sense of identity. Specifically, cultural heritages can stimulate tourism, which hence benefits the local economies, educate people with culture and history, and enhance social inclusiveness.

The importance of preserving cultural heritage is obvious, meanwhile, people are facing difficulties in the preservation process. So, what can AI and other modern technologies actually do in the process of preserving cultural heritages that makes Microsoft and other organizations invest in them?

Antiques Restoration

Due to natural disasters or man-made damages during the war, lots of precious historical heritages had been broken into pieces. Considering the time and cost efficiency, plus the difficulty of lacking pieces of evidence of their original looks, it’s very difficult to repair those historical artifacts and might need the efforts of several generations. Just like solving a big jigsaw puzzle without knowing what it should look like, everything starts from scratch and guesses. 

RePAIR

The ancient Roman city of Pompeii which was destroyed and buried by the eruption of Mount Vesuvius in A.D. 79, is now only left with the incomplete architecture and thousands of irregular fragments of ancient artifacts, such as vases and frescoes.

The RePAIR (Reconstructing the Past: Artificial Intelligence and Robotics meet Cultural Heritage) project is proposed by a group of scientists at the Italian Institute of Technology (IIT) to help with the time-consuming and labor-costly physical reconstruction process of Pompeii restoration. The technologies involved in the project are robotics, 3-D scanning, machine-learning algorithms and artificial intelligence. 

The first component is about scanning the fresco puzzle pieces using high-tech computers, using machine-learning algorithms to solve the puzzles, and having human expertise to guide the AI at the same time.

A team of archeologists from the University of Lausanne, who have attempted to solve the puzzle in the past, along with other professional art historians and scholars input their current knowledge about the Pompeii puzzles into the AI algorithm, then let it evolve and study by itself based on those examples, and apply to a larger scale. The computer software will compare all the fragments in pairs and evaluate their degree of similarity based on the shapes, the fitness, and how the illustrations on the pieces match up, which helps draw a picture of what the original complete piece looks like. 

The second component brings robots into the workplace. The robot should be able to scan those pieces on its own, and move the fragments together as a complete one based on the paired results generated by AI. In the end, it will reassemble the pieces together into a finished “puzzle”. Designed by Nikolaos Tsagarakis and his colleagues at the Humanoid & Human Centered Mechatronics Lab at IIT in Genoa, the robot will look like an arm that has a similar size as an average person, weighing 25 to 30 kilograms and lengthening 80 to 100 centimeters. A soft glove hand will be attached to the arm to grab and move the fragments with extreme care, and gather information about them while holding them in hand. The robot will help save a large amount of time and human resources. It doesn’t need to sleep or eat, but can work 24 hours a day without rest. Antonio Bicchi, a senior scientist at IIT in Genoa and the chair of robotics at the University of Pisa is in charge of the design of the robot’s hand, and says the ideal version of it in the future will also be able to allow humans to wear them and gather data during the process.

The project and its techniques are still at the experimental stage. The two experimental subjects will be the frescoes from the ceiling of the House of the Painters at Work, in the Insula of the Chaste Lovers, and the frescoes from the Schola Armaturarum. At the end of the project, the finished frescoes will be exhibited to the public for visitors to experience ancient Pompeii and for researchers to study. And this is the first time in the archeological field that AI will be used at such a large scale, and robotic hands will be responsible for such a large amount of pieces.

Therefore, the progress it can contribute to cultural heritage restoration and human history is significant and meaningful. “If the project succeeds, the technology can be used to reconstruct a variety of fragmented cultural artifacts whose restoration has remained ‘out of human reach’, ” said the project coordinator Marcello Pelillo, a professor of computer science and artificial intelligence at the University of Venice.

The RePAIR scientists aim to install the robot in Pompeii by the spring or summer of 2022. And hopefully, this project will succeed and benefit other cultural sites in the world that need to be repaired.

Scan4Reco

Another EU-funded project called Scan4Reco is also a great example of using artificial intelligence-assisted technologies for cultural heritages’ preservations.

They first use depth-scanning cameras to create a high-resolution 3D replica to transfer the artifact into digital files. Then, more information about the object’s underlying layers which cannot be witnessed directly by human eyes is added to the research by the sensor data.

“Very specialized sensors used in the conservation domain carry out non-destructive diagnostic analysis of the different layers that is equivalent to the more destructive chemical ones,” the deputy project coordinator Dr. Anastasios Drosou explains.

The sensor data is combined with predictive modeling technologies using AI, whose deep learning algorithms simulate the artifact’s appearance as it ages. Therefore, it can suggest conservators with accordingly treatments to protect the artifacts before their conditions get more terrible, or to minimize the existing damages and keep their current looks. According to Dr. Drosou, when the system detects a scratch or some color fading in a painting, the system would advise the conservators to apply a particular chemical on the scratch, or paint or cleaner in the other area.

Digitalization

Another beneficial and impactful progress that technology brings to the field is the digitalization of cultural heritages, where laser scanning and photogrammetry play essential roles. Having digital cultural heritage will help record history, establish a digital library for researchers and future generations to learn about them, and enrich cultural vibrancy.

The Mogao Caves, located in Gansu province in northwestern China, with over 2,000 sculptures and 45,000 square meters of unique and fragile painted murals, all of which are between 600 and 1,600 years old, is facing the challenge of preserving them in their current conditions. 

The caves are great demonstrations of the history of the Silk Road as well as related commercial trades and cultural exchanges across countries. However, due to thousands of years of natural disasters and damages, as well as the overloaded tourists after its development and reopening as a historical visiting site, the speed of painted statues damage and fresco degradation of Mogao Cave is dramatic.

To better preserve this UNESCO World Heritage Site, the researchers and scientists in China decided to use Laser Scanning and Multi-baseline Rotating Photogrammetry to turn the statues and frescos into digital formats, and store them in a safe data library for future studies and possible virtual exhibitions to let people appreciate the caves on their mobile devices with the similar experience as if they were in the caves.

• Photogrammetry: a sophisticated process by which information is extracted from photographs to create accurate three-dimensional maps and models. Using ultra-high-resolution aerial photographs, this practice combines UAV-mounted overhead sensors with powerful GIS mapping systems to create dynamic, measurable documents for a number of real-world situations and uses. (Source: Science Direct)

• Laser Scanning: an automatic, direct measurement of 3D points that can accurately measure and collect data from objects, surfaces, buildings, and landscapes. Laser scanners collect information in the form of point cloud data, which consists of millions of 3D coordinates (XYZ coordinates). (Source: Science Direct)

Another historical site that has been damaged and needed further protection and repair is the Notre-Dame Cathedral which was burnt by fire in 2019. The government is planning for its reconstruction and says hoping to finish by 2024.

Thanks to the digital scanning that they previously did before the fire, the team are now able to create a 3D BIM model of the Notre-Dame Cathedral as it existed before. Then, using recent reality capture scans to create the post-fire 3D model using a 3D design software called Autodesk, and comparing these two versions to rebuild the Cathedral.

Since the roof which was the most burnt part of the Cathedral was made of wood, this cultural tragedy rings the alarm for other historical buildings with wood structures. Performing 3D scanning and modeling to have digital forms of the heritages on file would be a great kind of look-ahead preparation for future possible accidents.

Conclusion

AI and other advanced technologies can help with the restoration and digitalization of tangible cultural heritage. They work as assistants of people on those works that cannot be achieved by human abilities or are time-consuming. They also work as tools for people to utilize and advance their working process to a higher level.

We’ve already seen some great progress that utilizing technologies in cultural heritages’ protections from the above projects and examples. In the future, we can also have a positive prospect that technologies would be able to make more achievements in this area by human inventions and under human supervision.

Bibliography

“AI for Cultural heritage.” Microsoft. Accessed February 25, 2022. https://www.microsoft.com/en-us/ai/ai-for-cultural-heritage.

“An Introduction to Drone Photogrammetry.” Mapware. Accessed March 4, 2022. https://mapware.ai/an-introduction-to-drone-photogrammetry/?gclid=CjwKCAiAjoeRBhAJEiwAYY3nDBfZ-wtHG1wd1Xv4sfiGfTEjObRb2Pm-CkaRFOXQycgG5RP7R-1R0BoCm-cQAvD_BwE#whatisutm_term=photogrammetry&utm_campaign=Traffic_Clicks_Global+%5BJoe%5D&utm_source=adwords&utm_medium=ppc&hsa_acc=7859616296&hsa_cam=15446499960&hsa_grp=136083379128&hsa_ad=566158081175&hsa_src=g&hsa_tgt=kwd319349969229&hsa_kw=photogrammetry&hsa_mt=p&hsa_net=adwords&hsa_ver=3.

Brandon, Elissaveta M. “How AI and Robotics are Reconstructing A 2,000-year-old Fresco in Pompeii.” Fast Company. December 23, 2021.

https://www.fastcompany.com/90708962/how-ai-and-robotics-are-reconstructing-a-2000-year-old-fresco-in-pompeii.

“Cutting Edge | Protecting and Preserving Cultural Diversity in the Digital Era.” UNESCO. October 28, 2020. https://en.unesco.org/news/cutting-edge-protecting-and-preserving-cultural-diversity-digital-era.

Denbo, Seth. “Digitalizing Cultural Heritage: Historic Preservation and The Mogao Caves in China.” Perspectives on History. December 1, 2016. https://www.historians.org/publications-anddirectories/perspectives-on-history/december-2016/digitizing-cultural-heritage-historic-preservation-and-the-mogao-caves-in-china.

“‘Digital Surrogates’ Assist in the Preservation of Cultural Artefacts.” CORDIS. February 19, 2019. https://cordis.europa.eu/article/id/248968-digital-surrogates-assist-in-the-preservation-of-cultural-artefacts.

Gandhi, Poonam. “World Heritage – Why Preserve, Conserve and Protect?” TravelWorld.com. April 20, 2021. https://travel.economictimes.indiatimes.com/speakeasy/world-heritage-day/4891.

“How Technology Is Preserving Our Cultural Heritage.” Reader’s Digest. Accessed February 25, 2022. https://www.readersdigest.co.uk/inspire/life/how-technology-is-preserving-our-cultural-heritage.

Horgan, Rob. “How Digital Technology is Aiding Notre Dame Rebuild, Two Years since Tragic Blaze.” New Civil Engineer. April 15, 2021. https://www.newcivilengineer.com/latest/how-digital-technology-is-aiding-notre-dame-rebuild-two-years-since-tragic-blaze-15-04-2021/#:~:text=In%20a%20bid%20to%20aid,existed%20before%20the%20catastrophic%20event.

Ibaraki, Stephen. “Artificial Intelligence For Good: Preserving Our Cultural Heritage.” Forbes. March 28, 2019. https://www.forbes.com/sites/cognitiveworld/2019/03/28/artificial-intelligence-for-good-preserving-our-cultural-heritage/?sh=bf2912f4e960.

“Laser Scanning Reveals Cathedral’s Mysteries | National Geographic.” National Geographic. June 23, 2015. Video, 4:10. https://youtu.be/jAi29udFMKw.

“Laser Scanning.” ScienceDirect. Accessed March 4, 2022. https://www.sciencedirect.com/topics/earth-and-planetary-sciences/laser-scanning.

Pinkowski, Jen. “Pompeii’s Ruins to Be Reconstructed by Robot.” Scientific American.December 6, 2021. https://www.scientificamerican.com/article/pompeiis-ruins-to-be-reconstructed-by-robot/.

“RePAIR Project.” Pompeii Sites. September 2, 2021. Video, 4:49. https://youtu.be/v9rpoLL5eLM.

“Reconstructing the Past: Artificial Intelligence and Robotics Meet Cultural Heritage.” CORDIS. Accessed February 25, 2022. https://cordis.europa.eu/project/id/964854.

“The RePAIR Project Begins – Robotics and Digitisation at the Service of Archaeology.” Pompeii. Accessed February 25, 2022. http://pompeiisites.org/en/comunicati/the-repair-project-begins-robotics-and-digitisation-at-the-service-of-archaeology/.

Werkheiser, Greg. “Artificial Intelligence: Cultural Heritage Required.” Cultural Heritage Partners. Accessed February 25, 2022. http://www.culturalheritagepartners.com/artificial-intelligence-cultural-heritage-required/.

“What Is Laser Scanning and How Can It Be Used?” TopoDot. June 11, 2020. https://blog.topodot.com/what-is-laser-scanning-and-how-can-it-be-used/.

Wiggers, Kyle. “Microsoft Launches AI for Cultural Heritage Program.” Venture Beat. July 11, 2019. https://venturebeat.com/2019/07/11/microsoft-launches-ai-for-cultural-heritage-program/.

Our world is becoming increasingly digital – in the past year, the metaverse and NFTs have become a household buzzword, causing a massive “gold rush” into developing bleeding-edge innovation surrounding immersive experiences.

Augmented Reality (AR) focuses on blending the physical and digital worlds together in cohesive experiences. AR overlays digital elements on top of the real-world, which “keeps the real world central but enhances it with other digital details, layering new strata of perception, and supplementing your reality or environment”. Perhaps the most well known example of this is Pokémon GO. Virtual Reality (a fully immersive digital experience that features computer-generated imagery and sounds that requires a head-mounted display, headset and haptic controllers) and Mixed Reality (experiences where users can interact with both physical and digital items and environments at the same time), concepts that are highly associated with AR, are often looped into the same conversation. 

Having said that, the majority of the fashion products and experiences currently available on the market primarily utilize AR; this article will mainly be focusing on how more established, traditional brands are embracing this immersive technology in comparison to those born in the Web3 era.

Traditional Fashion Labels:

Perhaps one of traditional fashion labels doing the most with AR is Louis Vuitton. In the past couple of years, they have launched a variety of AR products, collaborating with the likes of Dr. Helen Papagiannis, one of the most influential figures in the AR is. In addition to having worked with the technology and industry leaders in the space for over a decade, she has explored NFTs, had her work displayed in Time Square, and collaborated with some of the largest luxury fashion labels today. In October of 2021, her work was featured as 1 of 200 visionaries in Louis Vuitton store windows globally, where she had the opportunity to reimagine LV’s iconic trunk as “an abundant future garden blooming with possibilities and pulsing with life” through a mixed reality experience. The trunks (both physical and digital) will be on auction at Sotheby’s in 2022.

Louis Vuitton has also explored other digital projects and avenues. Other projects have included a Travel Book collection that acts as an invasion to real and virtual voyages. In an edition published in 2019, the book pages would become animated with sound when scanned with the Google Lens mobile app. Additionally, in a 2019 collaboration with League of Legends, Louis Vuitton released AR Instagram filters that would allow users to try on co-branded accessories.

In more recent news, during this year’s New York Fashion week, label Maisie Wilen presented their latest collection with 7-foot-tall holographic models. The virtual models were lined up and “performed” repetitive loops and featured retro futuristic clothes, created by 106 cameras around the models capturing 360 imaging in super sharp 6K resolution to create a “living GIF lookbook”. Schloss, the designer behind the collection, explained that by utilizing digital presentations and taking a hybrid approach, she would be able to access home viewers as well as in-person attendees. Additionally, the collection is available through their shoppable site, which also features AR versions of the models that allows fans of the brand to play with the digital assets in their own environments.

Burberry also released an A.R. app in 2021 that would allow users to customize sculptures of Burberry’s pocket bags using their designer patterns, prints, and shapes from the “In Bloom” spring/summer 2021 collection and place them virtually within their home. This was in continuation with their previous efforts of launching an AR shopping tool through Google Search technology, that would allow consumers to see an AR version of the product at scale within the context of their own environments. Additionally, Burberry also launched an exclusive AR experience activated through QR codes that was available in their flagship store opening in Ginza, Tokyo as well as a digital pop-experience powered by Google Lens.

Similarly, even retailers such as Farfetch and Goat, platforms that sell products from a variety of boutiques and brands are partnering with AR technology companies to produce their own versions of virtual try-on technology. Both have launched AR experiences within their respective apps to let users try on a range of sneakers from brands such as Gucci, Versace and Off White.

Unlike some of its competitors, Chanel’s immersive technology experience focuses on make-up, rather than fashion elements. Their product utilizes a Lipscanner, which uses AI to take any image and match it to a lip color from the brands selection of over 400 products. The patented technology incorporates “intensive algorithm training” and can identify exact shades and textures from images to render AR products onto the customers’ faces.

According to Erin Schmidt from Coresigh Research, “Virtual try-ons drove over twice the engagement, as well as higher conversion and retention rates, doubling the number of sessions from the previous quarter in its second quarter earnings report of 2021, dated 5 Feb., 2021”. In addition to all of the hype and attention related to Web3 and NFTs, with these discovered KPIs, it is no surprise that traditional brands have been implementing this technology within their branding and marketing experiences.

Web3 Fashion Labels:

In addition to the slew of established luxury brands exploring immersive technology, there have also been a number of newly formed companies born in the Web3 space that fully embrace it and rely on this technology for their core products.

AUROBOROS, a London based brand, is a fashion label focused on creating digital couture. Consumers can buy digital garments from their website by sending them a clear photo in good lighting. Their team then applies the garment onto the customer’s photos by customizing their “made-to-measure” digital clothes precisely to their client’s body shape. In 2021, the brand debuted their inaugural ‘Biomimicry’ collection at London Fashion Week’s DiscoveryLAB; consumers could scan a QR code featured on posters and billboards around London that unlocked an AR try-on experience on Snapchat.

As of now, digital clothing marketplace DressX embraces similar concepts and ideas as AUROBOROS to allow consumers to buy their garments on their website. The difference however, is that DressX focused on selling clothing from a variety of designers and brands at varying price points. The brand raised a seed round of $2 million, and has been developing an app that focuses on creating digital try-on experiences with AR, to ultimately replace the current buying experience. Co-founder Natalia Modenova explained that their clothing is “ultimately designed to be compatible across different universes online” including Zoom.

RTFKT studios is another brand that has explored AR technology, primarily through digital streetwear and sneakers. As part of their marketing, the brand creates viral content on social media that features their real-time AR try-on, enabled by recent hardware and software companies from Snapchat and Apple. By capitalizing on the hype from social media, the brand has been able to sell NFTs of their digital goods for as much as $15,000 USD. (Refer to Elon Musk wearing the “CYBERSNEAKER”). Their website also features 3 Snapchat filters than anyone can try by scanning a QR code, featuring try-on shoes, a massive meta pigeon that can appear in any environment, and their puffer meta jacket. The brand has since shifted their focus more to CloneX, an NFT profile picture project. Shortly after, it was announced that Nike acquired RTFKT as a step towards embracing the metaverse and the ever growing digital creative community.

Conclusion:

After exploring a variety of traditional fashion houses and newer Web3 brands, it is clear that companies value AR experiences and are focused on continuing to develop these technologies. Having said that, immersive experiences are still at its infancy. The majority of the experiences related to the space have been AR try-ons, and almost none of them incorporate VR and MR in substantial ways. In efforts to speed up development, Meta hosted their first Luxury Innovation Summit in 2021 and brought together 50 top luxury executives to discuss the potential of virtual and augmented reality for their brands. In addition to showcasing examples of how VR can enable potential clients to engage with the brand in new ways, Meta is also investing in tools to help brands have higher conversion rates. 

As more interest, hype and money continues to move into the metaverse and Web3, so will the technology related to immersive experiences. We are just getting started.

Resources:

• Artnet News. “Burberry Just Released an A.R. App That Lets You Design Your Very Own Virtual Sculptures Featuring the Brand’s Iconic Imagery.” Artnet News, Artnet News, 7 June 2021, https://news.artnet.com/style/burberry-ar-app-1977138.
  
• “Auroboros Presents Biomimicry Digital Collection at London Fashion Week In Partnership with IODF.” YouTube, Auroboros , 12 June 2021, https://www.youtube.com/watch?v=ldOUbyCxIRI.
  
• Chitrakorn, Kati. “An Augmented Reality Shopping Boutique Is Here.” Vogue Business, 19 Feb. 2021, https://www.voguebusiness.com/consumers/augmented-reality-store-machine-a-institute-of-fashion-design.
  
• Facebook, Vogue Business for. “The Augmented and Virtual Reality Opportunity for Luxury.” Vogue Business, Facebook, 16 June 2021, https://www.voguebusiness.com/technology/the-augmented-and-virtual-reality-opportunity-for-luxury.
  
• Ferere, Cassell. “Snap Inc.. and Complexcon Are Using Augmented Reality to Allow Attendees and Fans at Home to Shop.” Forbes, Forbes Magazine, 5 Nov. 2021, https://www.forbes.com/sites/cassellferere/2021/11/05/snap-inc-and-complexcon-are-using-augmented-reality-to-allow-attendees-and-fans-at-home-to-shop/.
  
• Jasman, Asri. “Try on off-White Sneakers Virtually with the Farfetch App.” Esquire SG, 14 Aug. 2020, https://www.esquiresg.com/farfetch-virtual-try-on-off-white-sneakers-wannaby-ar-technology/.
  
• McDowell, Maghan. “Why Ar Clothing Try-on Is Nearly Here.” Vogue Business, 26 July 2021, https://www.voguebusiness.com/technology/why-ar-clothing-try-on-is-nearly-here.
  
• “Nike, Inc.. Acquires RTFKT.” Nike News, 13 Dec. 2021, https://news.nike.com/news/nike-acquires-rtfkt.
  
• Nowill, Rob. “Is It Time to Invest in Virtual Fashion?” HYPEBEAST, HYPEBEAST, 25 Jan. 2021, https://hypebeast.com/2021/1/virtual-ar-fashion-tribute-rtfkt-aglet.
  
• ​​Onufrowicz, Martin. “AUROBOROS Is the Tech Fashion House Creating Celestial Cyber Couture.” Dazed, 12 Feb. 2021, https://www.dazeddigital.com/fashion/article/51913/1/auroboros-celestial-couture-alexander-mcqueen-vr-technology-digital-fashion.
  
• Palumbo, Jacqui. “Digital Dress Codes: What Will We Wear in the Metaverse?” CNN, Cable News Network, 15 Dec. 2021, https://www.cnn.com/style/article/metaverse-digital-fashion/index.html.
  
• Palumbo, Jacqui. “New York Fashion Week Show Staged with 7-Foot-Tall Holographic Models.” CNN, Cable News Network, 15 Feb. 2022, https://www.cnn.com/style/article/maisie-wilen-fall-2022-nyfw-holograms/index.html.
  
• Papagiannis, Helen. “XR Goes Pop.” XR Goes Pop, 6 Feb. 2022, https://xrgoespop.com/.
  
• Scott, Nateisha. “Chanel’s New Lipscanner Technology Is Proof That Virtual Reality Beauty Testing Is Here to Stay.” Yahoo!, Yahoo!, 22 Feb. 2021, https://uk.style.yahoo.com/chanels-lipscanner-technology-proof-virtual-101436907.html.
  

Many industries have begun utilizing 3D printing to make work easier and more efficient. This technology is even making its way into the arts. Broadway theatres, university theatre departments, and theatres everywhere are implementing 3D printing technologies to construct props, and even to build entire sets. 3D printing brings new elements of realism and creativity into theatre. It is the future of stage and set design, by enabling mass customization of specific designs, independent of outside factors, such as time constraints and availability. The market for 3D printers in theatre has really changed from DIY projects to machines for professionals. 

How Does 3D Printing Work? 

3D printing (also called additive manufacturing) is the process of making three dimensional solid objects from a digital file. The creation of a 3D printed item involves additive processes, where an object is created by laying down successive layers of a material until the object is fully created. Each of the layers is a thinly sliced cross-section of the object. 3D printing enables the user to produce complex shapes using less material than traditional manufacturing materials. 

To create a 3D printed object, it starts with a 3D model. This can either be created from the ground up, or downloaded from a 3D library. After you have a printable file, the next step is to prepare the file for the 3D printer (slicing). Slicing is the process of cutting up a 3D model into hundreds or thousands of layers, and is done with a slicing software. After the file is sliced, it is ready to be inserted into the 3D printer, which can be done via USB, SD, or wi-fi. The file will then be 3D printed layer by layer. 

3D printing technology used to only be used for prototyping and one-off manufacturing in early stages, but now is transforming into a widely-used production technology. 3D printing has become destined to infiltrate many industries, even the arts. 

The First 3D Printed Set

The first entirely 3D printed set was for a show called “Fra Diavolo,” at the Opera Theater in Rome, Italy. The set was created by a 3D printing company in Rome, WASP, and took around 3 months to complete. It was made using cheaper material than what is usually used in 3D printing, and a total of five printers. To begin the process, the scenographer gave WASP a 3D printed model of deformed historic buildings, that was divided into 223 pieces. This model was then used to create a lifesize model that could be used for the show’s set. 

3D Printing Takes Broadway

Broadway scenic designers, Kacie Hultgren and John Lee Beatty, know that 3D printing technology has revolutionized the modern-day theatre world. One of their first tasks for each show is to create a miniature model of the set to show to the director, and the rest of the team, what the scenic environment will look like. Usually, creating this miniature model is tedious, and takes around one week. But, with 3D printing, the models are created fairly quicker, and are made out of primarily plastic. The printer can create a fully-formed prototype of any object, and it doesn’t take nearly as long as it used to. It is also common to need more than one of the same piece. With 3D printing, the pieces are able to be made completely identical. For the production of ‘Outside Mullingar,’ Hultgren was able to create large sections of entire sets of furniture. The desktop printers used for Broadway shows are large, and able to print entire scenic units to scale. 3D printing technology has been used to create sets for productions such as Roundabout Theatre Company’s 2012 production of ‘Don’t Dress for Dinner,’ ‘The Nance,’ ‘Orphans,’ and ‘The Snow Geese.’ 

3D Printed Theatre Props

Although building an entire set with 3D printing technology is not all that common yet, 3D printing props is a quick, easy, and cheaper way of creating exactly what you need—a World War II-era handgun or a skull-shaped candy dish. It is an easier and cheaper way to acquire items that are longer being produced, such as brooches or hatpins for a period piece. It gives the set designer(s) a way to be more flexible and creative with their set design, and the team can be involved throughout the entire process. You can walk away and work on other things while printing, customize the props however you’d like, and go back and fix little details if needed. You can also use 3D printing to create miniatures on a small-scale model to save time instead of carving out all of the little details. The process, depending on how complicated a prop’s design is, can take 20 minutes to up to a couple of hours. 3D printing is also helpful if a prop needs to be broken in a scene. There can be multiple replacement items to replace it. The 3D printer is becoming an important tool in a theatre designer’s tool bag. And it’s now being taught to future generations of theatre professionals. 3D printing skills are giving them a competitive advantage in the job market. 

Implementing 3D Printing into Theatre Education

Not only is 3D printing being used in professional theatre, but it’s also being taught to future theatre technicians and scenic designers at universities, such as the University of Lynchburg, Baylor University, University of Pennsylvania, Arizona State University, Trinity College Dublin, and Royal College of Art. 

University of Lynchburg (Lynchburg, Virginia) 

Christopher Otwell, assistant professor of design and technical director at the University of Lynchburg, is teaching students to create props in a 3D printing course. Students use open-source software, such as Tinkercad and Simplify3D, to make props with the theatre department’s 3D printer. For one of their first projects, they were instructed to replicate an Old Bay seasoning jar. They have also made luggage tags, rubber ducks, and a heart-shaped candy box complete with plastic candies. After designing the props in Tinkercad, they then use Simplify3D to find out how many layers are involved, how much the project will cost, and how long it will take to print each item. The course is a win-win situation for both the Otwell and the students. The students learn a new skill, and Otwell is able to delegate prop making work for productions. 3D printing is making a direct impact on the university’s theatre productions. 

Baylor University (Waco, Texas) 

For Baylor University’s production of ‘Into the Woods,’ the team used a 3D printer to create synthetic beans, mushrooms, and more. Using 3D printing software, assistant professor of theatre arts at the university, Joe Kucharski, was able to tug, flatten, and pinch a digital “ball of clay” into the desired shape, two dozen beans and a dozen mushrooms for the Witch’s dress.

University of Pennsylvania (Philadelphia, Pennsylvania) 

At the University of Pennsylvania, former lecturer of theatre arts, Eric Baratta, is starting to incorporate 3D printing into courses using a software called SketchUp. Students were assigned to look at the first scene of Shakespeare’s Hamlet, specifically the elements of design (line, dimension, movement, light, color, and texture) and composition (unity, harmony, contrast, variation, balance, proportion, emphasis, and rhythm/movement) to build virtual scenes. They finalized their designs and made sure they were optimized for 3D printing. The simplicity of the students’ designs carried through elegantly to the physical model, and the projects were all extremely successful.

Arizona State University (Tucson, Arizona) 

Arizona State University’s, ‘Makerspace,’ is designed for students to collaborate and explore new skills. Students used the software, MakerBot Replicator+, to create costumes for a production of the Greek tragedy, ‘Ajax.’ Involved in this production are costumes with teeth. With their 3D printing technology, they were able to scan the teeth and print them in-house, decreasing their budget, and providing the students with a new learning opportunity. The use of a 3D printer has created an open and collaborative community at the university.

Trinity College Dublin (Dublin, Ireland)

The Lir, National Academy of Dramatic Art at Trinity College Dublin’s courses offer unique, practice-based experiences for their students. The Academy has a partnership with Craftbot, a brand of 3D printing technology. Craftbot provided the college with two 3D printers to enhance the learning possibilities for their students. Maree Kearns, Head of the Master of Fine Arts in Stage Design, notes that the printers are not only used for set design, but also costume design. 3D printing can be used to make fabric and fabric layovers, accessories, medals, pins, and buttons for costumes. The 3D printers are accurate and simple to use, and are compatible with multiple types of filaments, such as plastic, wood resin, or metal. Professors at the college know that as 3D printing becomes more mainstream, students need to be able to use it. 

Royal College of Art (London, England)

Royal College of Art’s production of ‘Farewell My Concubine,’ featured actors wearing unique pleated costumes created a selective laser sintering (SLS) 3D printer. Students who worked on this project created algorithms that could be read by printer software to generate pleated fabric to contour the actors’ bodies. This type of 3D printing technology uses these created algorithms to selectively harden parts of a mass powder in a vat. After curing by a laser, the pleated design remains intact and the excess powder is removed. This project was groundbreaking for the school, and taught the students a more complicated process of 3D printing technology. 

Conclusion

3D printing technology is becoming more and more widely used in the theatre industry. There have been many projects that have paved the way for a collaboration between theatre and this technology. 3D printing is transforming the industry, and making the process of set construction faster, cheaper, and higher quality. 

References

Andreoli, Maurizio. “The first 3d printed Scenography.” Wasp. September 29, 2017. https://www.3dwasp.com/en/the-first-3d-printed-theatre-scenic-design/.

“Autodesk Tinkercad.” Tinkercad. Accessed March 4, 2022. https://www.tinkercad.com/. 

“Complete 3D Printing Solutions Customized to Your Experience.” Craftbot. Accessed March 4, 2022. https://craftbot.com/. 

“Five Industries Utilizing 3D Printing.” Markforged. Accessed March 4, 2022. https://markforged.com/resources/blog/five-industries-utilizing-3d-printing.

Gentry, Bryan. “3D printer brings ideas to life for theatre students.” University of Lynchburg. March 12, 2018. https://www.lynchburg.edu/2018/03/3d-printer-brings-ideas-to-life-for-theatre-students/.

Goodrich, Terry. “Sculpting Costumes with 3D Printers is ‘the Way Theater is Headed,’ Theatre Arts Chair Says of Baylor’s New Technology.” Baylor University. October 8, 2014. https://www.baylor.edu/mediacommunications/news.php?action=story&story=147219.

“Improve Your Print Quality With Powerful 3D Printing Software.” Simplify3D. Accessed March 4, 2022. https://www.simplify3d.com/. 

“Lighting Up the West End With High Accuracy 3D Printing.” Formlabs. August 5, 2021. https://formlabs.com/blog/lamp-pencil-3d-printing-entertainment-light-design/.

“MakerBot Replicator+.” Trimech. Accessed March 4, 2022. https://trimech.com/products/makerbot-replicator. 

Ramsey, David. “Surprising Ways a Performing Arts Department Can Use 3D Printing.” TriMech. August 28, 2020. https://blog.trimech.com/surprising-ways-performing-arts-department-can-use-3d-printing.

Scully, Leah. “3D Printed Costumery Creates Juxtaposition between Culture, Theater, and Tech.” MachineDesign. November 9, 2017. https://www.machinedesign.com/3d-printing-cad/article/21836155/3dprinted-costumery-creates-juxtaposition-between-culture-theater-and-tech. 

Simonson, Robert. “3D Printers Revolutionize Work of Broadway Scenic Designers.” Playbill. March 2, 2014. https://www.playbill.com/article/3d-printers-revolutionize-work-of-broadway-scenic-designers-com-214428.

Somers, Ira. “State Design Just Got Easier with 3D Printing.” Massivit 3D. July 19, 2018. https://massivit3d.com/blog/stage-design-just-got-easier-with-3d-printing/.

“Theatrical plays with 3D printed props and decorations.” Craftbot. Accessed February 25, 2022. https://craftbot.com/casestudies/The-Lir-Academy. 

“What is 3D Printing?” 3D Printing. Accessed March 4, 2022. https://3dprinting.com/what-is-3d-printing/. 

“Where great ideas get to work.” SketchUp. Accessed March 4, 2022. https://www.sketchup.com/

Witherel, Claire. “3D Printing Meets Theatre Set Design @ Education Commons.” Penn Teaching, Research & Learning. May 16, 2018. https://penntrl.wordpress.com/2018/05/16/3d-printing-meets-theatre-set-design-education-commons/.

What is Google Arts and Culture?

Launched by Google in 2011 as a nonprofit initiative, Google Arts and Culture was created with the purpose of increasing the accessibility of art and culture. The platform describes itself as a new way of experiencing art and culture, with the main attraction being that the user is able to access various types of art regardless of the their location, or the location of an institution. The platform currently has over 2,000 global partners and over 100,000 works of art, giving museums, performing arts venues, and historical landmarks the ability to showcase various exhibitions, photographs and information about their institutions. Some prominent institutions include The British Museum, The Museum of Modern Art (MoMA) and the Sydney Opera House. When an institution becomes a partner, they obtain access to tools that allow for paintings and other materials to be digitized and uploaded to the platform. The Art Camera, which performs the digitization, is a high-resolution camera that allows for users of the platform to zoom in and observe up-close details.

Another digitization feature allows institutions to add virtual, 360 degree tours to their pages, similar to a view you would see in Google Maps.

AI, AR, and Experiments

Building off a substantial number of partnerships, Google Arts and Culture utilizes technologies such as artificial intelligence (AI)/machine learning and augmented reality (AR) for a variety of experiments that allow the user to establish personal connections to art, primarily by participating in the creative and art-making processes themselves.

One of the platform’s most prominent experiments is its selfie feature. Using the Google Arts and Culture app, users are able to use selfies to see what piece of art they resemble. When the user uploads their photo, Google’s software works to scan the selfie and go through the variety of paintings in the databases for a potential match. Though created with good intentions, this experiment did not come without controversy. Many BIPOC individuals found that when they used the feature, their results did not come with the same depth as those of White individuals; the works of art that displayed people of color were already limited, and when they were matched, it was often with figures that displayed “stereotypical tropes.”

This is not a unique instance of artificial intelligence reflecting human biases; Google as a larger enterprise—along with the artificial intelligence industry in general—has come under fire for its general lack of diversity. The reproduction of racist stereotypes is not necessarily the algorithm’s fault, but rather the fault of the people behind the algorithm. Some have also acknowledged how the platform’s mishap specifically reflects the lack of diversity in the art world.

In a less harmful way of establishing a personal connection to art, Google Arts and Culture introduced its Pet Portraits feature, which is also utilizes AI. The process is similar to that of the selfie feature: the user takes a picture of their pet, and the algorithm attempts to find an artistic match. AI has also been used for the purposes of preservation. Along with one of their partners, the Belvedere Museum in Vienna, Austria, they utilized machine learning to restore the paintings of Austrian painter Gustav Klimt in color.

Google Arts and Culture also makes prominent use of augmented reality (AR). The video below, uploaded by NASA and displayed on Google Arts and Culture’s website, utilizes AR and allows the user to “fly through a galaxy.”

Additionally, using the above painting, Fruto No. 1, I was able to use the Art Projector feature on the app, which lets you see what an piece of art would look like up close.

The leaders behind Google Arts and Culture, and the larger Google Cultural Institute, work regularly to explore and encourage the uses of AI and AR in the art-making process. In partnership with the Barbican Centre in London, a virtual exhibit detailing the “evolving relationship between humans and technology” and its implications for art, culture, and other aspects of our daily lives, is available to users.

Experiments such as the Blob Opera are fun way to receive deeper exposure to certain art forms (in this case, opera). There is also the Assisted Melody experiment, which allows the user to engage in a special music project. The user is able to write their own music, and the machine transforms it to make it sound like that of famous composer. On a different side of the musical spectrum is the Hip Hop Poetry Bot, an AI that was trained on rap and hip hop music, designed as a way to create a deeper appreciation for the art form. In my opinion, experiments such as these play a large part in regards to the access model that is integral to Google Arts and Culture’s platform and offerings.  

Disruptions and Implications

A form of disruption that I believe affects the art world is the aspect of personalization. In addition to the variety of AI and AR-based experiments that were mentioned earlier, a user can essentially curate their own art collections without having to leave their homes to do so. Because there are works of art from nearly every segment of the world, you can regularly see works of art that you otherwise might have never seen or heard of.

My initial perception of the Google Arts and Culture platform was that its presence would disrupt the visual arts world more than that of the performing arts. While the platform does showcase performing arts venues such as the Sydney Opera House and the Dance Theatre of Harlem, it is difficult to incorporate the entire experience of attending a ballet or a theatre production in an app such as Google Arts and Culture. At the same time, I still believe that many people visit museums not just to see the art, but to participate in the entire experience of being able walk around the physical building and engage with others who have come for a similar purpose.

To that end, I think that Google Arts and Culture serves more as a supplement, as opposed to a replacement, for cultural institutions. By utilizing the platform, a user is able to have a better idea of what they would like to see and experience if they were to visit the institution in person. In a way, the platform can serve as a marketing tool for partner institutions, as many of them do not publish the entirety of the collections that reside in their physical buildings. For larger artistic and cultural institutions, it could be worthwhile to look into establishing a partnership with the platform. It could also benefit smaller institutions, as your exhibitions and what you have to offer can be put on display for users from all around the world to see.

On the patron end, it allows users and potential visitors to be more informed of what the experience at a particular museum may be like. Because the app and the website are easily accessible, Google Arts and Culture lives up to its purpose of providing a new way of experiencing art, culture, and history: the app and website are both easily accessible, enabling users to engage in and view art whenever they want.

References

“12 Museums From Around the World That You Can Visit Virtually | Travel + Leisure | Travel + Leisure.” Accessed March 4, 2022. https://www.travelandleisure.com/attractions/museums-galleries/museums-with-virtual-tours.

“About Google Cultural Institute.” Accessed March 4, 2022. https://about.artsandculture.google.com/experience/.

“About Google Cultural Institute.” Accessed March 4, 2022. https://about.artsandculture.google.com/partners/.

“AI: More than Human — Google Arts & Culture.” Accessed February 25, 2022. https://artsandculture.google.com/project/ai-more-than-human.

“Assisted Melody by Simon Doury, Artist in Residence at Google Arts & Culture Lab with Google Magenta – Experiments with Google.” Accessed February 25, 2022. https://experiments.withgoogle.com/assisted-melody.

Bustle. “Google’s ‘Arts & Culture’ App Is Being Called Racist, But The Problem Goes Beyond The Actual App.” Accessed February 25, 2022. https://www.bustle.com/p/googles-arts-culture-app-is-being-called-racist-but-the-problem-goes-beyond-the-actual-app-7929384.

Google. “Exploring Art (through Selfies) with Google Arts & Culture,” January 17, 2018. https://blog.google/outreach-initiatives/arts-culture/exploring-art-through-selfies-google-arts-culture/.

“Google Used AI to Recreate Gustav Klimt Paintings Burned by Nazis.” Accessed February 25, 2022. https://mashable.com/video/gustav-klimt-ai-paintings-google-arts-and-culture.

Google. “How Artists Use AI and AR: Collaborations with Google Arts & Culture,” May 24, 2019. https://blog.google/outreach-initiatives/arts-culture/how-artists-use-ai-and-ar-collaborations-google-arts-culture/.

“Google Arts & Culture.” Accessed February 25, 2022. https://artsandculture.google.com/.

Google Arts & Culture. “Art Camera.” Accessed March 4, 2022. https://artsandculture.google.com/project/art-camera.

Google Arts & Culture. “Blob Opera.” Accessed March 4, 2022. https://artsandculture.google.com/experiment/blob-opera/AAHWrq360NcGbw.

Google Arts & Culture. “Fruto No. 1 – Soldedad Beltrán.” Accessed March 4, 2022. https://artsandculture.google.com/asset/fruto-no-1-soldedad-beltrán/pgHEHsf8R2zbCg.

Held, Amy. “Google App Goes Viral Making An Art Out Of Matching Faces To Paintings.” NPR, January 15, 2018, sec. America. https://www.npr.org/sections/thetwo-way/2018/01/15/578151195/google-app-goes-viral-making-an-art-out-of-matching-faces-to-paintings.

“Hip Hop Poetry Bot by Alex Fefegha in Collaboration with Google Arts & Culture – Experiments with Google.” Accessed February 25, 2022. https://experiments.withgoogle.com/hip-hop-poetry-bot.

Metz, Cade. “Who Is Making Sure the A.I. Machines Aren’t Racist?” The New York Times, March 15, 2021, sec. Technology. https://www.nytimes.com/2021/03/15/technology/artificial-intelligence-google-bias.html.

Muriuki, Purity. “What Is Google Arts & Culture?,” July 17, 2021. https://startup.info/what-is-google-arts-culture/.

NASA Video. Orion Nebula – 360 Video, 2018. https://www.youtube.com/watch?v=1-goEmM0c4Q.

“Pet Portraits — Google Arts & Culture.” Accessed March 4, 2022. https://artsandculture.google.com/camera/pet-portraits.

Schofield, Matthew. “LibGuides: Advanced Google: Google Arts and Culture.” Accessed February 25, 2022. https://library.sfc.edu/Google/artsandculture.

“Sydney Opera House, Sydney, Australia — Google Arts & Culture.” Accessed March 4, 2022. https://artsandculture.google.com/streetview/sydney-opera-house/KgGPW2YWtcHpBg?sv_lng=151.2135034733324&sv_lat=-33.85845058282505&sv_h=30.36855573629456&sv_p=2.382047067182725&sv_pid=opBj2MqN2Mk4yX5QdZPYDw&sv_z=1.

TechCrunch. “Why Inclusion in the Google Arts & Culture Selfie Feature Matters.” Accessed February 25, 2022. https://social.techcrunch.com/2018/01/21/why-inclusion-in-the-google-arts-culture-selfie-feature-matters/.

The arts market is one with a constantly shifting value, which auction houses capitalize on in order to drive interest in the pieces that they offer. Depending on the quality, creator, provenance, and history of each piece, expected prices can vary wildly based on the current market. David Bellingham encapsulates the unique nature of the auction business in his chapter, “The Auction Process,” from Art Business Today: 20 Key Topics:

Art auctions differ from other auctions in that each commodity is unique, and therefore an absolute ‘eternal’ value can never be established: the value of each work can only be defined by what an individual was prepared to pay for it at the time of the auction. – David Bellingham

Thus, auction houses are constantly adapting their practices to meet customers’ interests while setting the value of pieces appropriately in the current market. Bellingham goes on to describe how this process is particularly difficult with contemporary art. When pieces appear too quickly at auction, it can be a sign of a bubble, where the pieces will be valued highly at the moment but hold less future value once interest drops. As many customers are looking for pieces that are healthy investments, contemporary art pieces, such as digital works, can pose a risk for auction houses and their continued relationships with customers. However, auction houses are generally seen as reliable sources of high quality and valuable pieces, as their authentication methods are trusted and their prestige lends credibility to a piece’s value.

The value proposition of auction houses creates an interesting challenge for the arts market as a whole when it comes to adopting digital business methods. Because the arts have historically been a luxury market, and continue to be viewed as such, customers want intermediaries such as auction houses and galleries. Marios Samdanis describes this phenomenon in “The Impact of New Technology on Art,” from Art Business Today: 20 Key Topics. According to Samdanis, consumers seek out pieces from these intermediaries, rather than directly from producers, as they perceive a greater “symbolic value” when a piece is sold by a reputable intermediary. Samdanis, writing in 2016, described how digital tools were readily used to market pieces, while social media was being used to increase engagement with existing and potential customers. Technology increased the globalization of the arts market, both in procuring pieces from a wider variety of artists and selling to expanded markets. Partnerships, such as those between Sotheby’s and eBay and between Art Basel and Kickstarter, demonstrated the viability of reaching new audiences and connecting them to artists through technology. Audio, video, apps, and augmented reality were being used to increase viewer engagement and education. However, as of 2016, participants in the arts market were still hesitant to move to fully-digital methods, and digital art was still growing in popularity.

In 2020, however, the entire arts market was shaken by the COVID-19 pandemic. As with nearly every industry, participants in the arts world were forced to conduct their business virtually in the interest of public health and due to governmental restrictions on in-person work and gatherings. While the arts were adopting digital tools as described, no one expected to suddenly rely upon technology so heavily. Auction houses had to reconsider how to best connect to customers virtually, from marketing their collections to hosting auctions. Prior exploration of digital tools proved invaluable to continuing the business and reputation of these institutions.

However, an additional development in the arts world dramatically transformed the business of auction houses, arguably even more so than the shift to virtual business: an unprecedented increase in interest in digital media, particularly NFTs. This explosion was set in motion by the sale of Beeple’s EVERYDAYS: THE FIRST 5000 DAYS, sold by Christie’s for a record-setting $69,346,250 USD on March 11, 2021. This piece, shown below, is a compilation of Beeple’s daily work since 2007, when he first set out to create a new digital image every day. EVERYDAYS brings together these daily images, organized loosely by chronology, theme, and color, demonstrating the artist’s journey in both skill and subject matter.

An article written the week after the sale by Kelly Crow for the Wall Street Journal describes the intense impact of this sale on all auction houses: suddenly, all major institutions wanted to rapidly increase their sales of NFTs. The Beeple sale had demonstrated that this medium had value in the market, and auction houses were receiving calls from long-time customers and new audiences about buying NFTS. As Crow writes,

Marc Glimcher, president and chief executive of Pace Gallery, likened the atmosphere to a gold rush, saying he’s fielding 25 calls a day from longtime art collectors as well as newcomer cryptocurrency investors angling to get in on the NFT art craze. – Kelly Crow

Cryptocurrency was a major part of this digital craze. Beeple’s EVERYDAYS had been bought by a cryptocurrency millionaire from Singapore known as Metakovan, and as auction houses realized the market for NFTs, they also realized that this market was full of cryptocurrency investors who had an incredible amount of wealth, potential customers who largely were previously unconnected to the arts market. In another quote from Glimcher, he stated that arts institutions were “just now realizing that the crypto community is bigger and richer than the art world.” Arts brokers from all types of organizations, from galleries to auction houses, were investigating how to accept cryptocurrency and increase their portfolios of digital artists. Crow’s article, again written just six days after the sale of EVERYDAYS, demonstrates just how quickly the entire arts world realized the serious interest in digital media and was shifting to meet demands.

However, another article written the very same day expressed concerns with the “trendiness” of this market for NFTs and digital media. While the market demand for NFTs was taken seriously by auction houses and other brokers, there were plenty of doubts about the longevity of this demand. In his article, Connor Goodwin reflected on the growth in popularity of NFTs and particularly how non-traditional players factored into the demand. NFTs are able to be marketed and sold by nearly anyone, and so small organizations and artists themselves are able to directly market and sell their work. For example, Hashmarks, an online artist collective, was able to sell 16,384 NFTs for approximately $16 million on January 27, 2021. Hansen Wang, a co-creator of Hashmarks, reflected on how the success of this sale demonstrated how artist-consumer relationships can break down the hierarchy of the arts world, particularly removing middlemen like auction houses which slow down the sale of pieces and the transfer of funds. This very perspective explains why auction houses so rapidly jumped into selling NFTs and accepting cryptocurrency as payment: sales such as the one by Hashmarks generated enormous profits, and auction houses saw an opportunity to reach new consumers and get a piece of the NFT-money-pie. Goodwin continues to ask two questions: are NFTs smart investments? and are they good enough that the art world should really care? Goodwin expressed hesitancy in answering both of these questions. While the Beeple sale had certainly proved something, many participants in the arts world had doubts about the long-term viability of the NFT market or outright criticisms of the pieces. In March 2021, it was difficult to know how the market would continue to shift.

The rapid development of digital media and methods of sale meant that auction houses had taken up new perspectives only a few months later. Data from November 2021, collected by Art Market Research, stated that Sotheby’s had sold $65 million in NFTs by that point in 2021, while Christie’s had sold over $100 million. All told, NFT sales were approximately 5.5% of contemporary art sales for these institutions. Additionally, Sotheby’s shared that 70% of customers of “Natively Digital: A Curated NFT Sale,” held in June 2021, were new customers, and that many of these buyers were people who had grown wealthy from cryptocurrency.

Information about Christie’s involvement in the NFT market was shared by Neda Whitney, Head of Marketing for the Americas at Christie’s, in an interview with Alexandra Bruell at the Wall Street Journal. Whitney shared that in 2021, 34% of total Christie’s customers were new to Christie’s, while 72% of NFT bidders and buyers were new.

In the interview, Whitney shared her perspective on how the marketing for Christie’s needed to adapt to attract new customers. According to her, younger consumers view luxury more expansively than previous generations: luxury is not just traditional art, but also streetwear, wine, watches, and jewelry. Younger consumers can have a great deal of wealth to spend, particularly those employed in technology-based industries, and they are looking to connect with brands over social media and sites such as Discord. Whitney described how her work had grown to include virtual and in-person events to share information about NFTs, and she was behind the promotion of Christie’s auctions through social media campaigns. Overall, she said, she measures success for the institution in terms of new customers and increased brand affinity, as auction houses are seeking greater returns and more customer data.

Whitney’s personal approach, meanwhile, eloquently summarizes the entire reason why auction houses must shift their approach in a digitally-crazed arts world:

My personal goal is to get the narrative of Christie’s into the hearts and minds of all types of clients. – Neda Whitney

One year later, Whitney’s perspective is still accurate. Auction houses are still primarily seeking to explore the world of digital media and business methods in order to meet consumers’ desires, enabling both long-term clients and new customers to feel at home in these institutions. Once again, Beeple served as an example of how auction houses could “settle” in a newly transformed arts world by bringing together traditional sales methods, tangible art, and digital media. In October 2021, Christie’s offered Beeple’s Human One, a seven foot tall sculpture. The frame, made of white mahogany, holds LED screens displaying a walking astronaut. The whole sculpture rotates to display the figure from all angles as they walk through a variety of landscapes.

This sculpture demonstrates how auction houses can continue to auction physical objects with entirely new capabilities. Most notably, the artist can remotely update the sculpture so that the astronaut walks through different backgrounds. Unlike every static sculpture, the owner of this piece will be in constant “contact” with Beeple and experience an ever-changing work of art. Most artists transform over the course of their career: exhibitions, books, and documentaries are dedicated to exploring how famous artists changed in their approach to their work. However, these changes are often explored after the fact. EVERYDAYS did so by capturing in one piece how Beeple’s skill and subject matter transformed over more than a decade. Human One will again display the evolution of an artist, but these changes will be viewed in real time by those viewing the sculpture.

Meanwhile, other artists have been enjoying increased attention due to auction houses’ focus on digital tools and media. In another article by Kelly Crow, she describes how technology has largely diversified what types are art are being sought out. The art world had to reexamine itself over the past couple of years, in response to the pandemic, technological expansion, social justice, and economic shifts. Crow writes:

Huge, shiny sculptures and wall-size photographs oozing luxury or cynicism are out, for now; earnestly expressive portraits and politically potent works by Black artists and women are in—particularly if the artists are young or can be deemed rediscoveries. Tastemakers are also pivoting away from abstraction or coy, conceptual art in favor of pieces that echo issues being raised in the realms of social justice and social media. Some collectors are leaning further into technology by amassing digital artworks, while other sets of buyers are coping by prizing ceramics, with their fragile, lumpen tactility. Demand for dozens of artists has been upended and it’s compelling the art world’s power players to reckon with the fallout. – Kelly Crow

Arts brokers, including galleries and auction houses, have seen dramatic shifts in the marketability of certain pieces and artists as audiences’ tastes shift, and as the audiences themselves shift. Auction houses are critically considering what artists and what pieces they need to be seeking out in order to remain profitable and popular with consumers. In addition to expanding collections, auction houses are also recontextualizing mainstays in order to demonstrate new facets of their work which may resonate more strikingly with the current market.

While the new digitally-focused market has many benefits, relying on technology increases the opportunities for hackers, scammers, and money launderers to benefit. Because NFTs are natively digital, and sales are most frequently conducted online, fake NFTs and fake marketplaces can be used to steal credit card information. Scammers posing as artists can make hundreds of thousands of dollars before they are caught, and while they can then be shut down and/or barred from platforms, there is often no way to recover the stolen funds. Intrinsic features of the digital landscape, like decentralization and the participation of non-tech-savvy consumers, create more opportunities for scammers to profit. Cryptocurrency can also be affected by scammers, and auction houses are having to be vigilant to shut down customers attempting to launder money through cryptocurrency transactions. In response to these risks, companies like Verisart have developed technology to authenticate artworks, particularly NFTs, so that platforms decrease the risk of fraudulent sellers or fake pieces. Auction houses, because of their heightened security protocols, can provide vital verification services for digital pieces, much as they have verified physical artworks for centuries.

The digital landscape is still being formed, and thus auction houses will continue to negotiate and renegotiate their use of digital business methods and their sale of digital media. However, the opportunities for artistic exploration, artist diversification, customer engagement, and profit continue to be tremendously exciting. By balancing their history of trustworthiness with a future full of possibilities, auction houses can remain essential institutions for preserving and disseminating artworks for many years to come.

What is NFT?

“A unique digital certificate, registered in a blockchain, that is used to record the ownership of an asset such as an artwork or a collectible,” —— NFT’s definition by Collins Dictionary.

NFT = data + blockchain + smart contract

Data: a piece of digital information going to be turned into NFT asset

It can be a piece of music, a video, an in-game item, an image, a voice message, or even a tweet. The unlimited plasticity on the format itself gives everyone opportunities to turn anything into NFT. But minting into NFT doesn’t mean it can sell. The price of the data asset is determined by many other factors, such as its collectability, functionality, scarcity, and more. 

Blockchain: a chain of blocks recording data in ways that make the data difficult to change, hack, or cheat

It is achieved by duplicating and distributing one piece of data across over 50% of the chain. If the data on one block was changed, it would be immediately apparent it had been tampered with. If hackers wanted to corrupt a blockchain system, they would have to change every block in the chain, across all of the distributed versions of the chain. 

Smart Contract: the way data is recorded on the blockchain and it is programmable to define NFT’s time, organization, number of editions, which blockchain, initial price, transaction methods, previous owners, etc.

There are many proposed smart contracts commanding blockchain to store data differently and the one that successfully tracks distinguishable asset’s ownership individually and automatically is called ERC721, created in January 2018. Creator of ERC-721 on Ethereum, a decentralized and open-source blockchain, coined the word NFT to describe the broad universe of distinguishable digital assets. Since then, ERC-721 has had wider use on other blockchain platforms, but Ethereum remains to be the most popular one, where 83% of NFT transactions are happening. 

In summary, NFT is about representing anything unique as a digital asset.

The Magic of NFT Profile Picture Art? 

Among the earliest NFTs ever made, there emerges a series of profile picture art designed to be used as avatars for social media accounts, such as CryptoPunks, MoonCatResure, and CryptoCats. And it quickly takes over the market to become the one of most popular NFT art collectables types. In the last 7 days, the world’s first and largest digital marketplace for crypto collectibles, OpenSea shows that 17 out of the 20 top NFTs ranked by trading volume are profile picture art projects. 

One of the mega-successful profile picture art hits is Bored Ape Yacht Club (BAYC), with total sales now standing at $1.03 billion according to data from CryptoSlam. Launched in April 2021 by Yuga Labs with a floor price of 0.8 ETH, BAYC reaches this milestone in just 8 months, comparing the most expensive artwork sold in history is a piece of 500 years old Leonardo da Vinci for 475.4 million in 2017. 

BAYC is a collection of 10,000 unique ape NFTs on Ethereum. Ownership grants access to members-only benefits, the first of which is THE BATHROOM, a collaborative graffiti board. Currently, the average sale price for BAYC is over $200,000 and has exploded into other corners of the world, from Rolling Stone magazine covers to physical art galleries globally. 

What makes profile picture art projects different from most other NFTs? On the creative side, unlike many other one-off digital artworks, profile picture art initiatives usually are a series of thousands, each one in the collection with unique rarity and features. Because of the way NFTs are generated, it is the potential to gather a relatively larger group of local buyers. On the distribution side, the collection of profile picture art usually is airdropped randomly at a very low price, or even for free, to collectors. It is not only accessible but also comes with a thrill of gambling. 

But most importantly is the social value of profile picture art making them better performing than other NFT artworks. Member club model is exclusive to BAYC, but shared among many other profile picture art projects. According to the experience shared by owners of ape avatars, the top three drivers includes:

We love the look, it matches our digital personality

We are rich enough to buy such high valued asset, we are golden

We are true believers of NFTs, we are in the same community

On the practical side, profile picture art are one the most functional kinds of digital assets in the markets now. While owner of the most expensive NFT artwork ever the Everyday: The First 5000 Days can hardly show his treasure to the world, profile picture art can be used as avatars across all social media platforms, from Facebook, Instagram, Twitter, even WeChat, where people can use to establish their identity. Questmobile data shows young people, especially Gen Z, on average, spend half of their time in the digital world and they care about their virtual identities. Claiming profile picture art equals telling the rest of the world who they are, helping them find their own kind and make friends. 

Other than its usability as a social media personality statement, most profile picture art projects allow collectors to spark their own creativity, by giving the full commercial rights to the NFT, and building their own community within the group on messaging channels like Discord. The popularity of profile picture art projects indicates the germination of such social value. 

Who are the collectors?

Although blockchain technology is designed to ensure privacy and anonymity of creator and collector information, the identity of some big NFT influencers is revealed in the media. NFT has become a global dream and the demand for highly valued profile picture art apes is increasing day by day. 

Top 30 Influential People

Last December, ARTNET released an annual report, THE ARTNET NFT 30, giving a summary of the 30 most influential people who are shaping the field of NFTs. This group of new content creators, manipulators, and collectors shares a common background in crossing the field of technology and creativity before getting into NFTs. 50% of the NFT artists were hackers and others are illustrators and animators. And ⅓ are self-taught. 

Geographically, this group of influencers spreads from Nigeria, Sri Lanka, Canada, US, Singapore, China, and all over the world. Statistics from Finder shows that 32% Philippines own NFT (thanks to a Philippines Play-to-Earn NFT Game Axie Infinity), 27% of Thailands, 24% of Malaysians, 23% of UAE, and 17% of Vietnam.

Celebrity and Influencer

Besides the contribution of hardcore hackers, NFT walks into the public by celebrities effect. From singers like Justin Bieber and Eminem, NBA stars like Stephen Curry and Shaquille O’Neal, to other celebrities like Jimmy Fallon and Paris Hilton, all of them bragging about their millions dollar profile picture art triggers public’s curiosity.

In January, Twitter introduced a feature that allows users to use NFT as their profile pictures. “The NFT conversation on Twitter is thriving with 255.6 million tweets globally and continues to rapidly grow. Authenticity around NFT ownership is important to the crypto community and with the profile picture art, we are providing a way for people to prove ownership of their NFTs and support those making this art,” Twitter said in the announcement. It is foreseeable that this function will make the identity of community members more transparent.

NFT = Rich People’s Game?

Skyrocketed NFTs surely cuts off some investors.

News about stunning high valued profile picture art projects give an impression of all NFTs are high-priced and it’s a game for the rich only. Is it true? Of course not. Among the thousands of profile picture art projects, BYAC is the only one with sales over 1 billion. 95% has a floor price of around 0.02ETH (around $50 or less). 

Speculators

Market analysis by Chainalysis, a crypto-focus analysis platform, reveals that there are much fewer players in NFTs than previously thought. With the market size approaching 41 billion, there are only 800,000 unique buyers. Some of the most active NFT traders are speculators. Many of them constantly seek newly released profile picture art to buy low and sell high. Many of them are speculators in wish to drive up prices to pay rent. So in many cases, the fundamental value behind profile picture art as collectibles is neglected.

Companies

Another type of player in the market is companies, adapting NFT as part of their media strategy. Besides launching their own NFT projects like Nike and LV, other companies, like Timberland and Adidas, purchase profile picture art projects to convey the message of “speaking the same language” as their target audience. Companies are seeking a deeper relationship with consumers to establish brand trustability. For instance, limited editions are utilized widely to build a sense of brand uniqueness. However, before the NFT model, companies can hardly make a direct connection to the secondary market or community beyond marketing and transactions. This is where NFT steps in. Former Chief Digital Officer at LVMH recognized the value of NFT as a bridge between real consumers and brands to accumulate first-hand real consumer information and extend brand influence to younger generations. 

When everyone sees chance to win, game goes on

Thanks to the open-ended nature of NFT asset types, there remain many undiscovered business models to be explored and value touchpoints to be built. NFT art intersecting movies, music, real estate, gamification, blockchain, entertainment, and metaverse keeps pushing the limit of imagination. Besides technology advancement, how to deepen and broaden the scope of NFT applications depends heavily on the fundamental social value it brings. Who can resist living a happy and prosperous life?

Source of References

The Crypto Times. “15 Celebrities Who Own BAYC NFTs,” February 2, 2022. https://www.cryptotimes.io/15-celebrities-who-own-bored-ape-yacht-club-nfts/.

Castrovilli, Marco. “Why the World’s Largest Museum Is Embracing NFT Technology.” Cointelegraph, December 10, 2021. https://cointelegraph.com/news/why-the-world-s-largest-museum-is-embracing-nft-technology.

Chanthadavong, Aimee. “Twitter Blue Introduces NFT Profile Pictures for Subscribed IOS Users.” ZDNet, January 21, 2022. https://www.zdnet.com/article/twitter-blue-introduces-nft-profile-pictures-for-subscribed-ios-users/.

Eluemuno, Christopher. “Top 5 NBA Players Who Have Invested in the Bored Ape Yacht Club NFTs.” Sportskeeda, December 28, 2021. https://www.sportskeeda.com/basketball/news-top-5-nba-players-invested-bored-ape-yacht-club-nfts-steph-curry-shaquille-o-neal-lamelo.

Khomami, Nadia. “British Museum Enters World of NFTs with Digital Hokusai Postcards.” The Guardian, September 24, 2021. https://www.theguardian.com/technology/2021/sep/24/british-museum-nfts-digital-hokusai-postcards-lacollection.

Kim, Christine. “Artist Ai Weiwei Uses Ethereum to Make Art About ‘Value.’” CoinDesk, August 17, 2018. https://www.coindesk.com/markets/2018/08/17/artist-ai-weiwei-uses-ethereum-to-make-art-about-value/.

McDowell, Maghan. “The Blockchain Playbook: From LVMH’s Aura to Arianee.” Vogue Business, April 26, 2021. https://www.voguebusiness.com/technology/the-blockchain-playbook-from-lvmhs-aura-to-arianee.

Melinek, Jacquelyn. “Aligned Raises $34M To Grow DeFi and Web3 Infrastructure.” Blockworks, February 24, 2022. https://blockworks.co/aligned-raises-34m-to-grow-defi-and-web3-infrastructure/.

Network, Artnet Gallery. “5 Questions for Sydney Xiong, Director of the Art-Meets-Finance Foundation APENFT.” Artnet News, January 12, 2022. https://news.artnet.com/buyers-guide/5-questions-sydney-xiong-director-art-meets-finance-foundation-apenft-2059190.

NFT, Artnet. “The Artnet NFT 30 Report: Meet the Artists, Innovators, and Collectors Who Built Our New Crypto-Art Era (Part Three).” Artnet News, December 16, 2021. https://news.artnet.com/market/the-artnet-nft-30-report-part-three-2047797.

Roose, Kevin. “I Joined a Penguin NFT Club. Here’s What Happened.” The New York Times, August 12, 2021. https://www.nytimes.com/2021/08/12/technology/penguin-nft-club.html.

Smith, Oliver. “Kevin Abosch: The Artist Who Became A Bitcoin.” Forbes, June 27, 2018. https://www.forbes.com/sites/oliversmith/2018/06/27/kevin-abosch-the-artist-who-became-a-bitcoin/?sh=25e7f47647b4.

Symphony, California. “Audience Development: The Long Haul Model – California Symphony.” Medium, August 15, 2017. https://medium.com/@CASymphony/audience-development-the-long-haul-model-b63842da1399.

The Art Newspaper’s XR Panel. “What Is the Metaverse and Why Does It Matter to the Art World? Experts Weigh in and Predict Its Future Impact.” The Art Newspaper – International Art News and Events, January 28, 2022. https://www.theartnewspaper.com/2022/01/28/what-is-the-metaverse-and-why-does-it-matter-to-the-art-world-experts-weigh-in-and-predict-its-future-impact.

The Audience Agency. “Digital Engagement with Culture | Exploring the Act Two Survey.” Indigo, September 2020. https://www.theaudienceagency.org/asset/2347.

Vick, Aaron. “How NFTs Are Creating Social Value.” Forbes, February 24, 2022. https://www.forbes.com/sites/forbestechcouncil/2022/02/24/how-nfts-are-creating-social-value/?sh=1c5f35cf7ecb.

Webb, Bella. “Inside the Secretive World of Luxury Authentication.” Vogue Business, January 31, 2022. https://www.voguebusiness.com/sustainability/inside-the-secretive-world-of-luxury-authentication-vestiaire-collective-the-realreal-ebay-stockx-goat.

The world of theatre is forever changing and adapting with the continuous discovery of new technology. The most recent wave introduces the limitless possibilities that Artificial Intelligence (AI), Virtual Reality (VR), and Augmented Reality (AR) entices. The presence of these developments has changed, and still in the process of changing, the world of theatre as we know it. This redefinition of what is now capable in theatre through those introductions has also lead to modernized ways of including individuals who could not enjoy theatre in its entirety before. One of these advancements is AI empowered glasses for the blind and visual impaired.

Who does this Technology Affect?

There are roughly 3.22 million people in the United States with vision impairment (VI), as defined by the best-corrected visual acuity in the better-seeing eye. Vision disability is also classified amongst the top 10 disabilities amidst adults 18 years and older. The most common aid for this disability is the walking cane which was introduced in 1921 with little improvements to the design for the last 100 years. This was until the tangible exploration of AI Software to this current magnitude began in 2020 and piloted a piece of technology called AI Glasses. This revolutionary creation will permanently change the world for all affected by vision impairment and guide the world of theatre into the future of inclusion.

What are AI Empowered Glasses?

AI empowered glasses utilizes Artificial intelligence to extract information from the surrounding world and audibly speak a description to the user. The normal construction of this type of eyewear includes various materials such as metal, hard plastic, and glass. Most frames are of a slimmer build and integrate a camera, in addition to a simple touch or click function to activate the software. The AI software for these glasses could be hardwired to perform a multitude of functions, however, the general options include the analysis of text, script, faces, environments, images, and colors. The audio operation for most glasses could be programmed to speak out loud or personally per the user’s preference in a plethora of languages. The most advanced glasses present over 60 languages available for selection.

The creation of these glasses has made daily life more unrestricted as individuals who were unable before are now capable to live independently and unaided by the commonly practiced walking cane. Examples of the ease this technology provide include the audible description of grocery store products, books, street signs and directions, in addition to legible handwritten text/script. A way of life that was not an option before the user friendly development of these glasses. Many users who have beta tested and bought a style of AI glasses, recount tales of the simple luxuries seeing able individuals take advantage of every day. A few examples of these luxuries are taking a stroll, grocery shopping alone, utilizing public transportation, and simply reading a book. Many companies around the world are in the process of testing, manufacturing, and producing newly developed AI Glasses, however, the leading brand to utilize this technology is Envision.

Evolution of Envision

Envision is a global organization that began in the Netherlands as an app for smartphone users. The co-founder, Karthik Mahadevan, started the company “from our desire to empower blind and visually impaired people with greater independence and access to the visual world.” The app launched on IOS and Android Smartphones in 2017 and had the capabilities of turning text to speech, scanning QR codes, describing the environment, and scan for people.

With the integration of AI beginning in 2018 (publicly), the AI assistant made it possible to transform what was once an average app, to a high tech composition that was named Best Accessibility App at the 2019 Google Play Awards and a Hall of Fame Award Winner by AppleVis. No longer was non human descriptions being presented for the user to dissect in confusion. Now with AI technology, descriptions have an intelligent compound that makes it easily user friendly. The combination of this system in addition to the smartphone app integration has produced a mobile revolution that changed the rose in the ranks when compared to the alternatives.

In 2020, the leaders of Envision publicized a press release to announce the partnership with Google Glass to create what is now known as Envision Glasses. These glasses work in tandem with the mobile app and provide identical functions to its partner app, however, a few enhanced features were added. These features include a real time function that works alongside the live video feed to provide continuous information for what the camera is viewing. Another newly introduced feature is the commands of “Read, Find, Identify and Call.” When these commands are spoken by the individual, the signal is triggered to direct the app to complete the objective. The glasses do not work with the smartphone software directly. Instead, the app should be downloaded for full usage as a partner to the glasses. The app downloaded on multiple phones is the connection point and provides the ability to video and voice call other app users through the interface.

Disruption To The Arts World

With everyday that passes providing new opportunities to enhance and adapt the product, the integration to the arts sector becomes a more tangible possibility. Gone are the days when visually impaired individuals need to be assisted to view an art gallery. Special shows and dates no longer need to be made and adapted for this exclusive audience. Helping hands are not required when the need arises to paint, sculpt, create, dance, or practice any form of art. These guiding allies are instead invited in response to comfort level of the user engaging with the Envision Glasses instead of out of requirement.

The only point of contention that should be mentioned is the average price of these glasses. Envision Glasses’ current price is 3,268.91 (including taxes) euros which translate to $3,620.15. This is the standard rate for all AI Glasses circulation currently. While there are financing options available, this price makes it difficult for the average American to purchase a pair, especially when these glasses are considered technology instead of a need, and as a result, insurance will not cover them. This price also makes it difficult for organizations to pay for them in bulk and distribute them to individuals for the run of the show. Envision states that they are working on ways to make their glasses more affordable while keeping their integrity, however, this time will not come soon and will slow the introduction of this type of glasses to the artistic world.

Disruption to Theatre

The technology of AI Glasses has not yet made its introduction to the world of theatre as of yet. However, let’s explore the prospective opportunities that this technology generates.

The first is inclusion under normalcy instead of unique instances:

Many theaters hold similar core values and inclusion ranks amongst the top of nearly all lists. Most theaters, generally of the non-profit sector, are constantly revising their strategies to discover improved and modern ways to better serve their communities. The visually impaired members of society are a type of audience that requires alternative formalities to take place in order to enjoy the highly visual art form of theatre. Provisions such as separate show dates, assistance in describing tech aspects and what are considered visual cues, and time allocated to touch the sets have become apparent. However, these are still not widely practiced, except for a select number of theaters, due to a lack of time and resources. The introduction of these AI Glasses allows for Visionally impaired users to purchase a ticket on any performance of the production run, and experience the art at the moment with the rest of the audience.

The second is the exploration of visionally impaired performers:

It is with no uncertainty that I state that visually impaired actors deserve their credit due as we watched these individuals begin to command the stage in the previous decade. Through various outreach and communities efforts, visually impaired artists have taken the stage alongside seeing artists, as well as leading entire productions for the blind on their own. Broadway is on the list of a multitude of theaters avidly seeking productions to promote the presence of visually impaired actors as they began to release public casting calls in 2017. However, the process of preparing productions with the specification of this cast could be considered one that required those elements some theaters can not afford: time and resources.

With the into of this technology, those resources could be contained in a more widely manageable manner. The specific Envision Glasses contain the appearance of “normal” glasses from an audience’s perception. As a result, explaining their presence will not be required for audience understanding. This leaves the floor open to visually impaired audiences to utilize and then commandeer the stage. Longer rehearsals, touch sessions, and a more robust description will no longer be needed to factor in as the AI assisted glasses will take care of it all. These unassisted artists will be able to work on their craft respectfully and independently.

Conclusion

The world of theatre is forever changing. A statement I made at the beginning of this article that continues to ring true. With every advancement in technology, new possibilities occur. The introduction of AI Glasses has caused a ripple effect in the lives of individuals impaired visually. The mission to independence and inclusion has become more tangible to undertake for the community it is actively affecting as well as the society as we learn how to navigate a world with new rules. The same could be said for theatre organizations as they contemplate diversifying their audience and performers, new ways to complete their mission/vision/value statements, and discover redefined ways to tell the stories of life.

References

AI technology helps students who are deaf learn. The AI Blog. (2018, May 11). Retrieved February 11, 2022, from https://blogs.microsoft.com/ai/ai-powered-captioning/ 

Alice Finney |14 January 2022Leave a comment. (2022, January 14). Biped’s smart harness helps visually impaired people avoid obstacles while walking. Dezeen. Retrieved February 22, 2022, from https://www.dezeen.com/2022/01/14/biped-smart-harness-visually-impaired-obstacles-ces-2022/ 

Barrett, S. (2020, January 9). First look: Cre outreach presents world premiere of blind talent featuring vision-impaired actors. BroadwayWorld.com. Retrieved February 3, 2022, from https://www.broadwayworld.com/los-angeles/article/First-Look-CRE-Outreach-presents-World-Premiere-of-BLIND-TALENT-featuring-vision-impaired-actors-20200109 

Broadway embraces blind casting: WNYC: New York Public Radio, podcasts, live streaming radio, news. WNYC. (n.d.). Retrieved February 3, 2022, from https://www.wnyc.org/story/broadway-embraces-blind-casting/ 

Cahalane, C. (2017, December 20). How image recognition and AI is transforming the lives of blind people. AbilityNet. Retrieved February 15, 2022, from https://www.abilitynet.org.uk/news-blogs/how-image-recognition-and-ai-transforming-lives-blind-people 

Calif , A. (2020, March 9). Envision announces AI-powered smart glasses for the blind and visually impaired. Envision Announces AI-Powered Smart Glasses For The Blind And Visually Impaired. Retrieved February 15, 2022, from https://www.letsenvision.com/blog/envision-announces-ai-powered-smart-glasses-for-the-blind-and-visually-impaired 

Can visual-ai help people with impaired vision? VISUA. (2021, July 12). Retrieved February 15, 2022, from https://visua.com/can-visual-ai-help-visually-impaired-people%3F 

DiCYT, A. (2019, December 11). Artificial Intelligence and Accessible Theater for the deaf and blind. DiCYT. Retrieved February 20, 2022, from https://www.dicyt.com/news/artificial-intelligence-and-accessible-theater-for-the-deaf-and-blind 

Envision glasses. Envision Glasses. (n.d.). Retrieved February 15, 2022, from https://www.letsenvision.com/envision-glasses 

Jan-Willem Wasmann. (2022, February 1). Speech recognition apps for the hearing impaired and deaf. Computational Audiology. Retrieved February 15, 2022, from https://computationalaudiology.com/ai-speech-recognition-apps-for-hearing-impaired-and-deaf/

Madeleine Burry. Madeleine Burry is a Brooklyn-based freelance writer and editor. She’s written about health for several online publications. (2021, January 25). Ai hearing aids: How artificial intelligence can help hearing loss. Healthy Hearing. Retrieved February 16, 2022, from https://www.healthyhearing.com/report/53168-Hearing-aids-artificial-intelligence-deep-learning-oticon 

Rego, N. (2021, April 27). This new AI technology could help visually impaired people navigate independently. This New AI technology could help visually impaired people navigate independently. Retrieved February 15, 2022, from https://coolblindtech.com/this-new-ai-technology-could-help-visually-impaired-people-navigate-independently/ 

Leonard, A. (2019, January 17). Theatre performances for blind audiences. MaxiAids.com. Retrieved February 22, 2022, from https://www.maxiaids.com/theatre-performances-for-blind-audiences 

Strap technologies. STRAP Technologies. (2021, November 26). Retrieved February 22, 2022, from https://strap.tech/?gclid=Cj0KCQiAmeKQBhDvARIsAHJ7mF6ovQkSojhBqKI1Fp8nG62T72Dym-OfWDcrKK_lwnT4tHB9tzQsfOcaAtXiEALw_wcB 

Theatre performances for blind audiences. MaxiAids.com. (2019, January 17). Retrieved February 3, 2022, from https://www.maxiaids.com/theatre-performances-for-blind-audiences 

Gamification, in the most well-known and basic definition, is “the process of adding games or game-like elements to something (such as a task) so as to encourage participation” The definition of gamification may vary between people and situations, but the core is to utilize concepts from game development to solve problems in other fields. 

Though The term gamification was created by Nick Pelling, a British computer programmer and inventor, in 2003, people started exploiting the concept of gamification during the 19th century. A grocery company called the Sperry and Hutchinson Co. motivates its customers to consume by substituting money with the S&H Green Stamps. This case is the ever first gamification approach in the world. One hundred years later, key opinions leaders such as Jesse Schell and Jane McGonigal brought gamification to the public. From then on, people started thinking about “ the potential of gamification and its use in several aspects of our life” 

Nowadays, gamification is widely used in different industries such as catering, education, e-commerce, and of course, Arts by adding minor interactive gameplay on the website or application. Organizations, no matter for-profit or non-profits, are all persistently searching best way of optimization and believe that gamification may be applicable at some aspects of their businesses.

Why is gamification such a hit in the art world?

Art professionals and organizations have always held a strong interest in gamification even before the born of the word “gamification.” Besides, compared with other industries such as healthcare, the art world seems to have a special obsession with gamification. So why does this happen? 

The first thing is that arts and games are intrinsically related and undividable. Though whether the game itself is a kind of art is a question with polarized answers, the relationship between arts and games is undoubtedly close. Some of us may hear about the classic Role Play Game Assasin’s Creed by Ubisoft was chosen as a resource to help restore the burned Notre Dame de Paris. An artist from the Ubisoft game told the media that the game development team “has spent years literally fussing over the details of the building. She pored over photos to get the architecture just right and worked with texture artists to make sure that each brick was as it should be”.

Source: https://cdn.vox-cdn.com/thumbor/MFRFH1aMCZuaC4s1fOPY9yJHZSM=/0x0:2724×1530/1200×0/filters:focal(0x0:2724×1530):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/23036671/Screen_Shot_2021_11_22_at_11.26.11_PM.png

Alongside transferring art concepts to the game, arts organizations also appreciate games. The Victoria & Albert Museum collected the pixel-style casual game “Flappy Bird” and curated an exhibition. The process of art creation is actually a process of creating Intellectual Property, and IP is a just need in the gaming industry. On the other hand, the art world is always seeking a more inclusive, interactive exhibition mediation to broaden the boundaries of artistic creation. Game, which has an infinite capacity of presenting creativities, is an ideal vehicle for arts.

Source: https://i.pcmag.com/imagery/articles/01EkIOfisgxkfodZ4JmTIXa-1..v1569489280.jpg

Furthermore, Arts managers have always been looking for a better way to engage their audiences positively, and interaction is the core driver of engagements. Non-profits are willing to influence the public to do one of the two things:  to participate in a new activity that the audience would have avoided or not known about in the past; or to make changes to daily behaviors over a long period of time. In the first scenario, gamify cryptic art pieces makes art more understandable and less static. Meanwhile, as games are convenient to spread in the digital world and popular among people from many demographics, the target audiences could be more diverse. For the second scenario, games’ additional educational features could help. Unlike traditional arts education approaches, gamification makes arts education more effective than ever. In conventional art education practices, educators are used to presenting an art piece or concept as a whole to the students, which can result in misunderstandings and boards. However, while gamifying arts, we need to break down the original art production into artistic elements and then re-assemble them, and then players will accept the information step-by-step when playing the game. This is absolutely a rather exciting process. 

Another critical point is that game development, especially casual game development, is financially affordable for nonprofit arts organizations. Several accessible game engines such as Unity and Unreal eliminate the technical barriers, and the raising promotion channels like Google Ads and Facebook Ads make the publishing process straightforward. And even Unity itself is a popular ads channel in the game world. Take mobile casual games as an example. Usually, a hyper-casual game development team only requires 1-2 game developers and one graphic designer and may cost about $20,000. The core of game development is no fancy technology. It is more about creativity. Overall, for medium- and large arts organizations, building a game development team or hiring a gamification agency is an affordable way.

On the other hand, gamification might help nonprofits explore a new way of monetization. The gaming industry has always been lucrative with minimal entry barriers. According to mobile data and analytics provider App Annie, mobile games are on track to surpass $120 billion in spending in 2021, growing 20% over $100 billion in 2020. By gamify art collections, arts organizations may have a chance to generate revenue from the game and sustain a mission for social good. The gaming industry needs creativity, and the art world always seeks living resources. It seems like they are the best fit.

Last but not least, the unavoidable Covid-19. Pandemic restricts physical experience, but raising technologies such as Metaverse, VR/AR/XR makes the digital experience more genuine and even beyond reality. Gamification is an excellent way to present arts on digital equipment. I will further discuss this in the last part of the article.  

Gamification in Digital Arts Programming: Production-based and Organization-based

When implementing gamification in the real world, arts enterprises usually have two choices. The first choice is to gamify art collections. The Palace Museum in China has made several temptations on this. In 2018, The Palace Museum collaborated with NetEase, a top-tier technology company famous for game development and publishing, published a traditional Chinese art style Puzzle Role Play Game “Ink · Mountains and Mystery”. The game is inspired by one of the ten most precious Chinese traditional paintings, the “Thousands of miles of mountains and rivers”, which is The Palace Museum’s collection. The painting is a blue-green landscape silk painting finished in the Ming dynasty and has been appreciated for its vivid, imaginary color usage. Fortunately, the game flawlessly rebuilt the magnificent images utilizing 3D engines and gave a beautiful story to the original painting. At the end of 2018, the game had acquired more than 120 thousand downloads and effectively promoted traditional Chinese painting to younger generations. From then on, the traditional Chinese art style and elements have become a new trend in Chinese society, especially on social media. From this case, we can see that gamification can make arts more interactive, more authentic, and even rejuvenate a traditional art piece.

Another surprising thing is that via this gamified digital program, both the “Thousands of miles of mountains and rivers” and the Palace Museum obtained fans from younger generations: the game has more than 1.2 million followers on TapTap, a game channel for GenZers and Millenials in China. I believe this is not the end of this story. Imagine how many kids, teenagers, and young adults would be interested in traditional Chinese art and may become a member of the art community. An excellent gamification digital arts program could do more than improve audience engagement, and it can help a niche arts genre maintain sustainable development.

Another choice would be organization-based programming. In this field, the Metropolitan Museum of Arts in the U.S. and the Center Pompidou in French would be the two best examples. In 2012, only one year after the world’s first gamification summit closed in San Francisco, The Met published a game called “Murder at the Met: An American Art Mystery” which is a “web-based application (app) that guides users of smartphones and tablets through the galleries of American paintings, sculpture, and decorative arts at The Metropolitan Museum of Art to solve a murder mystery.”

The game is still accessible on the website today. From a 2022 point of view, this game is not visually appealing or attractive in gameplay design, but the concept it used was Avant-guard at that point. The Met’s foresight of the gamification trend is so precise that it keeps inspiring other organizations today. Going back to the game, we could quickly tell the difference between the Murder at the Met and the Mountains and Mystery: the prior one represents the organization while the latter focuses on the art piece. To better understand the outcome differences of these two different kinds of gamification digital arts programs, we can just take a look at another game produced by the Centre Pompidou, the Prisme 7. 

The Prisme 7 is “designed in collaboration with digital artists Bright and Game in Society, and with the support of the Ministry of National Education”, in the game, the players can “discover seven levels of play and interact with 40 major works from the biggest collection of modern and contemporary art in Europe. It also offers a unique sound immersion designed especially for the game by Ircam Amplify. ” Unlike the Murder at The Met, Prisme 7 doesn’t have a clear storyline, which more aligns with most casual and hyper-casual games. I attribute this to the different types of the two organizations. As the most famous visual arts organization with a heavy focus on art history, The Met would like to utilize a gamified digital program to do art history education. By contrast, the Center Pompidou is more contemporary, which means their collections and exhibitions are not that easy to interpret for the public, so they tend to use the interactive and attractive nature of the casual game to present their collections vividly and generate more audience engagement. 

Compared with the Mountains and Mystery, these two games create more brand exposure for the arts organizations and are more severe in terms of arts. The Met basically presented the whole story behind that piece, and Prisme 7 tries to convey contemporary art concepts as much as possible by utilizing the play-reward mechanism in games. Also, both of them have a solid organizational identity. The Met uses one of its most identical masterpieces, Portrait of Madame X and Prisme 7 restored the special building of Center Pompidou in the digital space. However, the Mountains and Mystery only picked some elements from the original painting and tried to modify the art further to make it easier to spread. Users can not find many clues about the original art piece or the organization behind it. What they know is only the beautiful, fictional story that happened in the blue-green landscape. The differences are mainly from distinct business collaboration models. The Murder in The Met and Prisme 7 is developed and published by the organizations, so the two games are just like two digital gamification programs conducted by the organizations. They should make the programs align with the organization’s mission and focus on the academic and educational features. But the Palace Museum cooperated with NetEase, which was entirely in charge of the game development and publishing. So, It’s easy to find the commercial elements in the Mountain and Mystery.

I would not judge which kind is right and which is wrong. I would rather say that there are always pros and cons. The Met and Center Pompidou definitely took a giant step in informative and interesting arts education, but their productions are still targeting arts fans, which constrained the scope of the impact. The Palace Museum approached audiences from the outside art world, but they sacrificed a lot on artistry. If the organization doesn’t have a strong voice, the art may be threatened by over-entertained. 

A glimpse of future: Gamification, XR technology and Arts

For decades, VR/AR/XR technology has been a viral topic. Most arts organizations have also paid attention to this trend for a long time. However, we still have less idea about what’s next to XR technology and how we can use it more creatively, or at least make it beyond a digital venue. However, with the raising gamification concept, we may garner a chance to look at the future.

In the field of arts education, there are some research and experiments on the effectiveness of VR-based gamified art history education. A research team in the Department of Systems Engineering, Universidad Nacional de San Agustin de Arequipa in Peru has tried to achieve the “implementation of a virtual reality game using gamification by elements of video games to improve the teaching of Art History“ To do so, they ”simulate a virtual museum, with predefined stages that expose works from different artistic currents with their respective representations all owing interact with the characters, objects, animations and sounds presented in the application.” The result shows that navigation activities, tasks, and movements are statistically correlated in the VR environment and significantly influence the outcome of learning. And, compared with implementing VR or gamification separately,  the combination of VR and Gamification has higher efficiency in arts education. 

Another possible development direction is the Metaverse. In general, Metaverse is “an immersive virtual world that people “enter” through the use of technologies like AR and VR” 

In terms of the art world, leading arts organizations such as Sotheby have just announced the launch of Sotheby’s Metaverse, the auction house’s new platform designed specifically for digital collectors.  Another well-accepted opinion is that humans need to interact with Metaverse via gamification. The most significant supporter of this idea is Satya Nadella, the CEO of Microsoft and the key character behind that shocking acquisition between Microsoft and Activision Blizzard. He told the media that “Metaverse is essentially about creating games,” and “It is about being able to put people, places, things [in] a physics engine and then having all the people, places, things in the physics engine relate to each other.” As I stated before, the game is closely related to the arts, just as it’s related to Metaverse and XR technologies. Metaverse somehow creates a platform for arts, technologies, and gamification, but right now, we cannot predict what will happen.

Conclusion

Through previous analysis, we can conclude that gamification is and will always be a key concept for every entity in the art world. Arts enterprises recognize it as a management tool,  artists use it as a vehicle of artistic creation, and audiences enjoy spending time with it. Overall, gamification significantly helps the short-term growth and the sustainable development of art. In addition, new technologies and technological concepts give gamification more possibilities. We can tell that gamification will combine with different emerging elements and build connections between arts and the future.

Finally, the need for a new appropriate museum and archive infrastructure is shown to preserve the art of our time.

Digital Art Through the Looking Glass, Grau, Hoth and Wandl-Vogt

Alongside the constant evolving of technology, digital art has modeled itself into its best possible version. While the talents use digital tools to create more and more diverse and cutting-edge art pieces, it gradually becomes a significant form of contemporary art standing on its foot. Feeling a strong need to create a space dedicated to presenting digital art in its own context, the Museum of Contemporary Digital Art came to life.

The Museum of Contemporary Digital Art (MoCDA) is a digital-based museum that “exhibits digital artworks to document, collect and advance the position of digital art.” Their primary focus is education and the endorsement of contemporary digital art. Its exhibitions could generally be categorized into three different types: virtual exhibitions based on the museum website, virtual exhibitions generated by other virtual world platforms, and new media arts exhibited directly on the website. System Shock – 777 Exhibition is a first type, in which users enter the virtual space by clicking on a link to the web page. It is viewed from the first perspective and adapts the design language of traditional exhibitions. One can zoom in and out on any part of the exhibition space. And in some cases, users can follow the arrow on the ground to enter several rooms. The exhibition is accompanied by background music. By clicking on the chat sign on the right bottom of the page, users will see how many people are in the exhibition and chat with each other.

The exhibition, MEMENTO MINTI – if you stop contributing, you will be forgotten, was presented via Decentraland, a 3D virtual world. It is an example of virtual exhibitions generated through VR platforms powered by blockchains, in which users can purchase virtual plots of land, craft, and monetize the contents within the virtual world. This shared metaverse allowed MoCDA’s exhibition to be much more sophisticated. Before entering the virtual exhibition, users get to create their own avatars (though with quite limited options for free). From a first glance, the environment shares the simple blocky graphics that would remind people of Minecraft. The exhibition space was held in the MoCDA virtual building and consists of three stories.

Like the web-based virtual gallery, the interior of MoCDA Building follows the basic design language of the real-space exhibition.  Similar applications such as background music and the chat room can also be found in the virtual museum.  Except this time, users have a lot much more freedom in controlling the movement of avatars, and are able to pull out more complicated action commands, such as jump or  interact with the artwork. In general, the experience with MoCDA’s virtual museum in Decentraland is more like navigating in a game setting.

The third type of exhibition is based on the web page, so visitors do not have to open another tab to enter the exhibition. The exhibition Digital Bodies – concerning bodily structures showcases blockchain-based tokens. Although it appears like a regular GIF on the website, it is in fact a unique 3D model registered on the blockchain. Thus, MoCDA is both creating traditional exhibitions in a virtual space, and presenting arts made with the latest technology.

In light of MoCDA’s commitment to education, the institution holds  live talks with digital artists, curators, and art experts. The talks are live-streaming, and those recordings of talks can be revisited on the museum website.

A museum without gravity, plumbing or code regulations.

Johan van Lierop

Unlike MoCDA, which showcases works meant to be consumed through VR, the collection of the Kremer Museum “forced” the old masterpieces into a digital recreation. The museum is an extreme case of VR taking over the physical entity. It displays over 70 Seventeenth-Century Dutch and French paintings of George Kremer’s collection in exclusively virtual space. 

Each painting has been photographed between 2,500 and 3,500 times with the “Photogrammetry” technique, resulting in an ultra-high-resolution model for each work. The collection includes great masters such as Gerrit Dou and Rembrandt and less famous but exciting and worthy artists such as Abraham Janssens and Theodoor Rombouts. The idea is to make it truly accessible to all people to see the masterworks in a museum setting. Johan Van Lierop, an architect and the designer of the museum, believes VR is ” to the 21st century what Dutch Realism was for the Golden Age, allowing the observer to escape into an alternative reality or mindset.”

The paintings of the Kremer Museum can be viewed under the “Collection” section. The full-sized painting and a detailed introduction to the work will pop up by clicking on each piece. Visitors can then zoom in on any part of the painting to view it close-up. It seems like they worked really hard to recreate the unique colors of individual artworks and feel their textures, such as small cracks of pigments resulting in a long period.

The Kremer Museum can be downloaded from various app stores, including Apple, Google Play, Steam, Meta Quest, as well as Vive Port. A VR experience is also made possible through the headsets.

A hub for debate and discussion around innovation through the digital.

Lee Cavaliere

Launching in 2020, Virtual Museum of Arts (VMOA) is the world’s first entirely virtual museum. It collaborated with some of the most prestigious museums worldwide, such as the Museum of Modern Art, Chicago Institutes, and the Musée d’Orsay. It recreated the 3D reproduction of works from over 40 artists varying in time, styles, and places, such as Henri Matisse, Francisco Goya, Frida Kahlo, and Bansky.

To visit VOMA, users can simply tap on the “Enter” button from the homepage and then launch on the entrance. By clicking on the white circles, users will be able to move around in the virtual space and drag the mouse to have a 720-degree view of the space. There is the built-in sound of running water, corresponding to the waterfront surroundings. At the same time, plants are in motion with the wind to create a generic imitation of nature. Architect Emily Mann even managed to change the scenery according to different times of the day and weather.

When entering the gallery space, users can click on each piece for more information, as well as to zoom in and out. Some of the video installations will be viewed via Youtube. Map, catalogs, and the chat function are available on the bottom of the page. Users can either use the map or follow the staircases to move to other sections. Compared to the virtual museum of MoCDA, which has users to take the trampoline-like installation to jump to different floors, transferring through the staircase seems a more realistic experience. However, the limitation of only being able to move to the designated spot of the circle may undermine such realistic efforts.

There is also a cafe, which serves as a chat space for users to message other users sitting on the same table. The reading room is an online archive with materials to featured artists, and the shop sold merchandise such as the reproduction of paintings, T-shirts, and notebooks. All of those applications are closer to the usual web-browsing experience instead of a more interactive touch.

Compared to the MoCDA, which wants to create a place for digital art to be displayed in its original context, and Kremer, who challenge the physical means of old masterpieces, VOMA is less radical in its agenda. Instead, it focuses on developing a fully immersive experience with the latest technology. Stuart Semple, the creator of the VOMA, brought together a team of programmers, video game designers, and architects to build the museum from scratch. They attempt to restore all the essential components one would expect from a physical museum, including a uniquely designed architecture, external surroundings, gallery space, cafeteria, and bookstore.

What does VR mean for museums?

We see VR museums grapple with the means of digital and push the boundaries. From MoCDA’s creating a space for digital art to Kremer’s converting physical paintings to an exclusive virtual view to VMOA’s fully immersive museum. They claim the legitimacy of digital art forms in their own language as something more than just a by-product of the art world. In doing so, they challenge audiences on to what extent we would refer to creation as art and who gets the access, and what to expect as an audience of the museum. 

One thing particularly interesting is that, among the three, Kremer is the only one who decided not to attach their paintings to a virtual wall. By depriving the works of the physical substance, but replicate the exact features of the original painting in a way that you will expect to see with human eyes (or even clearer and allows more freedom to observe in detail. Are there more we are asking from the museum? Is there something fundamentally essential and satisfying about the experience of physically walking through piece by piece? The answer can vary for different people, but the existence of virtual museums opens up the dialogue for it. 

Resources

“About: Museum of Contemporary Digital Art.” Accessed February 20, 2022. https://www.mocda.org/about.

“The History of the Kremer Collection.” The Kremer Collection. Accessed February 21, 2022. https://www.thekremercollection.com/history-of-the-collection. 

“Decentraland. “Accessed February 20, 2022. https://play.decentraland.org/?island=I9xzhu&position

=-21%2C116&realm=dg&server=loki-amber. 

“Home – The Kremer Collection.” The Kremer Collection. Accessed February 21, 2022. https://www.thekremercollection.com. 

“Intelligence 0296, VOV, Maurice Benayoun: Museum of Contemporary Digital Art.” MoCDA. Accessed 

February 20, 2022. https://www.mocda.org/maurice-benayoun-intelligence.

“The Kremer Museum.” The Kremer Collection. Accessed February 21, 2022. https://www.thekremercollection.com/the-kremer-museum/.

“Live Talks and Podcast: Museum of Contemporary Digital Art.” MoCDA. Accessed February 20, 2022. https://www.mocda.org/live-talks.

Magazine, Smithsonian. “The World’s First Entirely Virtual Art Museum Is Open for Visitors.” Smithsonian Institution, September 17, 2020. https://www.smithsonianmag.com/travel/worlds-first-entirely-virtual-art-museum-is-open-for-visitors-180975759/. 

“System shock – 777 exhibition.” Accessed February 20, 2022. https://www.mocda.org/system-shock-777-exhibition.

“Virtual Online Museum of Art (VOMA) Officially Opens Online.” Blooloop, January 18, 2021. https://blooloop.com/museum/news/virtual-online-museum-art-voma-open/. 

“Virtual Reality Is a Big Trend in Museums, but What Are the Best Examples of Museums Using VR?” MuseumNext, January 20, 2022. https://www.museumnext.com/article/how-museums-are-using-virtual-reality/.

“VOMA – Virtual Online Museum of Art: A Virtual Online Museum of Art.” VOMA virtual online museum of art. Accessed February 24, 2022. https://voma.space/. 

“World’s First Virtual Museum VOMA to Launch next Month – with Your Help.” FAD Magazine, May 7, 2020. Accessed February 24, 2022.https://fadmagazine.com/2020/05/07/worlds-first-virtual-

museum-voma-to-launch-next-month-with-your-help/. 

In a world where Siri can set alarms, give us directions, and look things up, shouldn’t machine translation be better by now? How does machine translation work? Why are museums still using human translation? And most importantly, what happens when machine translation is good enough that museums and other arts enterprises can use it without human oversight? 

This article explores how machine translation works and how it came to be in its current state, as well as how we evaluate state-of-the-art machine translation technology. Finally, this article examines how improvements in machine translation could disrupt and transform the museum industry.

How does machine translation work?

So, how does machine translation work? Underpinning machine translation is a field called natural language processing, or NLP for short. This area of study uses tools of computational linguistics combined with modern advances in computer science in order to “learn, understand, and produce human language content.” While NLP has progressed greatly since its inception, one foundational technique that is still used in some forms today is called “bag of words”, where all the words in a document are added up and each word’s frequency is calculated. This concept is a great illustration of many of the techniques used in NLP – counting words and calculating their relationship to other words within the same document or within a corpus of documents. 

Two other concepts to highlight within NLP are part-of-speech tagging and named entity recognition. Part-of-speech tagging is a computational linguistics tool in which descriptors (the tags) are assigned to each word’s role within a sentence. This allows for semantic understanding of the role each word plays, which is critical to grokking the full meaning of a sentence. Another important concept in NLP is named entity recognition, which is when a word or phrase that contains important information is identified and categorized. We need to be able to differentiate a name or location as distinct from an ordinary noun or pronoun to understand a sentence, and named entity recognition allows for that.

The techniques described above offer a window into how the field of natural language processing works and are illustrative of concepts integral to machine translation. Machine translation historically used statistical learning techniques similar to those described above. Launched in April 2006, Google Translate used a corpus of billions of words in both the origin and target languages to help craft computer-generated translations. While it is not the only company offering cutting-edge translation technology, Google has been an industry leader for many years.

With improvement in computing hardware as well as machine learning technology, Google was able to improve its translation service. In 2016, it launched a new version of its machine translation service that used neural machine translation instead of statistical models. Artificial neural networks are a type of machine learning algorithm that has quickly become useful in many industries. Using neural machine translation means that the model itself is made up of layers and layers of simple linear regression models. By combining these simple models, it mimics the way a human brain works in sending, translating, and receiving signals (hence why these models are referred to as “neural networks”). 

Google wasn’t done in 2016; although their current translation model still uses neural machine translation, they’ve made strides in other areas. Previously, they needed copious amounts of data in every language that they wanted to translate, but in using neural network machine learning techniques, their models can now perform better on low-resource languages, meaning they perform well even if they were not trained on very much data for that language. In 2019, Google released a massively multilingual, massive neural machine translation (M4) that was pretrained on over 25 billion sentence pairs and as a result, performed even better on low-resource languages. They have continued to focus on improving this low-resource language ability in order to expand their machine translation offerings, and as such released an updated model in 2020.

How do we know machine translation is improving?

So, machine translation works by taking natural language processing techniques, training a complex model with many layers of simpler models, and then inputting new text on which the model uses what it has learned. But how do researchers know that machine translation is improving? 

 There are a few metrics that exist for machine translation, and Google uses one called BLEU, which stands for “bilingual evaluation understudy”. The paper establishing the method was published in 2002, but despite its age the metric is still being used to evaluate cutting-edge language models. BLEU calculates a translation closeness score by evaluating each sentence as a unit and comparing them on a weighted average of words, allowing for other word choices that make sense in context. The score is then reported as a number between 0 and 1 with 1 being the highest score possible. Google has continued to evaluate its models using BLEU so that updated models can be easily compared to their predecessors.

Despite the progress that has been made, it’s notable to compare machine translation capabilities with natural language processing tools that we use in English every day. As mentioned in the introduction, people take for granted that they can use speech-to-text on their phone, or that they can interface with a chatbot on a website. Notably, a company in Canada called Cohere just raised $125 million to perform similar NLP tasks, including content moderation, conversational artificial intelligence, and search support. Another example of the expansion of English language model tools comes in the form of a startup called Forefront. Two weeks after the launch of an open-source large language model called GPT-NeoX-20b, Forefront announced that it was offering fine-tuning services to make the use of GPT-NeoX-20b more accessible. All these markers of progress demonstrate that NLP in English is rapidly advancing and expanding. 

Why isn’t machine translation good enough?

If AI can write papers and chat with us, why can’t it fluently translate my words into any language without errors? One of the shortcomings of large language models is that they require copious amounts of data to be effective. While there has been considerable effort to train machine translation models, there just isn’t as much training data available in many languages as there is in English. Despite Google’s best efforts to train their translation models to perform well on low-resource languages, they still aren’t nearly as good as the models that use English. And not only are the models not as good, they are also insufficient for professional use even in high-resource languages. As researcher Aihua Zhu states, current machine translation technology fails to meet the needs of use in professional contexts.

Despite the clear advantage that English has over other languages, there are still numerous shortcomings that exist with English large language models. Because many of them were trained on text from the internet, they tend to have biases such as stereotypes towards people of minority gender or ethnic identities. A recent illustration of this is that YouTube’s automated captioning service was spotted using profanity in captions of videos for young children. So while language models in English perform well enough to be useful to large swathes of society, they still have limitations. Comparatively, the limitations of multilingual language models are greater and do not allow for as much integration into high-level societal tasks, although they are very useful for many people (in 2016 Google stated that Google Translate has 500 million users and translates over 1 billion words a day). 

How are museums translating their material?

Turning back to the museum industry, we consider the current state of translation resources available to them. How are museums currently offering material in multiple languages? Many museums are employing professional translation services. For instance, the Field Museum in Chicago uses a company called Multilingual Connections, and the Denver Botanical Gardens, South Florida Science Center, and the Metropolitan Museum of Art use a company called Eriksen Translation. That is, museums must pay for professional human translators in order to offer material in multiple languages. When the Children’s Discovery Museum of San Jose decided in 2015 to translate materials so that they could engage more with Latino visitors, they decided to exclusively use human translations rather than machine translation because they wanted to ensure that their materials were true to the “spirit of the words,” not just that it offered visitors the gist of what was going on. 

While all of these are museums in the United States, it also seems that countries in Europe use a similar model of hiring professional translators – two contract bidding announcements by the Museum of the Quai Branly – Jacques Chirac and the Museum of Bastia in France demonstrate that human translation is still the industry standard in Europe as well. 

A great illustration of the current state of machine translation as useful but not quite good enough comes in an announcement for a translation sprint. Europeana Pro is an organization funded by the European Union that is dedicated to preserving and promoting cultural heritage. In their 2020 invitation to join sprint to translate foundational documents into more languages, the organizers of the sprint instructed participants to use a number of machine translation tools only “as long as you review the resulting text.” This instruction underlines the themes explored by this paper: machine translation is good, but not yet good enough for professional use without human review.

How will museums be affected when machine translation achieves parity with human translation?

So machine translation clearly is not yet good enough – but how could it change once the technology catches up to the need? User experience is naturally a priority for museums, and translation is an integral part of that. There are many museums in America that could better serve their non-English-speaking populations by expanding their offerings. And it isn’t just in serving their visitors; museums could also benefit greatly from better machine translation for their collections. A team of researchers from Beijing Jiaotong University used machine translation to evaluate ancient Chinese texts and translate them into English, demonstrating the promise of machine translation as a way to preserve and expand collection offerings. A research group based out of the University of California Los Angeles (UCLA) has a program dedicated to using automated translation to analyze and understand cuneiform languages on a series of tablets from southern Mesopotamia. A widely-used museum database software called Axiell has multilingual fields embedded in its data structures, and offers an automated translation tool so that curators have some idea of what they’re looking at even if it is in a language they don’t understand.

Another team of researchers commented on translation in museums as a narrative and a means by which to convey identity. They explore how bad translation can be seen as a failure of the museum and argue that the quality of the translation not only has an impact on the museum, but also on the message that the museum wants to convey. A professor and researcher from the United Kingdom synthesized the benefits of translation for museums into two central themes: economic value and social inclusivity. However, given that a professional translation service is often required, that is currently not always an option for museums. 

While professional translation services are the norm, some museums are starting to explore machine translation as a potential tool. The Computer History Museum conducted an experiment with artificial intelligence during which they used machine translation on a number of their audio and video files. While it is unclear if they have put it into effect so far, they underscored many of the points made in this article by commenting that machine translation promises to make museums and collections “more accessible to speakers of languages other than English.” 

All of this evidence points to the potential disruption and change that could occur for museums when machine translation becomes a ubiquitous option for professional-level translation. Besides machine translation’s promise of museums’ ability to better serve communities in America, the international museum industry also thrives off the ability to offer people a window into culture and identity that isn’t possible without accurate translations. The significant reduction of costs that would occur if machine translation improved would offer museums the ability to broaden their offerings and expand their visitor experience beyond their current capability.

References

Axiell. n.d. “Axiell – Bringing Culture and Knowledge to Life.” Axiell. Accessed March 2, 2022a. https://www.axiell.com/.

Axiell. n.d. “Interface Functionality: Editing or Translating Adlib Interface Texts.” Accessed March 2, 2022b. http://documentation.axiell.com/alm/en/ds_tmeditingtranslatingtexts.html#autotranslate.

Axiell. n.d. “Translations Manager: Introduction.” Accessed March 2, 2022c. http://documentation.axiell.com/alm/en/ds_tmintroduction.html.

Bapna, Ankur, and Orhan Firat. 2019. “Exploring Massively Multilingual, Massive Neural Machine Translation.” Google AI Blog (blog). October 11, 2019. http://ai.googleblog.com/2019/10/exploring-massively-multilingual.html.

Chen, Chia-Li, and Min-Hsiu Liao. 2017. “National Identity, International Visitors: Narration and Translation of the Taipei 228 Memorial Museum.” Museum and Society 15 (1): 56–68. https://doi.org/10.29311/mas.v15i1.662.

“Cohere.” n.d. Cohere. Accessed March 2, 2022. https://cohere.ai.

David C. Brock. 2022. “A Museum’s Experience With AI.” CHM. February 3, 2022. https://computerhistory.org/blog/a-museums-experience-with-ai/.

Eriksen Translations Inc. 2019. “Museum Audience Engagement: Translation Strategies to Promote Diversity.” Eriksen Translations Inc. September 19, 2019. https://eriksen.com/arts-culture/museum-audience-engagement-translation-strategies/.

Eriksen Translations Inc. n.d. “Translating and Typesetting the Met Guides into 6 Languages.” Eriksen Translations Inc. Accessed March 2, 2022a. https://eriksen.com/work/met-guides/.

Eriksen Translations Inc. n.d. “Translating Exhibition Materials for the South Florida Science Center.” Eriksen Translations Inc. Accessed March 2, 2022b. https://eriksen.com/work/exhibition-displays/.

Eriksen Translations Inc. n.d. “Translating the Garden Tool Mobile Website for Denver Botanic Gardens.” Eriksen Translations Inc. Accessed March 2, 2022c. https://eriksen.com/work/garden-tool-mobile-website/.

Eriksen Translations Inc. n.d. “Translation & Localization Services NYC | Eriksen Translations.” Eriksen Translations Inc. Accessed March 2, 2022d. https://eriksen.com/.

“Forefront: Fine-Tune and Deploy GPT-J, GPT-13B, and GPT-NeoX.” n.d. Accessed March 2, 2022. https://www.forefront.ai/helloforefront.com/.

Field Museum. 2018. “About.” Text. Field Museum. May 14, 2018. https://www.fieldmuseum.org/about.

Furui, S., T. Kikuchi, Y. Shinnaka, and C. Hori. 2004. “Speech-to-Text and Speech-to-Speech Summarization of Spontaneous Speech.” IEEE Transactions on Speech and Audio Processing 12 (4): 401–8. https://doi.org/10.1109/TSA.2004.828699.

Goel, Aman. 2018. “How Does Siri Work? The Science Behind Siri.” Magoosh Data Science Blog (blog). February 3, 2018. https://magoosh.com/data-science/siri-work-science-behind-siri/.

Hirschberg, Julia, and Christopher D. Manning. 2015. “Advances in Natural Language Processing.” Science 349 (6245): 261–66. https://doi.org/10.1126/science.aaa8685.

IBM Cloud Education. 2021. “What Are Neural Networks?” August 3, 2021. https://www.ibm.com/cloud/learn/neural-networks.

Lalwani, Tarun, Shashank Bhalotia, Ashish Pal, Vasundhara Rathod, and Shreya Bisen. 2018. “Implementation of a Chatbot System Using AI and NLP.” SSRN Scholarly Paper ID 3531782. Rochester, NY: Social Science Research Network. https://doi.org/10.2139/ssrn.3531782.

Leahy, Connor. 2022. “Announcing GPT-NeoX-20B.” EleutherAI Blog. February 2, 2022. https://blog.eleuther.ai/announcing-20b/.

Liao, Min-Hsiu. 2018. “Museums and Creative Industries: The Contribution of Translation Studies.” January 2018. https://jostrans.org/issue29/art_liao.php.

Marshall, Christopher. 2020. “What Is Named Entity Recognition (NER) and How Can I Use It?” Super.AI (blog). June 2, 2020. https://medium.com/mysuperai/what-is-named-entity-recognition-ner-and-how-can-i-use-it-2b68cf6f545d.

Martin, Jenni, and Marilee Jennings. 2015. “Tomorrow’s Museum: Multilingual Audiences and the Learning Institution.” Museums & Social Issues 10 (1): 83–94. https://doi.org/10.1179/1559689314Z.00000000034.

Matas, Ariadna. 2020. “OpenGLAM Translation Sprint at Europeana 2020.” Europeana Pro. September 2020. https://pro.europeana.eu/event/openglam-translation-sprint-at-europeana-2020.

Melisa Palferro. 2018. “Different Approaches to Museum Translation.” #ucreatewetranslate (blog). March 8, 2018. https://www.ucreatewetranslate.com/different-approaches-museum-translation/.

Menon, Yasmin, and Will Lach. 2021. “Creating Accessibility in Museums with High Quality Translations.” Eriksen Translations Inc. November 19, 2021. https://eriksen.com/arts-culture/accessibility-museums-high-quality-translations/.

Mingyu, Lu, and Si Xianzhu. 2010. “Application of Machine Translation to Chinese-English Translation of Relic Texts in Museum.” In 2010 International Conference on Intelligent System Design and Engineering Application, 1:355–58. https://doi.org/10.1109/ISDEA.2010.4.

Mitkov, Ruslan. 2004. The Oxford Handbook of Computational Linguistics. OUP Oxford. https://books.google.com/books?hl=en&lr=&id=yl6AnaKtVAkC&oi=fnd&pg=PA219&dq=part+of+speech+tagging&ots=_VVi1buLDj&sig=y-3BFecOVtTtptCNFrS8Br-EFGE#v=onepage&q=part%20of%20speech%20tagging&f=false.

MTAAC Team. n.d. “Machine Translation and Automated Analysis of Cuneiform Languages.” Machine Translation and Automated Analysis of Cuneiform Languages. Accessed March 2, 2022. https://cdli-gh.github.io/mtaac/.

Multilingual Connections. n.d. “Museum Translation Services.” Multilingual Connections. Accessed March 2, 2022. https://multilingualconnections.com/industry-experience/museums-cultural-institutions/.

Nadeem, Moin, Anna Bethke, and Siva Reddy. 2020. “StereoSet: Measuring Stereotypical Bias in Pretrained Language Models.” ArXiv:2004.09456 [Cs], April. http://arxiv.org/abs/2004.09456.

Och, Franz. n.d. “Statistical Machine Translation Live.” Google AI Blog (blog). Accessed March 2, 2022. http://ai.googleblog.com/2006/04/statistical-machine-translation-live.html.

Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. “Bleu: A Method for Automatic Evaluation of Machine Translation.” In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311–18. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics. https://doi.org/10.3115/1073083.1073135.

Scott, Josh. 2022. “Cohere Closes $125 Million USD Series B Round Led by Tiger Global.” BetaKit (blog). February 15, 2022. https://betakit.com/cohere-closes-125-million-usd-series-b-round-led-by-tiger-global/.

Siddhant, Aditya, Melvin Johnson, Henry Tsai, Naveen Ari, Jason Riesa, Ankur Bapna, Orhan Firat, and Karthik Raman. 2020. “Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation.” Proceedings of the AAAI Conference on Artificial Intelligence 34 (05): 8854–61. https://doi.org/10.1609/aaai.v34i05.6414.

Simonite, Tom. 2022. “YouTube’s Captions Insert Explicit Language in Kids’ Videos.” Wired, February 24, 2022. https://www.wired.com/story/youtubes-captions-insert-explicit-language-kids-videos/?utm_source=nl&utm_brand=wired&utm_mailing=WIR_FastForward_022822&utm_campaign=aud-dev&utm_medium=email&utm_content=WIR_FastForward_022822&bxid=60a682d211af1a6455755091&cndid=65158197&esrc=bouncexmulti_second%20&source=EDT_WIR_NEWSLETTER_0_TRANSPORTATION_ZZ&mbid=mbid%3DCRMWIR012019%0A%0A&utm_term=WIR_Transportation.

Turovsky, Barak. 2016a. “Ten Years of Google Translate.” Google. April 28, 2016. https://blog.google/products/translate/ten-years-of-google-translate/.

Turovsky, Barak. 2016b. “Found in Translation: More Accurate, Fluent Sentences in Google Translate.” Google. November 15, 2016. https://blog.google/products/translate/found-translation-more-accurate-fluent-sentences-google-translate/.

Zhu, Aihua. 2021. “Man-Machine Translation—Future of Computer-Assisted Translation.” Journal of Physics: Conference Series 1861 (1). http://dx.doi.org/10.1088/1742-6596/1861/1/012088.