Introduction
With the rapid development of technology, people’s lives are increasingly dependent on the Internet, and the boundaries between virtual and reality are blurring. We create the metaverse, a virtual world where people can work, play, and socialize. Since the virtual world is similar to the real world, will there be virtual residents besides the real human? Will there be virtual superstars who can get the fervent love of fans like the real idols? It sounds strange, but virtual idols are more popular these days than you can imagine.
What is a virtual idol?
Virtual idols are performers who do not have a living body and only exist in the virtual world. They have their own personalities, can sing, dance, and create more daily life possibilities like real humans. The technologies to create a virtual idol includes 3D modeling, artificial intelligence, motion capture, and virtual reality/augmented reality.
The concept of virtual idols has a history of around thirty years. Since virtual idols are a synonym for all virtual performers, there are different types of virtual idols. The simplest version of virtual idols is mainly animation characters. As an IP product that the animation company created, fans learn more about their personalities and backstories through anime and related games, music, and live concerts online. With the development of 3D models and real-time motion capture software, virtual idols become more like real humans in appearance and are able to make more real-time interactions. The virtual influencers, also known as vtubers, have a real person to control the figure, live-stream behind an animated face. Some other virtual idols are avatars of real idols in the virtual world to make more interactions with fans.
In this article, we will focus on one of the cases of virtual idols. They are customized idols that do not have a specific performer behind them, can sing any song that you want and be whatever you want them to be. The representative figures are Hatsune Miku and Luo Tianyi. As “superstars” who have sung on the stages of many concerts and music festivals, performed with real artists, and have countless fans around the world, they are actually software voicebank developed by Yamaha Vocaloid: an AI singing synthesis technology that can simulate human’s voice and produce songs by computer.
How were these idols created?
Yamaha Vocaloid is software to produce music by computer. It is like a “Siri-meets-Garage”. It records human voice in short samples, and these samples are stored in a database and constructed into a keyboard-style instrument within the Vocaloid software. Then anyone could produce their own songs by inputting lyrics and an instrumental melody into the software. Now, by incorporating the technology of deep learning and neural network, Vocaloid can do a better job to analyze the recordings of the real vocalists’ songs and speech. It can make the tunes and timbre closer to the vocalist, include the unique mannerisms and nuances, and clone the voice from just a few minutes of recordings.
The Vocaloid idols will take voice samples from a specific voice actor/actress. But its persona and music productions are completely independent of the voice provider. After having the initial image and character set by the company, their music could come from any creator who has purchased the software. Fans around the world will continuously add their detailed personalities and backstories.
Famous Vocaloid idols and their performance
Hatsune Miku is one of the most famous Vocaloid idols. The 16 years old girl with long, turquoise twintails, whose name means “The first sound of the future”, has millions of followers on social media platforms. She was created in 2007 and takes voice samples from Japanese voice actress Saki Fujita. Now she has released over 100,000 songs, 170,000 YouTube videos, and 1,000,000 artworks. Similarly, there is another popular Vocaloid idol in China, Luo Tianyi. She is a 15 years old girl dressed in Chinese elements who can detect “the song of your heart”.
Besides performing in the virtual world, these idols also attend music festivals and concerts like real singers. By using holograms or augmented reality, they are able to show up on stages and perform with real performance. Miku has opened for Lady Gaga’s tour and attended the Coachella festival. She also has live concerts worldwide every year. She toured Europe in 2020, performing in cities such as London and Paris. In the concert, Miku was projected onto the screen with live bands and backup dancers. The audience in all sorts of fancy outfits gathered in the venue, screaming and waving the glow sticks like every normal concert.
As a popular idol in China, Luo Tianyi also appears on various shows. She had a live performance accompanied by world-famous pianist Lang lang using the hologram in 2019 and performed on China Central Television Spring Festival Gala, which is the most popular show on the Chinese new year’s eve in 2021.
The live performance with Vocaloid idols has many advantages. Since the performers and stage settings are virtual, they’re not bound by physical costume changes and breaks in a setlist. They can change the costume and stage backdrop in a second. Some complex and exaggerated settings and effects can also be achieved. Although there is no need to cover the star’s travel and lodging fees and stage setting expenses, it is not a cost-saving performance. For the two hours’ Luo Tianyi performance, a team of 200 people from China and Japan worked for roughly six months. They used 3D modeling and motion capture to create the appearance and personalities. During the performance, a backstage voice dubber and motion-capture actress allowed Tianyi to interact with Lang Lang. Despite its complexity, it is still interesting to try things like that to break the boundaries between virtual and reality.
How can virtual idols change the entertainment industry?
Firstly, we should know why Vocaloid virtual idols are so popular. As virtual beings, they are perfect positive figures. A real idol may do things that do not match your imagination or even have scandals. But a virtual idol will never make you frustrated. Besides, they will always have time and energy, and stable vocal skills to provide you with a perfect performance. Because they do not need to consider physical function, they can make some performances that real idols can not complete. The design of props and scenes in the performances can also be more innovative. Besides, the Vocaloid technology can not only create idols for animation fans but also “clone” the deceased legendary singers. Yamaha has presented a live performance of a new song by Hibari Misora, a singer who passed away thirty years ago.
In addition, Vocaloid idols are idols created by everyone. They have the advantage of embodying not only the dreams of followers but also their creative skills. Since there are only basic settings, you can create your own version of Miku or Tianyi by writing stories or making artworks of them. Every fan is a part of the creation. That is why Miku has been popular in the 15 years since she was born — there are always new fans to bring different works. Virtual idols are not bound to a specific genre of music. Different songwriters give her different colors. Even songs in the live concerts are written by fans. Vocaloid company opens the use of the idols’ voices and images to the community, and they will check the most popular songs online and contact the writer to ask for permission of using them. Since the creators are mainly fans, it is an easy process for the management team. Some of Miku’s concerts were promoted with songwriting contests. Miku would perform the winning entry and sing songs that were written by individuals with no direct tie to her company. It may be a trend in the music industry, which is a decentralized model of music production. Non-professional people use virtual idols as a carrier to create great works and be heard while keeping this virtual idol in the spotlight. It is difficult for real idols to achieve the win-win situation.
The copyright issues
Since the essence of this model is to create works by fans, it is important to talk about the copyright. In the case of Vocaloid idols, the right of using Vocaloid software belongs to Yamaha, and the avatars belong to their respected companies. The companies control the intellectual property and brand image. However, it is hard to control the fan-made artworks due to their wide presence and rapid spread on the Internet. As a result, many Vocaloid idol companies do not profit by controlling what fans can do with their characters or clamping down on pirating or illegal downloads. For example, Miku’s company opens the right of using the image. As said in the previous part, it mainly gets profit by making promotions through fanfiction, choosing the best works such as songs and videos, and partnering with big companies such as SEGA and Toyota to make games, throw concerts, and create ad campaigns.
However, copyright is still a big issue for most virtual idols, especially for those who represent the real person. For example, if an avatar is created based on a real idol, the right to use the image after the artist leaves the company needs to be discussed. Besides, the creation of a virtual idol needs a team of engineers and artists. If the group of creators comes from different parties, the distribution of the rights can be complex.
The future of virtual idols
Having the advantages of perfect personalities, excellent performance skills, and opportunities for fans to make customizations, virtual idols may be a trend in the entertainment industry. In the future, there will be more collaborative performances by real and virtual idols. Although the current virtual idols are sometimes stiff in their voices and require the assistance of real humans to complete the interactions, as technology advances, these easily accessible idols will become an indispensable part of the entertainment industry.
Resources
Cryer, Hirun. “Nothing Could Have Prepared Me for Seeing Hatsune Miku Live.” USgamer.net. USgamer, January 24, 2020. https://www.usgamer.net/articles/nothing-could-have-prepared-me-for-seeing-hatsune-miku-live.
Hamamoto, Ben. “Making Miku.” Institute for the future, April 10, 2018. https://www.iftf.org/future-now/article-detail/making-miku/.
Jenkins, Mark. “This Singer Is Part Hologram, Part Avatar, and Might Be the Pop Star of the Future.” The Washington Post. WP Company, July 8, 2018. https://www.washingtonpost.com/entertainment/music/this-singer-is-part-hologram-part-avatar-and-might-be-the-pop-star-of-the-future/2018/07/05/e2557cdc-7ed3-11e8-b660-4d0f9f0351f1_story.html.
Ke, Jiayun. “Live Performances with Entertainers Who Aren’t Alive.” SHINE, February 24, 2021. https://www.shine.cn/news/in-focus/2102245044/.
“Legal Implications.” Vocaloid Wiki. Accessed May 4, 2022. https://vocaloid.fandom.com/wiki/Legal_Implications.
Li, Donna. “The Rise of Vocaloid, a Voice Synthesizing Software That Has Revolutionized the Digital Music Industry.” The Science Survey, June 22, 2021. https://thesciencesurvey.com/arts-entertainment/2021/06/22/the-rise-of-vocaloid-a-voice-synthesizing-software-that-has-revolutionized-the-digital-music-industry/.
“Luo Tianyi.” Vocaloid Wiki. Accessed May 4, 2022. https://vocaloid.fandom.com/wiki/Luo_Tianyi#cite_note-article1-13.
Poole, Robert Michael. “Fans Interact with Holographic Superstar Hatsune Miku.” Japan Today, September 2, 2011. https://japantoday.com/category/features/fans-interact-with-holographic-superstar-hatsune-miku.
Tangermann, Victor. “Thousands of Chinese Fans Paid to See a Virtual Hologram Singer Live.” Futurism. Futurism, March 4, 2019. https://futurism.com/virtual-idol-hologram-luo-tianyi-lang-lang-live.
Vale, Chris. “An Alternate Dimension: Virtual Idols – New Technology – UK.” An Alternate Dimension: Virtual Idols – New Technology – UK. Rouse, December 3, 2020. https://www.mondaq.com/uk/new-technology/1011984/an-alternate-dimension-virtual-idols.
“VOCALOID: from Hatsune Miku to Hibari Misora.” Music Technology Group, January 27, 2020. 2.https://www.upf.edu/web/mtg/news/-/asset_publisher/WM181VyAQipW/content/id/231857712/#.YnGAw9PMJEY.
“Who Is Hatsune Miku?” About HATSUNE MIKU | CRYPTON FUTURE MEDIA. Accessed May 4, 2022. https://ec.crypton.co.jp/pages/prod/virtualsinger/cv01_us.
“Yamaha Vocaloid:AI™ Faithfully Reproduces Singing of Legendary Japanese Vocalist Hibari Misora.” Yamaha VOCALOID:AI™ Faithfully Reproduces Singing of Legendary Japanese Vocalist Hibari Misora – News Releases – Yamaha Corporation. Accessed May 4, 2022. https://www.yamaha.com/en/news_release/2019/19100801/.
Ye-eun, Jie. “K-Pop Idols Bond with Gen Z and Alpha Fans on Metaverse Platforms.” The Korea Herald. The Korea Herald, March 20, 2022. http://www.koreaherald.com/view.php?ud=20220320000143.
Yu, Cheng. “Virtual Idol Means Music to Stock Market’s Ears.” Chinadaily.com.cn, January 5, 2022. http://global.chinadaily.com.cn/a/202201/05/WS61d4f406a310cdd39bc7f1dc.html.
Zhang, Jane. “Virtual Idols Leave Everything to Their Fans’ Imagination.” South China Morning Post, March 1, 2019. https://www.scmp.com/tech/article/2188238/chinas-virtual-idols-meet-their-fans-intersection-entertainment-and-technology.
Zoladz, Lindsay. “Hatsune Miku Is a Piece of Software. She May Also Be the Future of Music.” Vulture, November 19, 2014. https://www.vulture.com/2014/11/hatsune-miku-the-future-of-music.html.