Why Nars relied on humans to bring its ‘virtual’ ambassadors to life

This article on virtual ambassadors is part of our Vogue Business membership package. To enjoy unlimited access to our weekly Technology Edit, which contains Member-only reporting and analysis and our NFT Tracker, sign up for membership here. At Sephora’s annual beauty conference, Sephoria, Nars brought its life-sized virtual ambassadors out to play. For the first time, Maxine, Chelsea and Sissi — also known as the Nars Power Players — met customers in the physical world, where they could chat about beauty tips and pose for selfies. The twist is that the interactions weren’t fully virtual. To animate the characters, which were made using Epic Games’s 3D software Unreal Engine, a trained actor wore a motion capture suit in a hidden area behind the booth. The actor could see and hear the visitors via a camera, and because of the motion capture technology, their responses and movements were instantly translated into the character’s on-screen body. So, while the ambassadors’ images and characters are fictional, the real-time, animated conversations were human-led. The brand explored numerous mechanisms to bring the experience to life, says Dina Fierro, SVP of the Web3 and metaverse group at Nars’s parent company, Shiseido Americas. A “human-in-the-loop” emerged as the best route in place of a fully automated experience, largely because real-time motion capture generates significant engagement, and because a trained actor can have more fun, in-depth conversations — while also adapting to unexpected interactions. “It immediately sparks curiosity. You genuinely get a degree of surprise and delight from the consumer because they don’t necessarily understand how this incredibly realistic interaction is coming to life,” Fierro says. Naturally, many visitors did ask “probing” questions, just the way many do when interacting for the first time with an AI assistant like Alexa or Siri. The actor was then able to answer in a witty way, Fierro says. “That was the safest path forward in this current moment.” Generative AI, which can create realistic-seeming deep fakes, fictional images and authoritative-sounding text, has raised concerns as it’s entered the mainstream. Meta’s AI characters and AI chatbot, for example, use generative AI to generate responses that can seem more conversational and human; both Meta’s and Amazon’s new smart glasses will now come with their own generative AI assistants who can be awakened on demand. It’s specifically because the outputs are so convincing that they can be risky. Ethical considerations, according to a report by Deloitte, include concerns that responses might not indicate if the AI is uncertain about a result, the source of the information may not be easily identified or could be unreliable, and bias might be introduced. A report by advocacy organisation Public Citizen warns that “counterfeit humans” and “anthropomorphic AI” can manipulate people and, in the case of a marketing effort, might better convince people to spend money — similar to how a human associate would. This is partly why the CEO of OpenAI, which makes the popular AI tool ChatGPT, has joined other tech leaders to call for cooperation on establishing global regulations.There is also the general cultural uneasiness of interacting with a machine that mimics a human. Researchers call this discontent the “uncanny valley”; as robots get mostly lifelike, humans detect that something isn’t quite right. Fierro is well aware of the mixed consumer sentiment around virtual ambassadors, especially on Web2 social platforms. “As long as you’re speaking to the appropriate audience on social, the response is largely positive. But, as soon as you’re reaching that broader and more mass audience, you start to see a lot of the mixed sentiment come in and the concerns that people have about AI.” There are also concerns in terms of protecting the brand. Despite the enthusiasm about the efficiencies and creative potential of AI, beauty brands, after years of cultivating a spirit of authenticity, have reservations, she says. “They don’t want to lose touch of how important that is to consumers.” With avatars and artificially intelligent assistants all the rage, many visitors assumed, at first, that Nars’s virtual characters were fully powered using AI. However, the goal of bringing the virtual ambassadors to Sephoria wasn’t to create a fully automated experience; rather, it was to engage the audience, introduce them to Nars’s virtual ambassadors and to promote the brand’s new lip pencil product. It was also a chance for Nars to test consumer appetite and response to virtual personalities. “It felt like this magical moment that they were having with the brand. And it’s really difficult to put a value on that, because the engagement itself was genuinely so rich in terms of the time spent with each consumer,” Fierro says. As soon as the characters commented on something as specific as a visitor’s shoes, for example, reactions shifted, often leading to an engaging and warm interaction, Fierro says. “You saw the unlock for them when they began to engage in the real-time conversation with the avatar. That was the moment when a lot of guests realised that they were not engaging with AI.” The line between what’s AI and what’s not has blurred, and developers have found that familiarity can make new technology more palatable. A human touch — whether visible or not — often makes these technologies more appealing and accessible to consumers. Earlier this month, Meta announced new AI assistants with the faces and voices of well-known personalities, including Kendall Jenner, Paris Hilton and Snoop Dogg. Model Eva Herzigová recently digitised her likeness so that her avatar could appear in photoshoots in perpetuity. Both H&M and Burberry have experimented with lifelike holograms using tech from Proto, whose large rectangular boxes can create the perception that people and products are standing there in the flesh. In Tokyo, Rtfkt and Nike showed a collection of digital Air Force Ones on virtual humans designed by artist Takashi Murakami in Proto Epic hologram units, controlling the movements of the characters via dancers wearing motion capture suits. Avatars and AI personalities are also migrating out of the metaverse and into both physical and Web2 spaces: Meta’s AI assistants converse on Whatsapp, Messenger and Instagram. To promote a physical clothing collection inspired by “the immersive potential of the metaverse”, H&M hired AI avatar Kuki to walk in an ad campaign on Instagram, rather than a metaverse world. Meta reported an 11 times higher increase in “ad recall” with a mix of virtual creator content and campaign video versus campaign video alone. “In a way, the metaverse is collapsing from 3D back to 2D, with AI NPCs [non-playable characters] coming to messaging apps — and not virtual worlds — because that’s where most mainstream consumers still spend their time,” says Lauren Kunze, CEO and co-founder at Iconiq, which makes AI-driven virtual characters, including Kuki. Kunze says that the company has seen an uptick in interest in virtual personalities from brands, especially in light of Meta’s forthcoming AI Studio, which will enable brands to build upgraded custom chatbots similar to Meta’s celeb-inspired personalities. (Brands can’t “hire out” the celeb-backed personalities created by Meta, but hypothetically, a brand like Nars could animate its own ambassadors in a similar way.) Virtual personalities are already big business. The human celebrities hired by Meta to lend their likeness were reportedly paid $5 million. The company that created Lil Miquela was valued at $150 million before it was acquired by Dapper Labs in 2021. Synthetic celebrity company Superplastic, which has worked with Gucci and Tommy Hilfiger, has raised $58 million. Chatbot studio Character AI, which can impersonate people, is raising funding at a $5 billion valuation. For brands, virtual influencers can bring a host of advantages, including the ability to more quickly tap into trends, to appeal to Gen Z (which makes up more than half of those who follow virtual influencers on Instagram) and to have longevity, as a Gen Z-aged virtual influencer won’t age out of their target demographic, says The Soul Publishing VP of operations Patrik Wilkens, creator of “teenage musician” Polar, who is a virtual influencer whose TikTok channel has more than two million followers. Virtual influencers like Polar also bring platform versatility, meaning their talent can easily be translated to various platforms and formats. Wilkens advises brands to establish strong creative teams and to keep the target audience in mind when creating a virtual influencer. Going forward, there’s an opportunity to use AI to scale Nars’s three ambassadors’ roles to gaming and metaverse spaces. Nars already has a presence on Roblox and Zepeto. Based on the consumer interactions at Sephoria, Fierro says, there is also clear potential to bring the avatars into other-real world environments through future digital displays and installations. The same types of brand guidelines that were used to train the human actor on each persona could ultimately be used to train an AI tool (perhaps without quite as much leeway). Asian consumers, particularly in China, might be more ready, Fierro says. (In China, virtual personalities have been a way for brands to circumvent the government’s banning of celebrity and idol culture, calling it harmful to mental health.) In the vein of photorealistic human avatars and chatbots that sound like supermodels, future experiments will be less about the tech and more about what the tech enables, she says. “There’s a degree of inevitably around the metaverse and Web3 — even if these are terms we don't use in a consumer-facing capacity.” To receive the Vogue Business newsletter, sign up here. Comments, questions or feedback? Email us at feedback@voguebusiness.com. More on this topic: Can China’s virtual idols have real influence? Why Nars’s new brand ambassadors are all virtual What influencer marketing looks like in the metaverse

Make better business decisions

Make better business decisions