How Sephora is leveraging AR and AI to transform retail and help customers buy cosmetics


Beauty retailer Sephora followed customer technology trends to command the cosmetic industry worldwide. Here’s an inside look at its successful digital transformation.

In San Francisco’s Dogpatch district, a research laboratory explores new products that tap augmented reality (AR), artificial intelligence (AI), and RFID technology. Like many Silicon Valley startups, its team develops and pitches cutting-edge solutions and customer service experiences, promising to make life easier for its consumers.

But, this is no robotics company or maker of self-driving cars. It’s Sephora, the industry-leading chain of cosmetic stores that has leveraged digital transformation to take pole position as the number one speciality beauty retailer in the world.

“Digital and innovation have always been part of our DNA at Sephora,” said Mary Beth Laughton, Sephora’s executive vice president of omni retail. “We are very focused on our customers, and we know that her life is increasingly reliant on digital. So we know to be successful as a retailer, we’ve got to be where our clients are, and give her tools and experiences that meet her needs.”

While other cosmetic companies rely heavily on department store sales, Sephora offers customers a number of tech options that allow them to personalize their shopping experience by trying on makeup virtually using AR, matching their skin tone to a foundation with AI, and sampling a fragrance via a touchscreen and scented air.

Further, in Sephora’s Innovation Lab, launched in 2015, a team of executives hired from the marketing, product development, and technology industries source, develop, evaluate, test, and ultimately launch new offerings and technologies for shopping in the store and online.

“It’s about being open to new ideas, and working with partners to develop the right solutions—and being willing to do things that maybe our clients don’t even know that they want yet,” Laughton said. “That’s how we approach things, and as long as we’re doing it through the lens of our customer and finding ways to simplify and enhance their shopping experience, we’re willing to put focus there.”

Here’s an inside look at how the company has digitally transformed the beauty space to better meet the needs of customers and dominate the market—in a rapidly-changing retail environment.

Digital first

Sephora’s beauty retail concept was founded in France by Dominique Mandonnaud in 1970, and acquired by LVMH Moët Hennessy Louis Vuitton in 1997. The company now operates some 2,300 stores in 33 countries worldwide, with more than 430 stores across the Americas.

The company launched its website in the US in 1999, and kept moving forward in digital. It also invested early in mobile as more than just a place to make purchases.

Perhaps its most innovative and accessible mobile offering is Sephora Virtual Artist, an AR tool that allows customers to try on thousands of shades of lipstick, eyeshadow, false lashes, and many other makeup products sold at Sephora. It also lets users go through beauty tutorials on their own face digitally to learn how to achieve certain looks. A new feature called Color Match taps AI to help customers find the right color shade for their skin tone via an uploaded photo. Virtual Artist is available in the Sephora app as well as in select stores.

“Sephora Virtual Artist is a really good example of where there was a real customer need,” Laughton said. “It can be overwhelming coming into our stores or shopping online, but this makes it easy to help you find your favorite shade and save you time.”

Sephora worked with ModiFace to develop the technology. “We are based on technology developed at Stanford University and included face tracking and computer vision,” said Parham Aarabi, CEO of ModiFace. “But we found that in a beauty space, there was a need for this visualization, and that’s how we got started.”

For the past five years, ModiFace and Sephora experimented with AR. “A few years ago, Sephora realized that the technology had come to a certain level of realism and quality that they felt it could impact their business in a dramatic way,” Aarabi said. Since launching in 2016, every four months, ModiFace and Sephora add in more features or expand to other platforms.

The core of the technology is the ability to track facial features accurately, Aarabi said. “We have essentially trained a program that can measure where the lips and eyes are in real time and track those facial feature points,” he added. “Once we know where those elements are on the face, we can then know where the lipstick or eye shadow should go.”

ModiFace worked closely with Sephora to ensure every single color of a virtual product matches the product in real life, Aarabi said. As of this writing, the Virtual Artist includes more than 20,000 products sold at Sephora, including highlighters, blushes, concealers, lipsticks, and eye shadows.

“Before Sephora, we would have to go to brands and try to motivate them and show them why technology could make sense for their business,” Aarabi said. “Sephora has gotten it from day one, wanting and incorporating new ideas. It’s great to have a partner that believes in technology.”

Since launching on the app, Sephora Virtual Artist has seen 200 million shades tried on, and over 8.5 million visits to the feature, according to the company.

Sephora’s app also includes product ratings and reviews, and a feature that allows you to test out different hairstyles on the Android version.

Today, most clients visit the app several times per month, to check everything from content to personalized messaging, according to the company.

“What’s common across all of our digital efforts is that we’re not just doing technology for the sake of it or because it’s cool or new,” Laughton said. “Every digital product or tool or experience that we introduce is done to make shopping for our consumer more fun and more efficient—to help engage her, educate her, and entertain her.”

Innovation and inspiration

Finding the correct foundation color was one of the largest beauty concerns of Sephora’s customers, Laughton said. In response, the company partnered with Pantone to develop a shade-matching technology called Color IQ, a handheld device that will soon be in all North American Sephora stores that provides a read of each customer’s skin tone. The customer is assigned a Color IQ number, which helps them filter through thousands of lip colors, foundations, and concealer shades to choose the one that’s best for their specific coloring.

The company launched seven Beauty TIP (teach, inspire, play) Workshops across North America. At these concept stores, customers come together to learn via group beauty classes. Customers can leverage tutorials and Sephora Virtual Artist technology on integrated iPad stations, or gather inspiration from The Beauty Board, Sephora’s shoppable gallery showcasing user-generated content via a large digital screen.

“Sephora Virtual Artist is a really good example of where there was a real customer need.”

Mary Beth Laughton, executive vice president of omni retail, Sephora

And in two New York City stores, customers can try out the Innovation Lab’s newest tech venture, Tap and Try. The technology lets you pick any lip or lash product on an endcap display, and immediately try it on using Sephora Virtual Artist combined with RFID scanning.

Finally, a few stores across North America offer Sephora’s Fragrance IQ experience with first-to-market sensory technology called InstaScent. After filling out an online scent profile, InstaScent allows clients to test 18 different scents using a dry air delivery system, so they can test them out without actually trying them on.

Tech partner Inhalió was approached by Sephora to use its scent abilities for fragrances. “We went through a four-year R&D process, and decided to build an IoT-type platform that could launch scent into many different areas,” said Inhalió CEO Keith Kelsen. “The Sephora approach lets you explore the scents themselves, and decide what you like. Typically, when you’re in a store with fragrance, you are absolutely overwhelmed, and walk away with a headache. With our technology, you can sample several fragrances with no problem and no side effects.”

Sephora has benefited from top-down support from its parent company and executives, Laughton said. Further, with the Innovation Lab, “our proximity to Silicon Valley gives us access to a lot of the state’s digital players and tech startups,” Laughton said. “They’re a constant source of inspiration, and our team is definitely highly engaged with the surrounding environment.”

The approach appears to be paying off. In the first quarter of 2017, parent company LVMH’s selective retailing organic revenue, which Sephora is a part of, grew 11%.

The future of the beauty industry

The beauty industry began exploring digital technologies earlier than many other consumer product sectors, due to all of the possibilities to enhance front-end interaction with clients, said Gianni Giacomelli, senior vice president and business leader for digital solutions at Genpact. “Simple applications end up creating a really good customer experience for clients, and providing a value-added service,” Giacomelli said. “It’s not just a pretty promotional front end.” The data points that cosmetic companies can gather about their customers are also key for digital transformation, he added.

While other retailers have suffered financially in recent years due to online shopping, prestige beauty sales in the US rose 6% from February 2016 to February 2017, reaching $17 billion, according to The NPD Group. And four out of every five dollars earned in prestige beauty stemmed from makeup, the group found.

The beauty industry will likely become more and more personalized in the coming years, Aarabi said. “If you walk into a Sephora five or ten years from now, every aspect of your experience, from what you see to the products you are recommended will be customized based on your face shape, your sales history, and your preferences,” Aarabi said. “That personalization will really make life easier for shoppers. But it’s also better for beauty brands, because they can increase conversions by making sure their products are relevant to each shopper on a personalized level.”

Sephora 34th Street Store Grand Opening on March 30, 2017 in New York City.

Many companies in non-technical spheres are becoming digitally aware and proficient, said Brian Solis, an analyst at Altimeter. However, most are advancing very slowly.

“Sephora is among the most advanced and advancing retailer and beauty companies that I’ve studied over the years,” Solis said. And the actual technology is only part of this approach.

“Sephora is also diversifying its approach to digital transformation by focusing on the areas where other companies are struggling,” Solis said. “Specifically, the company has studied digital customers to understand her needs and beauty and social aspirations, how she feels and shops, and more, so they adapted models, processes, and resources internally to compete with purpose.”

The company has also recognized that its customers are not done evolving, Solis said; hence, the importance of the Innovation Lab to keep up on the latest consumer trends and technological innovations. The lab sponsors a Think Tank program, which grooms the next generation of leaders in the company and allows them to develop ideas and pitch them to the management team. It also oversees Idea Central, a program that sources ideas from employees nationwide in both stores and headquarters to get new ideas for ways to make the shopping experience better.

“We’re trying to foster a lot of innovation within the company as a whole,” Laughton said. “We definitely want people with different perspectives and backgrounds.”

Sephora has digitized its entire business strategy, Witcher said, with its in-store, in-app, and online solutions all providing value. “Sephora is still one of the few companies that has an actual dedicated part of their website where you can go in and personalize the experience for yourself,” Witcher said. “They are creating dialogues with customers, not monologues. And those dialogues—whether it be in store, on the app, or online—are what helps Sephora understand their customers better, and then deliver the kinds of experiences that not only meet but exceed customer expectations.”

Digital transformation advice

Many retailers today understand that they need to transform, but face difficulties creating a vision and a plan for doing so to improve business practices, Witcher said.

Every digital product or tool or experience that we introduce is done to make shopping for our consumer more fun and more efficient.”

Mary Beth Laughton, executive vice president of omni retail, Sephora

“The biggest takeaway from Sephora as a case study is to understand that your customers have pain points in their journey, and you need to solve those specific pain points,” Witcher said. Other non-technical companies with successful digital transformation projects have done this as well. For example, Starbucks’ payment app and order-ahead feature cut down time customers spent waiting in line.

“Focusing on a key problem and solving it well is extremely important,” Aarabi said. “If you try to do too many things, it’s hard to do everything well. So focusing on one technical challenge and really getting all the experts in and doing it well usually leads to the best outcomes.”

Whatever digital project you choose to undergo, the customer’s needs must be considered first and foremost, Solis said. “I highly recommend that companies understand how consumer behaviors, preferences, expectations, and values are changing as digital impacts their life and work,” he added. “I also recommend plugging into technology and societal trends to understand what’s possible and impactful in practice.”

Executives must also empower their employees to lead digital transformation work, and escalate this work to the C-suite, so that companies can innovate instead of just respond to market trends, Solis said.

“Above all, remain focused on your consumer and their needs,” Laughton said. “If what you’re building doesn’t make their shopping experience faster or easier or more fun, it’s probably not worth the investment.”