Creating Immersive Shopping Experiences using Image/Facial Recognition

Creating Immersive Shopping Experiences using Image/Facial Recognition

Customer experience is a phrase that has taken on several meanings. As customers aspire for better products, services and a great experience at the convergence point of convenience, best price points and functionality, retailers conspire to bring in as many platforms and technologies as possible to enable that experience. This translates into a simple mantra that is gospel to all retailers: engage customers and they will keep coming back for more! Several new age ideas such as provisioning omni-channel platforms[1], solutions for enabling better engagement in store, smart fitting rooms, 3D fitting options, easy and smart pay options by leveraging AI and IoT, developing insights into consumer behavior and more, all of which rests on the marriage of a futuristic vision and emerging tech[2], are making their place as offline and online retail begin merging to provide top notch and innovative experiences.

One fact however, remains – consumers, despite their leaning towards the conveniences of e-retail, still love the immersive experience provided in a physical store[3]. So, what if technology can recognize every single customer as he or she walks in or logs in and offers the same level of personal interaction that a brick and mortar store can? Facial or image recognition offers just such an ability. Retailers can get an image search on their Web and mobile stores, and power consumers to search for products in fashion and home decor segments and more by simply selecting or snapping a photo[4]. While Google Images already allows users to search the WWW using image content and eBay already uses it for its image search feature, using image recognition to identify products and shop for them is a whole new step forward.

When we speak of sophisticated image recognition software, the first store that comes top of the mind is undoubtedly Amazon Go. With image recognition software identifying and greeting customers individually as they enter to identifying items picked up, using beacons to bill them in a cashless environment, down to providing an e-receipt as customers walk out, these stores are almost like a glimpse into what the future holds[5].

As far as examples and use cases go, it is not just convenience where such tech is being leveraged; fashion brands and luxury shopping stores are also striving to create hyper-personalized experiences using image/facial recognition. Ruti, a fashion brand, is one such strong statement. Using AI and customized CRM, customers are provided with personalized recommendations based on their preferences, past shopping history, and a lot more information gathered from an in-depth profiling[6]., a start-up provides virtual 3D fitting services for the footwear, where a mobile scanner takes images of the feet to register the profile. The scanning technology creates a 3D model of footwear and an algorithm searches from 10,000’s of profiles to find the footwear of choice based on parameters such as design, size, style and more. Whodat Tech, another start-up enables an AR-based solution for customers to extend the image of a home décor item into the intended environment to get a look of the item in its intended setting[7].

DEXTR or the Digital EXperience Transforming Retail, which is a solution provided by Capgemini for beauty retail segment, combines AI and robotics to recognize and authenticate customers using facial and voice recognition and offers relevant products for consideration[8]. Based on its understanding of the specific customer, it offers relevant products that are brought using its automated robotic arm. HP and Intel have collaborated with Capgemini to enable this level of personalization[9].

How does all this actually work? Let us take the example of a company, C2RO, considered an innovator in the field of machine vision AI. They use real-time cloud AI to scan and match a customer’s face to the cloud database as they walk in into the store. Once authenticated, C2RO’s system alerts the brand’s CRM or retail ERP system in whose environment the customer is in, about the presence of the customer. The CRM system starts sending out personalized offers, recommendations, coupons, and a lot more to engage the customer[10].

So, what’s next? Perhaps some brand, in the distant future, may look at facial recognition to analyze the moods of the shoppers? Well, actually, that is not something we have to imagine; its already in the pipeline, with Walmart’s research project on how to analyze the different moods of the shoppers and identify dissatisfied shoppers[11].