AI Ice Cream Bar

The AI tool that captures a user’s facial expression and recommends the ice-cream flavour based on their reactions, derived from computer vision & facial recognition. This tool was showcased at The MedTech Conference, Boston.

Team

2 designers, 1 product manager,
5 engineers, 4 marketers

My Role

Interaction Designer

Responsibilities

Interaction design, Visual design, User testing, Developer support, Design for AI

Year

2019

The TEam

2 designers, 1 product manager,
5 engineers, 4 marketers

Responsibilities

Interaction design, visual design, user testing,
developer support

TimelinE 

August 2019 - December 2019

Project Overview

Designing experience of an AI prediction tool

The system is built on AI that derived attributes from computer vision & facial recognition technology that suggests the flavour preferences through attributes. The solution takes advantage of a camera and infrared sensor to initiate the process. The algorithm itself learns and adapts the more people use it.

The challenge was to combine multiple project touch points (marketing research, preliminary designed tool, and understanding of ai data model) into one, easy-to-use product. It was crucial to make sure the final interface and experience adapted well within the given timeframe and capture data for further training and deliverables.

Expected business outcomes

Eyeforpharma organizes one of the biggest pharmacy and life sciences event annually. The marketing team wanted to engage more users at these events in order to showcase the analytical and predictive capabilities of our products.

The Process

Gaining insight & empathy


Initial research was conducted on similar ai tools to gain empathy for users and understand what type of structure, layout, and functionality is typical of this product type. We wanted to identify what worked and what didn’t in order to design a better experience.


Understanding emotion recognition

The face API integrates emotion recognition, returning the confidence across a set of emotions for each face in the image such as anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. These emotions are understood to be cross-culturally and universally communicated with particular facial expressions.

Mapping emotions with ice cream

4 major human emotions were identified to depict the user's predicted mood and mapped with certain flavors of ice-cream to recommend to the user. These emotions were visually represented as a custom character emoji on the tool.

Final Design

Designing for efficiency and easy collaboration

The overall experience of the users interacting with the interface can be broadly represented within three stages. The process from start to finish is under 30 seconds to ensure a seamless user experience. The final designed screens for the showcase are as follows.

Conclusion

Learnings and outcome

It was my first project to create experience for an AI device where I collaborated with multiple teams during its developement. I conducted multiple tests with the engineers & data science teams to deliver this tool for showcase.

The tool was displayed at many med-tech conferences around the world to market the brand’s ai-cx potential and it was successful in engaging with a potential audience.