Affectiva’s emotion AI humanizes how people and technology interact
When machines understand human emotions …
Affectiva is one of the first Emotion Artificial Intelligence companies which aims to add emotional intelligence to any interactive product. Their main goal is to gain insights and analytics in consumer emotional engagement in different industries including online education, gaming, robotics, media and advertising, market research, automotive, etc {1,2,3,4}. The company has developed an emotion recognition technology that senses and analyzes facial expressions. The service is very easy to use, just by sending videos or images you get back highly accurate data without any setup or integration. Affectiva has analyzed so far 4.25 million videos from people in 75 countries with over 50 million data points which makes it the biggest emotion data repository in the world.
The most important part of the patented software is computer vision and deep learning, a specialized branch of machine learning. The product development depends entirely on how well the software recognizes emotions, which on the other side depend on how good the machine learning statistical analysis categorizes the facial expressions based on the existing database. What makes it more challenging is that different geographies might have different expressions. For example, there is a “polite smile” in India or southeast Asian countries but not in America, Africa and Europe {5}.
Affectiva is predicted to take the emotion AI into every corner of the world. In the future, products like Siri or google maps are expected to be empathetic with the user’s mood. Nowadays the Affectiva technology is used by 32 Fortune 100 companies. Further grow is dependent mainly on new deep learning developments that also include speech {6} and gestures on the top of the facial expressions.
The organization has invested large resources to tackle their main technical challenges in deep learning. Deep learning allows Affectiva to model more complex tasks with higher accuracy that other machine learning techniques. Such tasks include face detection and tracking, speaker voice activity detection and emotion classification. The suites that the company is developing are based on Convolutional Neural Networks and Recurrent Neural Networks {7}. The focus is in developing custom layers to provide accurate real time estimates on mobile devices. The company’s approach includes 3 steps: multitask learning, iterative benchmarking and model compression. Affectiva has continued to bolster its patent portfolio with 11 patents and 40 pending approvals {8}.
In the long term the company is not looking just at the technical obstacles but also social goals. Affectiva joined a consortium started from the big tech companies such as Google, Amazon, Microsoft and Facebook to ensure that artificial intelligence is used in a way to benefit society. There is not goal to develop lie detection features, the company requires explicit consent to use their software on users, people should be aware of when they want their emotions to be recognized. But the emotions world is so personal and there are so many questions that are not solved yet. Biases are very dangerous when training machines, and the way to avoid them is not straight forward. Avoiding replication of social injustices from the machines is a moral and business responsibility.
My short term suggestions to the company involve the cost aspect of deep learning. Affectiva has raised about $34 million {9} so far and the main focus is in developing useful on the fly technology that recognizes emotions. I would rephrase their goal by adding a “cost effective” adjective to the smart technologies. The need for repair and ongoing maintenance for machine learning programs is becoming more and more prevalent. Affectiva will benefit a lot if they start thinking now in a less academic oriented aspect and more how to make their products cost effective.
In terms of their long term challenges, I would recommend them to think and act about Artificial Intelligence ethical concerns on a broader spectrum, not only on the emotions side. Affectiva needs to pay attention to the effect of machine interaction on human behavior in any of the use cases their product is used.
There are two questions that I find very interesting to discuss on the machine learning space and would love to discuss them. One of them is the security of the AI systems that can potentially cause damage. How to handle it and make sure that there is no intrusion. The second questions in mitigation against unintended consequences since smart machines are thought to learn and develop independently.
{1} Bosker, Bianca “Affectiva’s Emotion Recognition Tech: When Machines Know What You’re Feeling”. The Huffington Post (December 24, 2012).
{2} Kerstetter, Jim “Building better Super Bowl ads by watching you watch them”. CNET. (February 2, 2013).
{3} Miller, Ron. “New Affectiva cloud API helps machines understand emotions in human speech”. TechCrunch. (October 26, 2016)
{4} Omar Jenblat Let’s Get Emotional: The Future Of Online Marketing Forbes (February 26, 2016)
{5} Rana el Kaliouby Does Facial Coding Generalize Across Cultures? A Spotlight on Asia (February 22, 2014)
{6} Scott Wisdom, Deep Recurrent NMF for Speech Separation by Unfolding Iterative Thresholding IEEE (September 21, 2017)
{7} Lynch, Zack “Let the NeuroGames Begin”. VentureBeat. (January 17, 2013).
{8} Bender Using Affect Within A Gaming Context – Patent US 9,247,903 B2 (February 2, 2016)
{9} Affectiva funding Crunch base (September 10,2018)
In terms of your first question regarding security, I would imagine that the particular use of facial recognition in this case is means for concern. In order for the machine to assess emotions, one would need to feed it millions of images and changing facial expressions of each sample. This may require a video recording either from a laptop or cell phone. With the current cyber security debate going on, I would imagine that Affectiva will face major concerns in their collection of image-based data and the need for video-capabilities.