A wearable Al system that is capable of predicting the mood of a conversation perfectly is being developed by MIT. Interpreting the way a person communicates a sentence’s mood and tone can considerably vary the meaning of a conversation. In the end, the listener is left with the responsibility of cracking its meaning. It is the most important part of the conversation to tell apart the emotions a person is revealing. But, everyone is not capable of distinguishing the tones.
A conversation might be interpreted in a different way than it was intended by some people, especially those who suffer from anxiety or Aspergers. This miscommunication can make social interactions really stressful.
According to the researchers from MIT’s ‘Computer Science and Artificial Intelligence Laboratory (CSAIL)’ and ‘Institute of Medical Engineering and Science (IMES)’, they may have the solution: a wearable AI device capable of differentiating if a conversation is happy, sad, or neutral by actively monitoring the way a person speaks.
Tuka Alhanai, a graduate student described:
“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious. Our work is a step in this direction, suggesting that we may not be that far away from a world where people can have an AI social coach right in their pocket.”
A person’s speech patterns and physiological signals are actively evaluated by the mood-predicting wearables in order to ascertain the tones and moods articulated in a conversation with ‘83 percent accuracy’. The system has been programmed in such a way so as to record a “sentiment score” every five seconds during a conversation.
As per Ghassemi, ‘As far as we know, this is the first experiment that collects both physical data and speech data in a passive but robust way, even while subjects are having natural, unstructured interactions. Our results show that it’s possible to classify the emotional tone of conversations in real-time.’
As more people will use the system, more data will be generated for the algorithms to examine and hence the performance of the system will continue to improve via deep-learning techniques. The privacy of the user is protected by processing the data locally on a device to prevent potential privacy breaches. Nevertheless, as the device can potentially record the conversations of unassuming individuals, there may still be privacy concerns.
A participant was required to artificially portray a specific emotion for the previous studies analyzing the emotion of a conversation. MIT researchers had participants tell a happy or sad story in order to create more organic emotions.
A Samsung Simband, a device capable of capturing high-resolution physiological waveforms to measure many features such as heart rate, blood pressure, blood flow, and skin temperature, was worn by the participants of the study. In addition to this, this device also records audio data at the same time which is then analyzed to ascertain tone, pitch, energy, and vocabulary.
Björn Schuller, professor and chair of Complex and Intelligent Systems at the University of Passau in Germany, stated:
“The team’s usage of consumer market devices for collecting physiological data and speech data shows how close we are to having such tools in everyday devices. Technology could soon feel much more emotionally intelligent, or even ‘emotional’ itself.”
31 conversations were recorded by the MIT researchers which were then used to train two separate algorithms. The first one comprehends the conversation to categorize it as either happy or sad. The secondary algorithm finds out whether the conversation is positive, negative, or neutral over 5-second intervals.
Alhanai says, ‘The wearable system picks up on how, for example, the sentiment in the text transcription was more abstract than the raw accelerometer data. It’s quite remarkable that a machine could approximate how we humans perceive these interactions, without significant input from us as researchers.’
Unexpectedly, the algorithms successfully ascertained most of the emotions which a human would expect during a conversation, even though the results of the model were only ‘18 percent above chance’. The new method is ‘7.5 percent more accurate’ when compared with existing approaches, regardless of the small percentage.
Regrettably, the wearable gadget’s model is not yet developed enough to be of any practical use as a social coach. But, researchers intend to scale up the data collection by enabling the system to be used on commercial devices like the apple watch. As Alhanai said, ‘Our next step is to improve the algorithm’s emotional granularity so that it is more accurate at calling out boring, tense, and excited moments, rather than just labeling interactions as ‘positive’ or ‘negative. Developing technology that can take the pulse of human emotions has the potential to dramatically improve how we communicate with each other.’