Image © Tina Gonsalves
Exhibition: Chameleon Prototype 7 by Tina Gonsalves, at Lighthouse, Brighton, until March 28 2009
A man and a woman are arguing. The man, his face a fierce grimace, has raised his voice and is swearing as he aggressively bobs his bald head forwards and back. The woman is different. She looks sad and barely moves. "I am really angry at you," she says, quietly. "I know it doesn't look like I am, and it never really looks like I am, but I am really very angry at you."
It is an uncomfortable scene to watch, but thankfully not one taking place between fellow visitors to Tina Gonsalves' residency at the Lighthouse space.
The heated exchange comes from Prototype 3 - two screens showing individually recorded actors in the grip of various emotions, the different clips matched by a complex computer algorithm that aims to recreate the way emotions are transferred between people.
It is a typically technically and audacious part of the ongoing Chameleon Project, which combines art and science in an intriguing investigation into emotions, empathy and non-verbal communication. Six prototypes have been completed so far and Gonsalves and her scientific collaborators are currently working on number seven.
Gonsalves has been exploring this area since 1994, at first making short films that aimed to translate internal emotional states into an external medium. Finding the basic film format increasingly inadequate, she moved on to installations and the use of more innovative technology.
This included the electronic jewellery of the Medulla Intimata project, which used sensors to read the undercurrents in a conversation – not what people are saying but how they are saying it – and then display an image of the wearer expressing their true feelings.
Picture © Tina Gonsalves
"It was speaking what was unspoken," remembers Gonsalves. "It would let people know when you were bored. You would try and act interested but the picture would be yawning. It made you feel vulnerable, but it also changed social dynamics – all of a sudden people would try and be more interesting as they could see you were bored, and would reveal more of themselves in the process."
A desire to find a more precise criteria for different emotions lead Gonsalves to the world of neuroscience, where scientists and psychologists study the physical processes of the human nervous system in incredible detail.
Now she is halfway through the two-year Chameleon project, in collaboration with social neuroscientist Prof Chris Frith, emotion neuroscientist Prof Hugo Critchley, and computer scientists Dr Rosalind Picard and Dr Rana Kaliouby.
Their recent work falls into three stages: creating a face reading technology, coding a video engine to mimic emotional "contagion", and then evaluating the accuracy of this code.
"A lot of empathy is mimicry – we’ll start dropping the same language, we’ll start using the same tonal range, we’ll start making the same expressions," explains Gonsalves. "But there's also an agenda. If you're angry I'll most likely defuse your anger – I'll be sad. And if you don't understand my sadness and are still angry, then I'll get angry because you haven't understood that I am trying to defuse it."
Prototype 3 emulates this process, and Prototype 7 takes things further by introducing an element of interactivity.
A participant enters the room to find a screen showing a neutral human face. A video camera monitors the expression of the participant, and a computer establishes what emotion they are currently feeling from a set of six - disgust, happiness, anger, neutrality, sadness and surprise.
Picture © Tina Gonsalves
This information is used to determine what emotion the person on the screen will display. As they emote, the viewer's expression – and so their mind - continues to be read and the process continues, all under the guidance of the computer’s emotional code. A strangely affecting dialogue is formed.
Participants have been venturing deep into Lighthouse to help Gonsalves with the development of the prototype. Their reactions are monitored and they have a chance to give their opinion on whether they feel the piece is successful or not.
Technical problems are to be expected on such an ambitious set of projects, and the Chameleon team have had their fair share.
Prototype 4, which trawls the net looking for blog posts from people displaying a certain emotion, had to be altered. Rather than displaying people's faces, it now resembles an ongoing script, with a line from one person following a line from another. Surprisingly this "conversation" often seems to make sense.
Soon Gonsalves is heading to France to work with art group Experientiae Electricae on Prototype 8. This will break out of the screen displays previously used, and instead work with a "flexible modular display" made up of large white squares, which will act like individual pixels on a computer screen. Then work commences on the final prototype, which, if the earlier ones are anything to go by, is going to be pretty special indeed.