CALLAS project
Conveying Affectiveness in Leading-edge Living Adaptive Systems (CALLAS)
CALLAS is an integrated project funded by the European Commission under FP6, running until April 2010. It brings together 18 partners from eight countries, including academic institutions and commercial entities, to pursue developments in the design and implementation of multimodal interfaces for digital arts and entertainment, focusing on the field of affect and emotion.
Headed by Professor Marc Cavazza, researchers in the Digital Futures Institute (DFI) have developed innovative methods for affective multimodal fusion and overseen the creation and evaluation of example artistic showcases, which utilise the project’s technology.
CALLAS aims to facilitate the conveying and perception of emotions within settings of pervasive interaction with digital media applications, delivering a framework for developing arts and entertainment installations and supporting research in the relationships which affect user experience. The emotional experience of a spectator is not just a passive reaction to media but is incorporated directly into an enhanced performance as a vital source of communication.
CALLAS partners contribute a wide variety of hardware and software components, which interpret and process emotional aspects of interaction that can be combined to construct emotionally aware artistic applications. These include speech and voice analysis, gesture and facial expression detection, and the display of virtual emotions through avatar expressions and music. Applications are built using a common model and representation of human affective behaviour, together with tools and methods for integrating the numerous modalities of input and output.
The project is also developing a number of scenarios in digital art, theatre and home entertainment, as representative showcases of research and technology, and as a setting for experimental evaluation of underlying theory and methods of affect and experience. Attention is given to many aspects of interface and interaction that are unique for the digital arts, dealing with the complexity of multiple modalities. This approach ensures that creative industries and artists are free to develop these new forms of interactive art, while managing the technology burden and preserving the naturalness of user interaction and the spontaneity of experience.
The emotional models and affective integration methods developed at Teesside have become the foundation of affective processing within CALLAS. They build on psychological theory and existing notions of multimodal integration, and are designed to flexibly integrate CALLAS components in a range of interactive scenarios with multiple participants, continuously integrating emotional inputs and outputs within a showcase.
CALLAS Showcase E-Tree
The CALLAS Showcase E-Tree was developed by DFI researchers led by Dr Stephen Gilroy, in conjunction with renowned digital artist Maurice Benayoun. This is an augmented reality art installation, in which a virtual ‘living’ tree is animated to display growth and movement that is influenced by the perceived emotional response of spectators. Augmented reality technology bridges the gap between virtual and physical environments, allowing direct interaction through a combined display of surroundings captured by real-time video and overlaid 3D graphics.
E-Tree exhibits sophisticated growth patterns in terms of branching, speed of growth, and the movement of branch orientation. Furthermore, growth cycles are non-monotonic and parts of the tree can fade and re-grow in response to the changing emotional reactions of spectators. This creates the basis for a rich interactive experience in which the installation induces an emotional ‘feedback loop’ between tree behaviour and spectator interaction. In this context, emotions are the content of a metaphorical dialogue between the installation and the spectators, which provide substance for the interaction.
E-Tree has been the subject of a number of user studies carried out at Teesside investigating the relationship between the measurement and representation of emotional affect and the user experience of a digital artistic installation. Volunteers are observed interacting with the installation in pairs while conversations, emotional reaction and physiological signals are recorded and analysed. An example study, involving 20 participants aged 19-45, established a link between the constructed emotional representation and both physiological and psychological measures of emotional response and disposition. A further study suggested a compatibility with extended theories of user experience in entertainment. Further studies are planned to explore these links, with additional physiological measurement facilities and the development of additional showcases exploring alternative artistic scenarios.
E-Tree has been successfully demonstrated in the field at conferences and symposia, most notably at ICT 2008 in Lyon, an event bringing together 4,500 European researchers and policy makers. E-Tree was also featured in Euronews coverage of the event, which reaches an audience of 248 million households in 135 countries.
