Projects

Machine Understanding for interactive StorytElling

The MUSE project will introduce a new way of exploring and understanding information by “bringing text to life” through 3D interactive storytelling.

Machine Understanding for interactive StorytElling

MUSE is a FET-open call project funded by the European Commission under FP7, which started in September 2012 and running until August 2015.

The MUSE project will introduce a new way of exploring and understanding information by “bringing text to life” through 3D interactive storytelling. Taking as input natural language text like children’s stories or medical patient education materials, MUSE will process the natural language, translate it into formal knowledge that represents the actions, actors, plots and surrounding world, and then render these as virtual 3D worlds in which the user can explore the text through interaction, re-enactment and guided game play.

To enable such a system, MUSE will make targeted advances in natural language processing that enable the translation of natural language text to the necessary knowledge representations, as well as targeted advances in the action representation and story planning necessary for interactive storytelling. In natural language processing, MUSE will develop new techniques for finding explicit action structures in text and combining them with implicit action structures inferred from the context based on probabilistic models of translation and automatic methods for acquiring world knowledge from large corpora.

In interactive storytelling, MUSE will develop action and object representations that bridge the gap between natural language and virtual worlds, and will create advanced techniques for planning virtual world stories given inconsistent and incomplete information. The proposed methodology will be evaluated and showcased on two scenarios: one for creating immersive children’s stories from text and one for allowing medical patients to interact with patient education materials.

Teesside University (Intelligent Virtual Environments Lab) is directly involved in the definition of knowledge representation formalisms underpinning the behaviour of virtual agents (characters and objects) in the virtual world which can be instantiated from natural language semantics and supports both behaviour and inference. We also bring to the development of the MUSE project our long-term expertise in creating several compelling interactive storytelling systems over the last decade.

This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 296703.

More details about the Machine Understanding for interactive StorytElling project