The case study Audiovisual and Embodied Interfaces for Multisensory Experience combines contributions from two EU-funded projects dealing with multisensory interaction and user experience: Enabling Audiovisual User Interfaces (Marie Curie Intra-European Fellowship, 2014-2016) and Moving Digits (Creative Europe, 2018-2020).
Enabling Audiovisual User Interfaces (Enabling AVUIs) investigated how human-computer interactions can be audiovisualized – that is, both sonified and visualized – in order to improve user experience and usability. To address this issue, a new UI (User Interface) paradigm was conceived – AVUI (AudioVisual User Interface). AVUI links interaction, sound and image, building upon the concept of Graphical User Interface (GUI) by adding interconnected sound and image. The project’s website is http://avuis.goldsmithsdigital.com.
Moving Digits: Augmented Dance for Engaged Audience aims to enhance audience understanding and engagement in contemporary dance performances, while allowing to experience dance in an augmented way (even after the performance). The project also aims to empower dancers, choreographers and designers with further tools for expression, archival and analysis. To achieve these aims, we use different digital techniques and artistic approaches to visualize information from dancers – specifically, physiological and movement information. This information is accessible after performances via Virtual Reality (VR) visualization – for audiences and dance artists. Project website: https://movingdigits.eu.
Grants
-
- EU Marie Curie fellowship FP7 REA grant 627922
-
Creative Europe Project no. 597398-CREA-1-2018-1-PT-CULT-COOP1
References to the Research
- Correia, N. N., Masu, R., Pham Hoang Dieu, A., & Feitsch, J., In press. Connected Layers: Evaluating Visualizations of Embodiment in Contemporary Dance Performances. Proc. TEI’21. ACM, Salzburg.
- Jürgens, S., Correia, N. N., & Masu, R., In press. The Body Beyond Movement: (Missed) Opportunities to Engage with Contemporary Dance in HCI. Proc. TEI’21. ACM, Salzburg.
- Correia, N. N., & Tanaka, A., 2017. AVUI: Designing a Toolkit for Audiovisual Interfaces. In: Proc. CHI ’17. ACM, Denver, CO. http://dl.acm.org/citation.cfm?id=3026042
- Metatla, O., Correia, N.N., Martin, F., Bryan-Kinns, N., Stockman, T., 2016. Tap the ShapeTones: Exploring the Effects of Crossmodal Congruence in an Audio-Visual Interface. In: Proc. CHI ’16. ACM, San Jose, CA. http://dl.acm.org/citation.cfm?id=2858456
Sources to Corroborate the Impact
- Dr. Atau Tanaka, Professor of Media Computing – Goldsmiths, University of London
(Host of the Enabling AVUIs project, CHI 2017 paper co-author) - Dr. Chris Geiger, Professor of Mixed Reality and Visualization – Hochschule Düsseldorf
(Partner of the Moving Digits project) - Dr. Nick Bryan-Kinns, Professor of Interaction Design – Queen Mary University of London
(Partner of the Enabling AVUIs project, CHI 2016 paper co-author) - Dr. Oussama Metatla, Senior Lecturer in HCI – University of Bristol
(Partner of the Enabling AVUIs project, CHI 2016 paper co-author) - ENABLING AVUIS Report Summary http://cordis.europa.eu/result/rcn/187646_en.html
- Dr. Atau Tanaka, Professor of Media Computing – Goldsmiths, University of London
People
- Nuno N. Correia
- Stephan Jürgens
