Audio-Visual Analytics

Identifying Research Gaps for Integrating Sonification and Visualization

Conceptual process of audio-visual data analysis: data are transformed and encoded to form visual and auditory representations for the human analyst who interactively steers the idiom for their data analysis

Visualization and sonification are two approaches for conveying data to humans based on complementary high-bandwidth information processing systems. Kramer et al. defined sonification as “the use of nonspeech audio to convey information.” Tamara Munzner defines visualization as follows: “Computer-based visualization systems provide visual representations of datasets designed to help people carry out tasks more effectively.” Both visualization and sonification address the purpose of involving human analysts in data analysis. There are several similarities between the methods and design theories of both approaches, such as the use of perceptual variables to encode data attributes, and the role of interaction in manipulating the data representations.

Some illustrative examples:

Over the recent decades, both fields have established research communities, theoretical frameworks, and toolkit support. Although extensive research has been carried out both on the auditory andvisual representation of data, comparatively little is known about their systematic and complementary combination for data analysis. One example of multimodal research is Keith Nesbitt’s dissertation. Also, Walker and Kramer pointed out that research on the design and the use of multimodal sonification is important to drive sonification and visualization research forward. There are potential powerful synergies in combining both modalities to address the individual limitations of each one. Nevertheless, existing research on combinations has often focused only on one of the modalities.

Actual multimodal approaches should be based on complementary and mutually supportive interplays between data representations on the visual and the auditory domain. This workshop series aims to build a community of researchers from both fields, to work towards a common language, and to find research gaps for a combined visualization and sonification theory.

Upcoming workshops:

Latest Posts

IEEE VIS Workshop next week

Next Monday, our 2nd Workshop on Audio-Visual Analytics will be hosted at IEEE VIS 2021, the premier forum for advances in visualization and visual analytics.

AudioMostly Workshop accepted

Yesterday, the AudioMostly accepted our workshop proposal. There will be two workshops on Audio-Visual Analytics in autumn 2021, which exposes the topic to both the sonification and the visualization communities.

IEEE Workshop accepted

Great news: our workshop proposal for IEEE VIS 2021 was accepted.