Spearcons as X-axis markers in Sonifications

From early sonification development with lead-PI Amy Bower and our co-design session at Perkins School for the Blind, we discovered that sounds for location and context are essential. I started including spearcons, sped-up speech as auditory icons, to indicate x-axis ticks in the data as part of sonification prototypes.* Beyond contextual cues, there is some evidence to suggest spearcon learnability over basic earcons; earcons are more abstract, non-speech sounds (Dingler, Lindsay, and Walker 2008). My students in MUS 479 Data Sonification Winter 2022 term also engaged in spearcon development as part of their class projects with much success. I anticipate creating x-axis, and perhaps y-axis tick mark sounds as a separate audio track of our data sonifications to better contextualize the sonified data. While we may not be able to develop an interactive tool to allow users to mute and unmute x and y-axis tick sounds due to the limitations of the pilot study, we recognize the importance of auditory icons (spearcons or others) for contextualizing data. With so much focus on sonifying the data files, it can be easy to overlook that the axes of the graph—that many visual readers rely upon for contextualizing information— are hidden from blind and visually-impaired users.

Two examples of x-axis spearcon markers are below. The first example is a sonification without any axis markers. The second example is the same sonification and includes x-axis spearcon markers.

For those interested in similar work, I also created a public repository on GitHub of speech synthesis voice recordings of common markers found in axes of graphs. These voice recordings are for (re)creating axes in data sonifications. If you use recordings, please give this project a shout-out in credits.

Footnote*: Speech tick-marks at 1.5x speed could be considered a spearcon-speech hybrid as there is no clearly defined speed that makes a spearcon, simply that spearcons are "no longer recognizable as speech" (Walker & Nees 2011).