Inclusive Multimodal Voice Interaction for Code Navigation

Paudyal, Bharat and Creed, Chris and Williams, Ian and Frutos-Pascual, Maite (2022) Inclusive Multimodal Voice Interaction for Code Navigation. In: ICMI22 - 24th ACM International Conference on Multimodal Interaction, 7th - 11th November 2022, Bengaluru, India.

Paudyal_ICMI_2022.pdf - Accepted Version

Download (712kB)


Navigation of source code typically requires extensive use of a traditional mouse and keyboard which can present significant barriers for developers with physical impairments. We present research exploring how commonly used code navigation approaches (e.g. locating references to user-defined identifiers, jumping to function definitions, conducting a search for specific syntax, etc.) can be optimized for multimodal voice interaction. An exploratory study was initially conducted with five developers who have physical impairments to elicit insights around their experiences in navigating code within existing voice-controlled development environments. Findings from this study informed the design of a code editor integrating different navigation features tailored for multimodal speech input. A user evaluation with 14 developers with physical impairments was conducted with results demonstrating that all participants were able to successfully complete a series of standard navigation tasks. Participants also highlighted that the code navigation techniques were intuitive to use and provided a high-level of usability.

Item Type: Conference or Workshop Item (Paper)
Identification Number:
10 August 2022Accepted
7 November 2022Published Online
Uncontrolled Keywords: Assistive technology, Programming tools, Speech recognition, accessibility
Subjects: CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
Divisions: Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology
Depositing User: Chris Creed
Date Deposited: 27 Sep 2022 15:38
Last Modified: 22 Mar 2023 12:00

Actions (login required)

View Item View Item


In this section...