Auditing the information quality of voice assistants

Authors

Mykola Makhortykh, Ernesto de Léon, Aleksandra Urman

Abstract

Voice assistants (VAs) are AI-powered software products which are trained to recognize user speech and react to user requests. VAs are used for a broad range of purposes, including retrieving information about general concepts and providing news updates. Such diverse functionality makes VAs highly popular around the world (3.25 billion users worldwide for 2019; Statista) as well as in Switzerland (37% of Swiss population as of 2018; Swissinfo). However, there are pending questions about the quality of information provided by VAs both in terms defining general concepts (e.g., what is the Holocaust) and answering general questions (e.g., whom shall I vote for?) as well as news sources selected for mainstream (e.g., what is COVID-19 situation in Switzerland) and more niche requests (e.g., how many crimes did migrants commit in Bern this week?). To address these questions, we aim to audit performance of three most popular voice assistants: Alexa, Siri, and Google Assistant.

Duration

2021 – 2022

Keywords

algorithm audit, voice assistants, conspiracy theories, Holocaust, information quality, artificial intelligence