After words began life as a response to the datasets we kept stumbling across in our research. In the Toronto Emotional Speech Set, for instance, researchers asked actors to read the carrier phrase, ‘say the word…’ followed by two hundred different ‘target words’, in seven different ‘emotions’. The idea was that the resulting 2800 entries would then be used to teach machine learning systems how to recognise emotions in speech, with various potential commercial applications: all involving unprecedented forms of monitoring and data extraction, of course. But the recordings just sound so weird, even to the point of being uncanny or unsettling. And we were really struck by the theatricality of it all: the fact that these actors were performing for a specifically algorithmic audience.

In another example, researchers in France had their colleagues simulate eleven categories of office sounds—keyboards tapping, papers shuffling, doors shutting, phones ringing etc—in what is surely one of the most boring performances in history. This dataset was then used in a ‘sound event detection’ challenge that has been running annually since 2013. Some of these audio event classes have already been incorporated across various Google products.

It turns out there’s a long history of computational theatre like this. The very first ‘artificial intelligence’ was performed by actors in Pittsburgh before it was ever ‘run’ by a computer. And of course, we also perform for our devices, every time we ‘wake up’ Siri or Alexa. Engaging with voice assistants also involves a kind of scripting, into which we are relentlessly entrained. We need to be taught how to make ourselves understood by machines.

After words is about these performances. It’s about how we speak to and perform for machine listening systems, and about what is happening to language as a result. It's about what happens to our linguistic performances, about their afterlives, as datasets, as commands, or prompts, and about the near future in which language has been fully operationalised: where every word we speak has a computational audience, effect and residue.

Most of the sounds and many of the voices in After Words come from the kinds of datasets mentioned above. But the script was also written with and against an ‘autoregressive language model’ trained on a dataset with 175 billion parameters. And it was performed by voice actors all doing amazing work of their own in critical data studies. That was a key thing for us. The recording sessions felt like a kind of mutual education: thinking together, with and about the work. The result is a strange set of semi-fictional tales of computational scripting, instruction, production, and performance, staged at ACCA in 8-channel audio. Our hope is that the piece contributes in some small way to people’s understanding and experience of machine listening, as a system or set of systems with which we are now deeply enmeshed.