Talking Barbies are ushering in a new era of mass surveillance

– Talking Barbies are ushering in a new era of mass surveillance (Business Insider, June 20, 2015):

One of my great pleasures in life is attending conferences on fields I’m intrigued by, but know nothing about. (A second pleasure is writing about these events.) So when my friend Kate Crawford invited me to a daylong “Listening Machine Summit,” I could hardly refuse.

What’s a listening machine? The example of everyone’s lips was Hello Barbie, a version of the impossibly proportioned doll that will listen to your child speak and respond in kind. Here’s how The Washington Post described the doll back in March: “At a recent New York toy fair, a Mattel representative introduced the newest version of Barbie by saying: ‘Welcome to New York, Barbie.’ The doll, named Hello Barbie, responded: ‘I love New York! Don’t you? Tell me, what’s your favorite part about the city? The food, fashion, or the sights?’

Barbie accomplishes this magic by recording your child’s question, uploading it to a speech recognition server, identifying a recognizable keyword (“New York”) and offering an appropriate synthesized response. The company behind Barbie’s newfound voice, ToyTalk, uses your child’s utterance to help tune their speech recognition, likely storing the voice file for future use.

And that’s the trick with listening systems. If you can imagine reasons why you might not want Mattel maintaining a record of things your child says while talking to his or her doll, you should be able to imagine the possible harms that could come from use—abuse or interrogation of other listening systems. (“Siri, this is the police. Give us the last hundred searches Mr. Zuckerman asked you to conduct on Google. Has he ever searched for bomb-making instructions?”)

As one of the speakers put it (we’re under Chatham House rules, so I can’t tell you who), listening machines trigger all three aspects of the surveillance holy trinity:

1. They’re pervasive, starting to appear in all aspects of our lives.
2. They’re persistent, capable of keeping records of what we’ve said indefinitely.
3. They process the data they collect, seeking to understand what people are saying and acting on what they’re able to understand.

To reduce the creepy nature of their surveillant behavior, listening systems are often embedded in devices designed to be charming, cute, and delightful: toys, robots, and smooth-voiced personal assistants. Proponents of listening systems see them as a major way technology integrates itself more deeply into our lives, making it routine for computers to become our helpers, playmates, and confidants.

A video of a robot designed to be a shared household companion sparked a great deal of debate, both about whether we would want to interact with a robot in the ways proposed by the product’s designers, and how a sufficiently powerful companion robot should behave.

….