Facebook patents system that can use your phone's mic to monitor TV habits

The patent application describes a system where an audio fingerprint embedded in TV shows or ads, inaudible to human ears, would trigger the phone, tablet or long-rumoured smart speaker to turn on the microphone and start recording “ambient audio of the content item”. The recording could then be matched to a database of content to allow Facebook to identify what the individual was watching – like Shazam for TV, but without the individual choosing to activate the system.

A diagram in the patent application. Photograph: US Patent Office

Diagrams accompanying the patent highlight how the technology would know which adult or child within a household was watching a particular broadcast.

The patent, first spotted by the New York Times, positions the technology as a way for broadcasters to know exactly who is watching their TV shows or ads and for how long. The same system could then be used to build viewing profiles of individual members of a household for better content recommendation and more targeted advertising.

Privacy experts are concerned about the intrusion into people’s homes, particularly as the ambient audio recording would likely catch snippets of people’s private conversations without their knowledge.

“It’s extremely disconcerting for privacy to have an inaudible beacon as it means they want to make it not obvious to the user that the device is listening,” said William Budington, a senior staff technologist at the Electronic Frontier Foundation.

Such a system could also give Facebook a better understanding of people’s social connections as it would show the social network which people were meeting up in real life.

Facebook was quick to downplay the patent filing.

“It is common practice to file patents to prevent aggression from other companies. Because of this, patents tend to focus on future-looking technology that is often speculative in nature and could be commercialised by other companies,” said Facebook’s head of intellectual property, Allen Lo, in a statement.

“The technology in this patent has not been included in any of our products, and never will be.”

Facebook isn’t the first company to design a system that uses secretly broadcast audio signals to track people’s viewing habits. In 2015, a company called SilverPush developed ultrasonic audio “beacons” within TV commercials that could be identified by any device running apps that incorporated SilverPush’s software.

The system allows for far more accurate tracking of people’s viewing habits at an individual level.