A view from DC: A two-sided market of AI deception

Economists often talk about two-sided markets, a concept where an intermediary facilitates two distinct groups of economic actors, such as app users and developers or consumers and advertisers. The value each group derives from the interaction increases as the groups grow, resulting in profitable network effects. 

For the platform, this dynamic also equates to a chicken-and-egg problem. It is hard to attract either group of users without a sizable market of the other type. This is true for a marketing platform selling targeted ad segments. On one side, it may find it hard to attract advertisers without sufficient consumer data to fuel its supposedly unique insights. And on the other, it is hard to afford to generate specialty data segments without the buyers to fund them, especially in the hyper-competitive ad targeting landscape.

There are many legitimate ways to solve this challenge and grow a profitable company. But, as it turns out, there are also plenty of illegitimate ways.

This week, the U.S. Federal Trade Commission announced a settlement with CMG Media Corporation, doing business as Cox Media Group, a major American mass media conglomerate. In coordination with two small marketing firms, MindSift and 1010 Digital Works, CMG marketed a service called "Active Listening," which claimed to use artificial intelligence to eavesdrop on "casual conversations" via smartphones and smart speakers to identify consumers who were actively interested in specific services. 

It should come as no surprise that these companies' marketing claims raised the eyebrows of privacy and consumer protection attorneys at the FTC. If true, the promises described in the FTC's complaints would likely have brought the firms squarely into the crosshairs of privacy enforcers. Instead, the business model was allegedly built on a web of deceptive claims, which likely invited even greater scrutiny.

Collectively, the settlement requires the three firms to pay USD930,000. 

Let's first examine the consumer data side of this market — and the privacy side of this marketing scheme. The company relied on one of the most persistent, though generally unfounded, privacy anxieties of our time: that pervasive microphones in consumers' smart devices are "always listening" for purposes of identifying targeted advertising interests. Had these claims been true, CMG would have been part of an unprecedented digital surveillance operation, almost certainly violating the privacy expectations of consumers.

The FTC complaints quote extensively from the companies' marketing materials — "every casual conversation between two consumers becomes a tool for you to target, retarget, and retain customers" — and employee training scripts — "When the device/app's microphone picks up spoken keywords/phrases, that user is qualified by the AI algorithm… and added to a list."

As for the data in question, the contents of consumers' conversations, the companies allegedly claimed that it was collected on an opt-in basis. One quote included in the complaint claimed that Google would "validate" the opt in, though according to the FTC the companies otherwise claimed opt ins were based on general acceptance of terms of service.

'Creepy? Sure. Great for marketing? Definitely.'

If the technologies had worked as described, the FTC would have likely treated the nonconsensual collection of voice data from device microphones as an unfair practice under Section 5, consistent with precedent around other types of sensitive personal information. Of course, without meaningful informed consent, listening to private conversations could also trigger violations of criminal wiretapping statutes like the Electronic Communications Privacy Act. 

And, in case it needs to be said, the FTC press release explicitly reminds us that "clicking through mandatory terms of service does not constitute 'opt-in consent' for such an invasive service or for use of consumers' voice data from inside their homes."

As it happens, the service the companies pitched to advertisers using the catchphrase, "It may seem like black magic, but it's not—it's AI," was, according to the FTC, not in fact AI. Nor did the advertising segments used for targeting actually rely on recorded conversations, with consent or otherwise.

Instead, the companies found themselves under the ire of the FTC because the service they were selling suffered from a Wizard of Oz problem. Instead of a great and powerful AI surveillance wizard behind the curtain, there was just a flesh-and-blood corporate person, allegedly buying email marketing segments then repackaging and selling them with a black magic upcharge.

Given these facts, the FTC settled the case on behalf of the small business advertisers who had allegedly been sold a bill of goods, treating them as deceived consumers under the FTC Act. In addition to the large fine, which will go toward restitution, the agency entered into a 20-year consent decree with the companies requiring ongoing compliance reporting.

As Christopher Mufarrige, director of the FTC's Bureau of Consumer Protection, said in the press release, "It is a basic rule of business that you need to be honest with your customers, and these companies failed to do that."

Counting the forms of AI deception

In the age of AI, it is worth keeping our eyes open for many flavors of possible deceptive practices. This case demonstrates one form: passing simpler processes off as AI. Not only in some cases does this carry the possibility of materially deceiving customers, but it also contributes to the current inflationary dialogue around AI. This makes the big policy questions of our day even harder to solve. If everything is AI, how do we decide which systems require additional governance mechanisms?

Claims about AI can also deceive by presenting a claim of measuring and reporting a significant signal when all that exists is noise. There are many examples of these modern forms of phrenology, or what Princeton University's Center for Information Technology Policy Director Arvind Narayanan calls AI snake oil.

Still other kinds of AI deception are also proliferating, though perhaps not yet rising to the level of consumer protection enforcement actions. There is the lie of the inherent objectivity of machines, what author Meredith Broussard calls "artificial unintelligence." And there is the possibility of claiming performance results from an AI system, even if it is designed to achieve them, that simply have not been validated.

Whether your customers are consumers, small businesses or AI agents, and no matter how many sides your market has, properly aligning marketing claims with reality is a vital obligation. The FTC and state consumer protection authorities are watching closely.

Please send feedback, updates and snake oil recipes to cobun@iapp.org.