[The alarming trend toward a world in which we can’t trust any of the mediated messages we encounter is illustrated vividly in this story from The Washington Post about scammers using easily-created AI voices of people’s loved ones to trick them out of their money (see the original story for a second picture). A December 2021 ISPR Presence News post described isolated cases in which business executives were targeted by voice scammers and an August 2022 post
highlighted how scammers are using deepfakes to get hired by businesses for “money, espionage, access to company systems and unearned career advancement.” But the thousands of victims in the Post story are individual members of the public, particularly older people. Coverage by Ars Technica
includes this:
“The courts have not yet decided when or if companies will be held liable for harms caused by deepfake voice technology—or any of the other increasingly popular AI technology, like ChatGPT—where defamation and misinformation risks seem to be rising. There may be increasing pressure on courts and regulators to get AI in check, though, as many companies seem to be releasing AI products without fully knowing the risks involved. … In 2021, the FTC released AI guidance
, telling companies that products should ‘do more good than harm’ and that companies should be prepared to hold themselves accountable for risks of using products. More recently, the FTC last month
told companies, ‘You need to know about the reasonably foreseeable risks and impact of your AI product before putting it on the market.’”
For a vivid demonstration of how convincing deepfake voice technology can be, see a 7:34 minute segment from CNN This Morning on March 9, 2023 (or just the taped report it contains). –Matthew]
[Image: Credit: Illustration by Elena Lacey/The Washington Post]
They thought loved ones were calling for help. It was an AI scam.
Scammers are using artificial intelligence to sound more like family members in distress. People are falling for it and losing thousands of dollars.
By Pranshu Verma
March 5, 2023
The man calling Ruth Card sounded just like her grandson Brandon. So when he said he was in jail, with no wallet or cellphone, and needed cash for bail, Card scrambled to do whatever she could to help.
“It was definitely this feeling of … fear,” she said. “That we’ve got to help him right now.”
Card, 73, and her husband, Greg Grace, 75, dashed to their bank in Regina, Saskatchewan, and withdrew 3,000 Canadian dollars ($2,207 in U.S. currency), the daily maximum. They hurried to a second branch for more money. But a bank manager pulled them into his office: Another patron had gotten a similar call and learned the eerily accurate voice had been faked, Card recalled the banker saying. The man on the phone probably wasn’t their grandson.
That’s when they realized they’d been duped.
“We were sucked in,” Card said in an interview with The Washington Post. “We were convinced that we were talking to Brandon.”
As impersonation scams in the United States rise, Card’s ordeal is indicative of a troubling trend. Technology is making it easier and cheaper for bad actors to mimic voices, convincing people, often the elderly, that their loved ones are in distress. In 2022, impostor scams were the second most popular racket in America, with over 36,000 reports of people being swindled by those pretending to be friends and family, according to data from the Federal Trade Commission
. Over 5,100 of those incidents happened over the phone, accounting for over $11 million in losses, FTC officials said.
Advancements in artificial intelligence have added a terrifying new layer, allowing bad actors to replicate a voice with just an audio sample of a few sentences. Powered by AI, a slew of cheap online tools can translate an audio file into a replica of a voice, allowing a swindler to make it “speak” whatever they type.
Experts say federal regulators, law enforcement and the courts are ill-equipped to rein in the burgeoning scam. Most victims have few leads to identify the perpetrator and it’s difficult for the police to trace calls and funds from scammers operating across the world. And there’s little legal precedent for courts to hold the companies that make the tools accountable for their use.
“It’s terrifying,” said Hany Farid, a professor of digital forensics at the University of California at Berkeley. “It’s sort of the perfect storm … [with] all the ingredients you need to create chaos.”
Although impostor scams come in many forms, they essentially work the same way: a scammer impersonates someone trustworthy — a child, lover or friend — and convinces the victim to send them money because they’re in distress.
But artificially generated voice technology is making the ruse more convincing. Victims report reacting with visceral horror when hearing loved ones in danger.
It’s a dark impact of the recent rise in generative artificial intelligence, which backs software that creates texts, images or sounds based on data it is fed. Advances in math and computing power have improved the training mechanisms for such software, spurring a fleet of companies to release chatbots, image-creators
and voice-makers that are strangely lifelike
.
AI voice-generating software analyzes what makes a person’s voice unique — including age, gender and accent — and searches a vast database of voices to find similar ones and predict patterns, Farid said.
It can then re-create the pitch, timber and individual sounds of a person’s voice to create an overall effect that is similar, he added. It requires a short sample of audio, taken from places such as YouTube, podcasts, commercials, TikTok, Instagram or Facebook videos, Farid said.
“Two years ago, even a year ago, you needed a lot of audio to clone a person’s voice,” Farid said. “Now … if you have a Facebook page … or if you’ve recorded a TikTok and your voice is in there for 30 seconds, people can clone your voice.”
Companies such as ElevenLabs, an AI voice synthesizing start-up founded in 2022, transform a short vocal sample into a synthetically generated voice through a text-to-speech tool. ElevenLabs software can be free or cost between $5 and $330 per month to use, according to the site, with higher prices allowing users to generate more audio.
ElevenLabs burst into the news following criticism of its tool, which has been used to replicate voices of celebrities saying things they never did, such as Emma Watson falsely reciting passages from Adolf Hitler’s “Mein Kampf.” ElevenLabs did not return a request for comment, but in a Twitter thread the company said it’s incorporating safeguards to stem misuse, including banning free users from creating custom voices and launching a tool to detect AI-generated audio.
But such safeguards are too late for victims like Benjamin Perkin, whose elderly parents lost thousands of dollars to a voice scam.
His voice-cloning nightmare started when his parents received a phone call from an alleged lawyer, saying their son had killed a U.S. diplomat in a car accident. Perkin was in jail and needed money for legal fees.
The lawyer put Perkin, 39, on the phone, who said he loved them, appreciated them and needed the money. A few hours later, the lawyer called Perkin’s parents again, saying their son needed $21,000 ($15,449) before a court date later that day.
Perkin’s parents later told him the call seemed unusual, but they couldn’t shake the feeling they’d really talked to their son.
The voice sounded “close enough for my parents to truly believe they did speak with me,” he said. In their state of panic, they rushed to several banks to get cash and sent the lawyer the money through a bitcoin terminal.
When the real Perkin called his parents that night for a casual check-in, they were confused.
It’s unclear where the scammers got his voice, although Perkin has posted YouTube videos talking about his snowmobiling hobby. The family has filed a police report with Canada’s federal authorities, Perkin said, but that hasn’t brought the cash back.
“The money’s gone,” he said. “There’s no insurance. There’s no getting it back. It’s gone.”
Will Maxson, an assistant director at the FTC’s division of marketing practices, said tracking down voice scammers can be “particularly difficult” because they could be using a phone based anywhere in the world, making it hard to even identify which agency has jurisdiction over a particular case.
Maxson urged constant vigilance. If a loved one tells you they need money, put that call on hold and try calling your family member separately, he said. If a suspicious call comes from a family member’s number, understand that too can be spoofed. Never pay people in gift cards, because those are hard to trace, he added, and be wary of any requests for cash.
Eva Velasquez, the chief executive of the Identity Theft Resource Center, said it’s difficult for law enforcement to track down voice-cloning thieves. Velasquez, who spent 21 years at the San Diego District Attorney’s Office investigating consumer fraud, said police departments might not have enough money and staff to fund a unit dedicated to tracking fraud.
Larger departments have to triage resources to cases that can be solved, she said. Victims of voice scams might not have much information to give police for investigations, making it tough for officials to dedicate much time or staff power, particularly for smaller losses.
“If you don’t have any information about it,” she said. “Where do they start?”
Farid said the courts should hold AI companies liable if the products they make result in harms. Jurists, such as Supreme Court Justice Neil M. Gorsuch, said in February that legal protections that shield social networks from lawsuits might not apply to work created by AI.
For Card, the experience has made her more vigilant. Last year, she talked with her local newspaper, the Regina Leader-Post, to warn people about these scams. Because she didn’t lose any money, she didn’t report it to the police.
Above all, she said, she feels embarrassed.
“It wasn’t a very convincing story,” she said. “But it didn’t have to be any better than what it was to convince us.”
|
