Free exposure scan. Cleanup plans start at $1 for the first month. Start cleanup →
Home/Blog/AI Voice Cloning Scams
Back to Blog
Security Guide

Voice Phishing Rose 442%: How AI Voice Scams Use Broker Data

11 min read

CrowdStrike's 2025 Global Threat Report reported a 442% increase in voice phishing (vishing) between the first and second halves of 2024. That figure is CrowdStrike's own tracked activity, not all scams. Separately, Microsoft's VALL-E research describes cloning a voice from a 3-second sample. But a cloned voice is only half of a convincing scam. Attackers also want personal details: your name, your phone number, your family members, your address. Some of that can be found on people-search sites run by data brokers.

The Scale of the Problem

The FBI's Internet Crime Complaint Center reported about $20.9 billion in total losses for 2025 in its 2025 Internet Crime Report. Its AI Related descriptor covers 22,364 complaints and about $893 million in losses. IC3 counts only complaints people file.

In its 2024 financial statement, the engineering firm Arup said that in January 2024 criminals used fake voice, signatures and images to execute a fraud against it.

How AI Voice Cloning Works

Modern voice cloning operates on a simple input-output model. Feed the system a short audio sample (a voicemail greeting, a TikTok video, a podcast clip, an earnings call) and it generates synthetic speech in that person's voice, saying whatever text you provide.

Commercial and open-source tools now do this. Microsoft Research's VALL-E paper reports synthesizing a speaker's voice from a 3-second enrolled recording, and says the model can preserve the speaker's emotion and acoustic environment. How much audio a given product needs varies.

Quality has reached the point where a cloned voice can be hard to tell apart from the real one, so voice alone is not proof of identity.

The Data Broker Reconnaissance Layer

A cloned voice by itself may not convince you. A call that gets the name wrong or cites nothing real about your life is easier to hang up on. Context makes it more convincing, and some context can come from people-search sites.

People-search sites such as Spokeo, BeenVerified and WhitePages publish or sell profiles that can include:

  • Full legal name and aliases
  • Current and previous phone numbers
  • Home address and address history
  • Family members and known associates
  • Employer and job title
  • Email addresses
  • Age and date of birth

With details like these, an attacker could choose a family member to impersonate and cite your real address or employer. Combined with breached data, that is a fuller picture of you before the call is made. We are describing what is possible. We do not have data on how often it happens.

How an Attack Could Work

Step 1: A scammer looks up a target's profile on a people-search site. Step 2: They find a family member's voice sample online. Step 3: They clone that voice with an AI tool. Step 4: They call the target with the cloned voice and real personal details. This is an illustration of the risk, not a measured pipeline.

Why Traditional Defenses Fail

Spam call blocking is built to catch high-volume robocalls. A targeted call using a cloned voice looks different from a robocall, and a number shown on caller ID can be spoofed.

STIR/SHAKEN, a caller ID authentication framework for phone carriers, helps verify that calls come from the number displayed. It does not tell you that the person speaking is who they claim to be.

The fundamental problem is that these scams exploit human trust. We are wired to recognize and respond to familiar voices. When your mother calls and sounds panicked, your first instinct is not to verify. It's to help. Scammers exploit that instinct, and standard identity theft prevention measures don't address it.

Removing Your Data Cuts the Pipeline

Reducing what is findable about you is one countermeasure. If your phone number, your family members' names and your address are not on people-search sites, a scammer has less to work with. It does not stop all scams, because scammers have other sources.

The challenge is scale. Many brokers hold data, and a one-time opt-out from a single site covers only that site. GhostVault sends deletion demands to registered data brokers and follows up with the ones that stay silent. A request is not a removal until the broker confirms it. Plans start at $7.99/month or $69/year.

How to Protect Yourself and Your Family

  1. Establish a family safe word. Choose a word or phrase that only your immediate family knows. If anyone calls claiming to be a family member in an emergency, ask for the safe word before taking any action. This is a simple defense that does not depend on how good the voice sounds.
  2. Verify through a second channel. If you receive an urgent call from someone you know, hang up and call them back at their known number. Do not use the number displayed on caller ID. If the person is genuinely in trouble, they will answer. If it was a scam, you've just ended it.
  3. Remove your data from people-search sites. Eliminate the personal details that scammers use to construct targeted attacks. You can do this manually (our social media privacy guide covers the basics) or use GhostVault to automate requests to registered data brokers.
  4. Limit voice samples online. Be selective about posting videos, voice messages, and audio content publicly on social media. Set TikTok, Instagram, and YouTube videos to friends-only where possible. Any public audio can be a voice sample.
  5. Be skeptical of urgency. Every social engineering attack relies on urgency: "I need help now," "don't tell anyone," "send money immediately." Legitimate emergencies allow time for verification. Any caller who pressures you to act before thinking is almost certainly running a scam.
  6. Enable voicemail transcription. If your carrier offers voicemail transcription, use it. Reviewing a written transcript removes the emotional impact of hearing a familiar voice and makes it easier to evaluate the message rationally.

Where GhostVault Fits

GhostVault sends deletion demands to registered data brokers and shows you every reply. A request is not a removal until the broker confirms it, and no service can guarantee deletion. Removal reduces what is findable about you. It does not stop voice cloning.

The Regulatory Landscape

In February 2024 the FCC issued a declaratory ruling that AI-generated voices in robocalls are "artificial" under the Telephone Consumer Protection Act, so such calls generally need prior consent. That rule is about robocalls, not every voice scam. We did not verify state deepfake laws for this post.

On the data broker side, California's Delete Act (SB 362) created a centralized deletion platform, DROP, for California residents. Other states have data broker registries. Those apply only within their states and rely on broker compliance.

Frequently Asked Questions

How many seconds of audio does AI need to clone a voice?

Modern AI voice cloning tools can produce a convincing replica from as little as 3 seconds of audio. Services like ElevenLabs and open-source models such as OpenVoice and VALL-E can generate realistic speech from short samples found in voicemail greetings, social media videos, podcast appearances, and conference recordings. The quality improves with longer samples, but 3 seconds is sufficient for a phone call that fools most listeners.

What role do data brokers play in AI voice cloning scams?

Data brokers provide the reconnaissance layer that makes AI voice cloning scams effective. People-search sites sell detailed profiles including full names, phone numbers, home addresses, family member names, employer information, and social connections. Scammers purchase this data to identify targets, determine which family member to impersonate, obtain the correct phone number to call, and reference real personal details that make the cloned voice call believable. Without this data broker intelligence, a cloned voice alone lacks the context needed to convince victims.

How much money has been lost to deepfake and voice cloning scams?

Deepfake-related fraud losses reached an estimated $1.56 billion by 2025-2026. The FBI's IC3 recorded $16.6 billion in total cybercrime losses in 2024, a 33% year-over-year increase, with voice phishing representing one of the fastest-growing categories. Individual losses range from a few hundred dollars to tens of millions. In one case, scammers used a deepfake voice on a video call to authorize a $25 million bank transfer from a Hong Kong company.

How can I protect myself from AI voice cloning scams?

The most effective protection combines reducing your data broker footprint with behavioral safeguards. Remove your personal information from people-search sites to cut off the targeting pipeline. Establish a family safe word for emergency calls. Never act on urgent financial requests received by phone without verifying through a separate channel. Be skeptical of any call requesting money or sensitive information, even if the voice sounds familiar. Use a data removal service like GhostVault to file deletion demands with registered data brokers for $69 a year.

Can AI-cloned voices be detected?

Detection is possible but increasingly difficult. Commercial products from companies like Pindrop, Resemble AI, and Reality Defender can analyze audio for artifacts typical of synthetic speech: spectral irregularities, unnatural breathing patterns, and timing anomalies. However, the quality of AI-generated voices improves faster than detection technology. In controlled studies, human listeners correctly identify AI-cloned voices only about 50% of the time. The practical advice is to never rely on voice alone for identity verification. Always verify through a second channel.

Related guides

Popular on GhostVault