Widespread Voice Cloning Tools Fall Short on Safeguards

Recent findings highlight the troubling vulnerabilities inherent in popular AI voice synthesis tools. While artificial intelligence enables realistic voice cloning, research reveals that many of these tools lack adequate protections against potential misuse, raising alarms about identity theft and fraud.

The Consumer Reports study analyzed AI-powered voice cloning products offered by six companies: Descript, ElevenLabs, Lovo, PlayHT, Resemble AI, and Speechify. Alarmingly, the analysis concluded that four of these providers implement minimal or no effective security measures to mitigate abuse risks. Such deficiencies could empower malicious actors to exploit these technologies for their gain.

Specifically, tools from ElevenLabs, Speechify, PlayHT, and Lovo only require users to confirm their legal right to use a voice clone via simple self-attestation, significantly obstructing the enforcement of responsible usage. Conversely, Descript and Resemble AI have introduced stronger measures aiming to deter misuse, although these protections still fall short of rigorous standards.

The registration processes for several of these services are alarmingly straightforward, asking only for an email address and name. This minimal requirement allows bad actors to easily access advanced voice cloning capabilities without being subjected to stringent identity verification or the necessity of explicit consent from the individuals whose voices they may attempt to replicate.

The evolution of AI-driven voice cloning technologies has enabled the generation of remarkably lifelike imitations. While these advancements offer beneficial applications, such as assisting those with speech impairments or enhancing customer service operations, they concurrently pose significant risks, including the potential for financial fraud and sophisticated impersonation schemes.

Consumer Reports policy analyst Grace Gedye emphasized the pressing need for enhanced safeguards, noting that the current lack of protective measures could substantially exacerbate the prevalence of impersonation scams. She remarked, “Based on our assessment, there are foundational steps companies can implement to reduce the likelihood of voice cloning occurring without an individual’s knowledge; yet, many companies are neglecting to do so.”

The Federal Trade Commission (FTC) reported over 850,000 impostor scams in 2023, culminating in financial losses of $2.7 billion. Although data regarding the involvement of AI voice cloning in these incidents remains unclear, instances of fraudulent audio deepfakes have already made headlines, signaling an urgent need for more effective regulatory measures.

While some companies, such as PlayHT and Speechify, openly market their software for deceptive uses, others have adopted more cautious approaches. Microsoft, for instance, has chosen not to release its VALL-E 2 voice synthesis project, citing the risks associated with impersonation. Similarly, OpenAI has limited access to its Voice Engine as a precaution against similar threats.

Last year, the FTC advanced a rule prohibiting the AI-generated impersonation of government and corporate entities, although discussions surrounding broader bans on individual impersonations are still unfolding. Gedye further pointed out that regulatory actions at the state level may emerge as a more immediate solution than federal actions, especially considering the growing trend of undermining consumer protection agencies.