Apple and Google have received cease-and-desist letters mandating the removal of specific applications that enable users to create “nudified” or nonconsensual nude images. These legal notices, issued by San Francisco City Attorney David Chiu, call on both tech giants to halt their financial gains from potentially harmful technologies.
On Thursday, Chiu requested the immediate removal of 13 face-swapping apps from the companies’ app stores, highlighting that these tools enable the generation of AI-driven explicit images without consent. The city attorney emphasized that Apple and Google should cease their “aiding and abetting” of the distribution of explicit deepfake content and sever ties with the developers behind these apps.
“Generating non-consensual intimate images is illegal, harmful, and entirely unacceptable,” Chiu stated, revealing that his office has previously taken action against 16 notorious deepfake websites. He raised concerns that Apple and Google have likely profited significantly from these applications, suggesting that they must enhance their moderation processes to prevent such items from appearing on their platforms.
According to the legal letters, California law explicitly prohibits support for services responsible for creating deepfake pornography. Notably, the apps in question leverage in-app payment systems, from which Apple and Google receive a portion of the revenue. “The involvement of some of the largest technology firms in this issue must come to an end,” Chiu added.
Experts have consistently found numerous applications in both Apple’s App Store and Google Play Store that facilitate the generation of sexualized images through AI. Alarmingly, some of these applications have received ratings deeming them suitable for children. Despite new laws seeking to combat explicit deepfake content online, technology and social media companies continue to direct millions of users towards potentially harmful tools.
Both Apple and Google have established developer policies that specifically ban pornography, abuse, and harassment on their platforms. They have previously removed multiple nudify and deepfake applications following findings from researchers and journalists. Google spokesperson Dan Jackson stated that the company has taken measures to delete “hundreds” of apps with nudifying features, including the five flagged by Chiu’s office.
Jackson reaffirmed, “Google Play does not permit apps containing sexual content, and we are proactive in identifying and removing harmful content.” He explained that the company investigates reported violations and has taken decisive actions, which include suspending hundreds of offending apps and limiting search terms like “nudify” within their store.
As of now, Apple has not provided an official comment regarding the matter. Over the past five years, the market for deepfake “nudification” technology has expanded significantly, most notably with the January utilization of xAI’s Grok to generate millions of sexualized images. Various applications allow users, primarily men, to upload images—often of women and girls—and digitally manipulate them to simulate sexual scenarios.
Creating these deepfake images frequently requires only a reference photo and a few clicks, with some results available almost instantaneously. The advancements in generative AI technology have made the resulting images increasingly realistic, with many services offering either free or low-cost options for the creation of such harmful content. Previous investigations have revealed numerous cases in at least 90 schools where sexualized deepfake images of minors were generated, calling into question the responsibility of tech companies in mitigating this pervasive threat.