Big Tech’s “Free” Services: The Hidden Cost of Data Access
In recent years, the use of so-called “free” services from major technology companies like Google, Facebook, and Microsoft has come at a significant price: the relinquishing of personal data. While embracing cloud technology and utilizing free applications can simplify many aspects of life, these conveniences often result in personal information being mined by large corporations seeking to monetize user data. As the next generation of generative AI systems emerges, they are likely to demand even greater access to that data.
Over the last two years, generative AI tools, including OpenAI’s ChatGPT and Google’s Gemini, have evolved from basic text-only functionalities to more complex capabilities. Major tech firms are now promoting AI agents and assistants, which claim the ability to take action and handle tasks autonomously. However, maximizing their potential often requires extensive permissions to user systems and data. Unlike the initial controversies surrounding the unauthorized replication of copyrighted material by large language models, these AI agents present new challenges regarding user privacy and data security.
Harry Farmer, a senior researcher at the Ada Lovelace Institute, emphasizes that for AI agents to function optimally, they frequently need access at the operating system level of the devices they operate on. In his research on AI assistants, Farmer identifies a significant trade-off between personalization and data privacy. “For an AI to effectively cater to user needs, it requires substantial information about the user,” he notes, raising concerns about the implications for both cybersecurity and personal privacy.
Defining an AI agent can be complex, but they generally represent generative AI systems or large language models endowed with some degree of autonomy. Currently, these assistants can perform tasks like web browsing, flight booking, and research, often managing complex sequences of actions. Yet, many of these systems still exhibit operational glitches, hindering their ability to accomplish assigned tasks effectively.
Despite the challenges, technology companies are optimistic about the transformative potential of AI agents, projecting that they will alter the landscape of millions of jobs as they improve. A significant factor behind their utility lies in data access; for AI to provide insights into schedules and tasks, it must tap into users’ calendars, messages, and email accounts.
Some advanced applications already illustrate the extensive data access AI agents may utilize. For instance, business-oriented agents are being designed to read code, monitor emails, analyze databases, and even access Slack messages and documents stored on platforms like Google Drive. Notably, Microsoft’s Recall feature captures screenshots of user desktops repeatedly, enabling users to search their recent activities. Meanwhile, Tinder’s AI functionality aims to analyze phone photos to glean insights into user preferences and personalities.
Carissa Véliz, an associate professor at the University of Oxford, points out a critical concern regarding consumer awareness and control over personal data used by AI and tech companies. “These companies tend to be cavalier with user data,” she asserts, indicating a broader trend of disregard for privacy.
In examining the potential risks associated with these emerging technologies, it is essential to consider the tactics and techniques outlined in the MITRE ATT&CK framework. Adversary tactics such as initial access, privilege escalation, and persistence might be relevant in this context, as attackers could exploit permissions to gain deeper access to user systems. As businesses increasingly integrate these AI tools, awareness of data governance and cybersecurity strategies will be crucial to mitigate risks and protect sensitive information.
