Recent findings by cybersecurity researchers have unveiled vulnerabilities in the Hugging Face Safetensors conversion service, potentially allowing adversaries to hijack user-submitted machine learning models, effectively leveraging them for supply chain attacks. The implications of this discovery raise significant concerns for businesses relying on the Hugging Face platform for their machine learning model management.
A report from HiddenLayer published last week detailed how compromised pull requests containing attacker-controlled data could target any repository on the Hugging Face platform, posing risks to models processed through the conversion service. This scenario enables malicious actors to exploit a hijacked model intended for conversion, allowing them to masquerade as the conversion bot, thereby gaining unauthorized access to repositories.
Hugging Face serves as a collaborative platform enabling users to host pre-trained machine learning models and datasets, alongside capabilities for building and deploying these models effectively. The Safetensors format, developed by the company, aims to enhance security in storing tensors, moving away from traditional methods such as pickle formats, which have been weaponized in the past to execute arbitrary code.
The vulnerability stems from a potential exploitation of the conversion service, wherein an attacker could utilize a malicious PyTorch binary to compromise the hosting environment. HiddenLayer’s analysis indicates that if the token related to the official SFConvertbot, responsible for generating pull requests, is exfiltrated, a malicious pull request could disrupt any repository, impeding model integrity and posing severe risks through implanted backdoors.
HiddenLayer researchers noted that an attacker could execute arbitrary code whenever a conversion is attempted. This poses a severe threat as models might be hijacked during the conversion process without any user notification. Furthermore, should an attempt be made to convert models from a private repository, sensitive Hugging Face tokens could be at risk of theft, granting adversaries unwarranted access to internal models and datasets.
The situation is exacerbated by the fact that any user can submit conversion requests for public repositories, which creates opportunities for adversaries looking to hijack or alter widely-used models, thus introducing considerable supply chain risks. The researchers indicated that, despite efforts to secure the ecosystem, the conversion service remains vulnerable, with the potential to instigate widespread supply chain attacks through the official Hugging Face channels.
These developments come on the heels of another notable vulnerability disclosed by Trail of Bits, known as LeftoverLocals (CVE-2023-4969, CVSS score: 6.5), which allows local attackers to access data across different processes, including interaction with large language models. This highlights a critical gap in security measures where sensitive information may be at risk due to inadequate isolation of process memory.
Security experts have emphasized the severe implications of data leaking in the context of machine learning systems, where local memory often stores essential model information. The ability of attackers to gain unauthorized access to these resources creates a challenging landscape for organizations that rely on advanced machine learning tools for their operations. The ongoing developments necessitate a reevaluation of security strategies associated with machine learning systems, especially within collaborative platforms like Hugging Face.
