Researchers have raised concerns about vulnerabilities that could compromise AI service providers operating Hugging Face by loading custom malicious models.
Wiz's analysis showed that the researchers were able to execute arbitrary code after loading manipulated models into Hugging Face, leveraging this within Hugging Face's inference API feature for greater control.
Attackers could have a “devastating” impact on the Hugging Face environment if they successfully exploited these vulnerabilities, according to the study, granting them access to millions of private AI models and applications.
Worryingly, Wiz does not believe these findings are unique in any way, and the researchers cite this as a likely ongoing challenge for the AI-as-a-service industry.
“We believe those findings are not unique to Hugging Face and represent the tenant separation challenges that many AI services companies will face,” Wiz researchers said.
“We, in the security community, should partner closely with those companies to ensure that secure infrastructure and guardrails are put in place without hindering this rapid (and truly incredible) growth,” they added.
This vulnerability raises serious concerns for those seeking to use or provide AI as a service, adding a New and innovative AI-related attack route that should concern companies.
How does this vulnerability work?
Wiz's research led them to define two critical risks present in the Hugging Face environment that a theoretical threat actor could have exploited.
In the first case, called “shared inference infrastructure acquisition” risk, the researchers took advantage of the AI inference process, in which a trained model is used to generate predictions for a given input.
Wiz discovered that inference infrastructure often runs “untrusted” models that use the “pickle” format. A “pickle-serialized” model could contain a remote code execution payload, granting a threat actor enhanced privileges or cross-tenant access.
In the other form of attack, called “shared CI/CD acquisition” risk, threat actors could compile malicious AI applications with the intention of gaining control over the CI/CD pipeline to perform a chain of attack. supply, again paving the way for privileges. and access.
Attackers could also target different components through various methods, attacking, for example, an AI model directly through inputs to create “false predictions.”
Wiz made clear the need for developers and engineers to operate with greater caution when downloading models, as untrusted AI models could introduce obvious security risks into an application.
Another mitigation approach recommended by Wiz Research was to enable IMDSv2 with hop limiting to prevent pods from accessing the IMDS and gaining the role of a node within the cluster.
“This research demonstrates that using untrusted AI models (especially those based on Pickle) could have serious security consequences,” Wiz researchers said.
“Organizations should ensure they have visibility and governance of the entire AI stack being used and carefully analyze all risks, including the use of malicious models, exposure of training data, sensitive data in training, vulnerabilities in AI SDKs, AI, exposure of AI services and other combinations of toxic risks that can be exploited by attackers,” they added.
