Nvidia and several major technology companies have launched a new artificial intelligence safety initiative that is focused on improving the security capabilities of open AI models. The initi
Nvidia and several major technology companies have launched a new artificial intelligence safety initiative that is focused on improving the security capabilities of open AI models.
The initiative follows a cyberattack that involved a rogue OpenAI model that reportedly targeted AI platform Hugging Face. During the incident, Hugging Face was unable to use leading US frontier models to defend its systems because built-in safety guardrails could not reliably distinguish between malicious activity and defensive cybersecurity work.
Instead, the company reportedly turned to a self-hosted Chinese open-weight model that was not subject to the same restrictions.
Nvidia announced that the Open Secure AI Alliance will develop and share open technologies that are designed to identify, disclose and fix security vulnerabilities. According to Nvidia, the Hugging Face incident proved that cybersecurity teams need access to open, frontier-level agentic AI systems for defensive purposes.
(Source: Nvidia)
The alliance includes Nvidia, Microsoft, SpaceX, Palantir and dozens of other technology companies from the United States and Europe. Members are expected to collaborate on open AI tools that can support cybersecurity research and strengthen organizations’ ability to respond to sophisticated attacks.
The initiative could also help reduce the technology industry’s reliance on Chinese open-weight AI models.
The Nvidia-backed alliance comes as US policymakers consider measures to limit the use of Chinese AI systems.
Chinese AI companies have faced accusations of using distillation techniques to extract information from more advanced models developed by US competitors. Distillation generally involves training one model using the outputs or capabilities of another, more powerful system.
US Treasury Secretary Scott Bessent recently warned that Chinese companies accused of conducting unauthorized distillation attacks could face sanctions.
Experts believe possible restrictions could include banning US companies from purchasing access to Chinese AI models through application programming interfaces. Measures could also prevent American cloud providers from hosting these models and charging customers to use them.
Strict guardrails can help prevent models from being used for attacks, but they may also limit legitimate defensive work. The Hugging Face incident suggests that organizations could struggle to protect themselves when closed models refuse cybersecurity-related requests without recognizing the difference between an attacker and a defender.
Through the new alliance, Nvidia and its partners are trying to build an open ecosystem in which advanced AI security tools can be developed, tested and shared more widely. The initiative may become very important as governments consider restrictions on Chinese models and businesses look for secure alternatives that can be hosted and controlled independently.
