Nvidia has launched a new industry alliance with major tech companies to strengthen the safety and cybersecurity of generative AI.

This move comes after recent security risks of autonomous AI were highlighted following an incident where OpenAI's AI agent went out of control and hacked Hugging Face.

▲ Adobe, Dell, CrowdStrike among participants... 'Open Secure AI Alliance' launched

According to Reuters on the 27th (local time), Nvidia announced the launch of the 'Open Secure AI Alliance' to jointly develop AI safety and cybersecurity technologies and share related tools.

Founding members include Adobe, CrowdStrike, Hugging Face, and Dell Technologies.

This alliance was promoted as an extension of an open letter 'Open Weights and American AI Leadership' recently announced by multiple AI companies including OpenAI. The letter calls for policy support for open-weight AI models.

▲ "Excessive regulation weakens AI defense capabilities"

Through an official blog, Nvidia argued that regulations that restrict cutting-edge open AI models could actually weaken security defense capabilities and concentrate power and dependency on a small number of closed AI companies.

The company emphasized its position that an open AI ecosystem enables higher levels of security and innovation based on transparency and collaboration.

▲ OpenAI agent hacking incident becomes turning point

The launch of this alliance was promoted following the recent OpenAI AI agent hacking incident.

On the 21st, OpenAI disclosed that one of its AI agents went out of control and made unauthorized access to the AI platform Hugging Face.

Reuters reported on the 25th that the AI agent continued hacking activities, and OpenAI only became aware of it after the threat had already been blocked and reported to the Federal Bureau of Investigation (FBI).

This incident is being evaluated as a catalyst for autonomous AI agents' control and monitoring systems emerging as an important issue.

Nvidia (Photo: [Reuters/Yonhapnews])

▲ Open weights AI support movement expands

Nvidia also signed the open letter 'Open Weights and American AI Leadership' on the 24th, supporting the expansion of an open AI ecosystem.

Open weights AI models refer to AI models that disclose core model weights and key components so that researchers and developers can freely utilize them.

Additionally, open-source software is drawing attention because by disclosing source code so anyone can review and improve it, it can enhance both technological accessibility and security.

▲ Nvidia also discloses AI agent control technology

Nvidia announced plans to provide the alliance with open AI models, model weights, datasets, and research results on AI agent control technology.

In particular, the open-source project 'Object-Oriented Agents' developed by Nvidia Labs was also released through the code sharing platform GitHub.

The company explained that the framework supports more effective management of AI agent behavior and can greatly simplify the process of testing, tracking, reviewing, and regulating AI behavior.

▲ Balance between openness and safety emerges as core issue in AI industry

The industry analyzed that securing both openness and safety simultaneously has emerged as a core issue in the AI industry as the autonomy of AI models and agents rapidly increases.

In particular, as the possibility of AI agents exhibiting unexpected behavior becomes reality, as in the recent Hugging Face hacking case, it is expected that efforts by AI companies to jointly develop security technologies beyond competition will expand further.