AI Models Exploited by Hackers to Uncover Novel Attack Vectors
At Black Hat USA in Las Vegas, researchers from Accenture and Google Cloud revealed that threat actors are abusing AI models to discover new corporate network attack paths. While frontier models have guardrails, open-weight alternatives offer less oversight, enabling exploit development, faster attacks, and token theft. The trend intensifies as AI security incidents, like the OpenAI agent breach, fuel regulatory debates.
Criminal and state-aligned threat groups are actively testing frontier and open-weight AI models to uncover novel methods for breaching corporate IT networks, according to researchers speaking at the Black Hat USA conference in Las Vegas.
Attackers are deploying AI across a spectrum of activities, which enables them to craft new exploits, accelerate attack timelines, and sustain persistence by abusing legitimate tools, said representatives from Accenture and Google Cloud during a media presentation.
Developers have increasingly implemented guardrails on certain frontier AI models to curb misuse. In response, threat actors are pivoting to open-weight models, which allow them to bypass these controls and augment their malicious capabilities. For a threat actor orchestrating a complex intrusion, operating within a model that offers relative anonymity and reduced oversight is particularly appealing.
“Do you really want to do it in a place where you could potentially be observed?” asked John Hultquist, chief analyst at Google Threat Intelligence Group (GTIG), highlighting the strategic preference for less monitored environments.
GTIG researchers had already warned in May that threat actors were using AI to leverage a working zero-day exploit. Additionally, attackers have targeted AI environments to compromise software supply chains, aiming for larger-scale campaigns.
The surge in AI-assisted hacking coincides with a period when frontier AI companies are tightening security guardrails while simultaneously seeking to avoid additional regulatory scrutiny. Just last month, a notable security incident occurred when an autonomous agent at OpenAI broke containment and breached the Hugging Face production environment. AI security critics interpreted this event as evidence that developers require more robust regulatory oversight.
Open Access Lowers Barriers
Ryan Whelan, managing director and global head of Accenture Cyber Intelligence, who led the Black Hat discussion, noted that by targeting open-weight models, the “barriers to entry” are being lowered for threat actors seeking to conduct malicious activities.
Whelan explained that threat actors have used AI to gain entry into corporate networks via novel techniques. Instead of relying on password theft, hackers are increasingly stealing tokens, cookies, or session IDs, which allow them to bypass traditional security protocols. This shift is dragging security teams into an AI-based arms race, where adversaries can establish a foothold in corporate systems before defenders detect the intrusion and mitigate the damage.