AI’s Habit of Making Things Up Becomes a Security Nightmare

AI coding assistants have a well-known flaw: they hallucinate. Ask one to recommend a popular library or tool, and it might return a name that sounds legitimate but doesn’t actually exist. That quirk has always been annoying. Now, researchers have turned it into a weapon.

The attack is called HalluSquatting, and it’s disturbingly simple. The idea: figure out which fake names a given AI model reliably invents, register those names on package repositories before anyone else does, and then wait for the assistant to recommend your malicious package to an unsuspecting developer. When the developer’s machine installs it, the malware executes.

The research, led by a team of security experts, demonstrates a new class of supply chain vulnerability — one that exploits the very thing we’ve been told to tolerate as a harmless side effect of generative AI.

How HalluSquatting Works: The Three-Step Trap

The attack unfolds in three stages. First, the attacker probes a coding assistant — like GitHub Copilot or any large language model trained on code — to find packages it hallucinates. This means asking it repeatedly for package recommendations and recording the names it invents.

Second, the attacker registers those hallucinated package names on real package registries such as PyPI (for Python) or npm (for JavaScript). They upload a package that includes functional code — but also a hidden payload: a botnet client, a backdoor, or crypto miner.

Third, the attacker waits. When a developer asks the same AI assistant for that package, the model confidently suggests the hallucinated name. The developer runs a pip install or npm install command. The package downloads. The malware activates. And the developer’s machine becomes part of a botnet.

Why This Is Harder to Spot Than Traditional Typosquatting

Traditional typosquatting relies on misspellings — registering requrests instead of requests. Developers can catch those with a careful eye. But HalluSquatting packages don’t resemble any real project. They’re entirely fabricated names that the AI invented. A developer has no reason to be suspicious: the name looks like something that should exist, and the AI seems confident about it.

“The assistant doesn’t know it’s wrong,” the researchers note. “It’s just predicting the next token. But the developer trusts it.”

The Scale of the Problem: Thousands of Hallucinated Packages

The research team tested multiple AI coding assistants and found that they consistently hallucinated package names at alarming rates. In one experiment, a model invented fake package names for over 20% of queries about niche libraries. Across all tests, the researchers identified thousands of unique hallucinated package names that could be registered by an attacker.

Some of these packages were hallucinated repeatedly — the same fake name appearing across multiple sessions with different users. That reliability is exactly what makes the attack viable. An attacker doesn’t need to guess; they can just ask the AI, record the answer, and register the name.

Real-World Implications: Botnets and Beyond

The attack isn’t theoretical. The researchers demonstrated a proof-of-concept by registering a hallucinated package on PyPI, complete with a benign payload that phoned home to a server they controlled. Within days, the package was downloaded by automated build systems that had been prompted by AI assistants.

In a real attack, the payload could be a botnet client — turning every infected developer machine into a node for DDoS attacks, credential theft, or cryptocurrency mining. The potential for lateral movement is significant: a compromised developer machine often has access to production servers, source code repositories, and deployment pipelines.

This is a supply chain attack that doesn’t require compromising a legitimate package. It creates a fake one from scratch, using the AI’s own hallucination as the attack vector.

What Developers Can Do Right Now

There is no easy fix. The underlying issue — that AI models hallucinate — is a fundamental limitation of current technology. But developers can take practical steps to reduce risk:

• Verify before installing: Always check whether a recommended package exists on the official registry before running an install command. A quick search on PyPI or npm takes seconds.
• Use package pinning: Lock dependencies to specific versions and hash values. This prevents a malicious package from being swapped in later.
• Run in sandboxed environments: Execute AI-suggested code in isolated containers or virtual machines, especially when it involves installing new packages.
• Treat AI suggestions as starting points: Consider AI recommendations as research suggestions, not trusted answers. Cross-reference with documentation or community resources.

The researchers also suggest that package registries could implement proactive scanning for newly registered packages that match hallucinated names from popular AI models. But that would require cooperation between AI companies and registry maintainers — something that doesn’t exist yet.

The Bigger Picture: Hallucination as an Attack Surface

HalluSquatting is part of a growing recognition that AI hallucinations aren’t just a quality problem — they’re a security problem. When a model confidently produces false information, and that information is acted upon by a human or an automated system, the consequences can be severe.

This attack is specific to coding assistants, but the principle applies more broadly. Any AI system that generates plausible-sounding but false references — citations in academic writing, product recommendations in e-commerce, configuration settings in DevOps — could be exploited in a similar way.

For now, the coding assistant space is the most vulnerable. Developers are trained to trust their tools. And an AI that sounds certain is very persuasive. The HalluSquatting research is a reminder that in the rush to adopt generative AI, we’ve introduced a new class of attack that no one was watching for.

The question isn’t whether attackers will exploit this. They already are. The question is how quickly the industry can adapt.

False Positives Are a Plague. Capital One Built a Tool to End Them.

Software security teams have a dirty secret: most of the alerts they chase every day are noise. False positives pile up, developers tune out, and real vulnerabilities slip through the cracks. Capital One says it has had enough — and it just released the weapon it built to fight back.

The company open-sourced VulnHunter, an AI-powered tool designed to find exploitable code flaws, trace the paths an attacker might take, and recommend specific fixes. It is not a passive scanner that drowns teams in alerts. It is an agent that reasons about code.

Chris Nims, Capital One’s EVP and Chief Information Security Officer, described the problem bluntly in a LinkedIn post: overwhelming false positives create friction and slow down daily workflows. VulnHunter was built with a developer-first mindset to solve that exact pain point.

How VulnHunter Works: Agentic Reasoning, Not Just Pattern Matching

Traditional vulnerability scanners look for known signatures — they match code against a database of bad patterns. VulnHunter does something different. It uses an agentic reasoning workflow to understand the context of a flaw, determine whether it is actually exploitable, and map out the attack path an adversary would take.

Then it proposes remediations. Not generic advice, but targeted code changes tailored to the specific repository.

This is a significant departure from the status quo. Most security tools generate a list of problems and leave the investigation to humans. VulnHunter attempts to do the investigation itself — and then hand the developer a fix.

Capital One claims that, during internal use, the tool quickly identified and remediated vulnerabilities across thousands of repositories spanning tens of business areas.

Available Now on GitHub — With One Big Catch

VulnHunter is live on GitHub, alongside a quickstart guide, architecture documentation, and example workflows that show how the tool traces code paths and generates remediations.

There is a catch: you need access to Claude Opus 4.8 and a working Claude Code environment. The tool leans heavily on Anthropic’s frontier model for its reasoning capabilities. That means it is not a drop-in, zero-cost solution for everyone. But for organizations already using Claude, it could be a powerful addition to the security stack.

Why Open Source? Supply Chain Realities

Capital One did not keep VulnHunter for itself. The company open-sourced it, and the reasoning is worth reading.

Modern software supply chains are deeply interconnected, the company explained. A single vulnerability in a widely used open-source component can ripple across thousands of enterprises simultaneously. No single organization can solve that challenge alone.

“The defensive tools to address this reality need to be just as widely distributed, tested, and improved as the codebases they protect.” — Capital One

That is a refreshingly honest take from a financial giant that could easily have kept the tool proprietary. By open-sourcing VulnHunter, Capital One invites the broader security community to audit, improve, and extend the tool — which, in turn, makes it more effective for everyone.

What This Means for Developers and Security Teams

For developers, the promise is simple: fewer false positives, faster fixes, less friction. For security teams, it means an AI agent that does the heavy lifting of triage and root-cause analysis.

The tool also reflects a broader shift in the industry. Agentic AI — where models don’t just answer questions but take actions — is moving into cybersecurity. CISA reportedly using Anthropic’s Mythos to scan government software for flaws is another sign that this trend is accelerating.

VulnHunter is not a silver bullet. It requires a specific AI infrastructure to run, and its effectiveness depends on the quality of the model underneath. But it represents a genuine attempt to solve one of the most painful problems in software security: the signal-to-noise ratio.

If it works as advertised, it could change how enterprises approach vulnerability management — from reactive alert-chasing to proactive, AI-assisted remediation.

The Old Playbook Is Outdated

For years, the conventional wisdom held that Iran’s state-sponsored hackers mainly went after big, obvious targets: power plants, water utilities, oil refineries. The logic was simple. Disrupt a nation’s critical infrastructure and you create real-world chaos without firing a shot. But that focus is shifting — and the shift matters for every company with a server connected to the public internet.

Iranian cyber groups are broadening their scope. They’re no longer laser-focused on critical infrastructure alone. The new reality: if your organization has any internet-facing vulnerability, you’re a potential target. Obscurity is not a defense. A small logistics firm, a regional hospital, a mid-sized manufacturer — all are now in the crosshairs.

Why the Expansion?

Several factors drive this evolution. First, Iran’s cyber apparatus has matured. Groups like APT33, APT34 (OilRig), and affiliated hacktivist personas have built sophisticated toolkits and operational experience. They can pivot faster than ever.

Second, the geopolitical calculus has changed. With sanctions tightening and regional tensions simmering, Tehran sees cyber operations as a low-cost, high-impact lever. Hitting a power grid makes headlines. But quietly compromising dozens of smaller targets — stealing data, deploying ransomware, establishing persistent access — can yield intelligence, leverage, and revenue without triggering a massive response.

Third, the attack surface has exploded. Remote work, cloud migration, and IoT proliferation mean more exposed systems than ever. Many of these belong to organizations that never considered themselves likely targets. They do not have the cybersecurity budgets of a national grid operator.

From Espionage to Extortion

The tactics are evolving too. Traditional Iranian operations focused on espionage and sabotage. Today, you see a wider mix. Ransomware deployments by groups with Iranian fingerprints are on the rise. Data theft for extortion is common. Hacktivist fronts launch DDoS attacks and defacements against perceived enemies of the regime.

Take the recent campaigns targeting Albanian government infrastructure, or the ongoing intrusions into Israeli water and energy sectors. These are not isolated incidents. They signal a willingness to hit any connected target that serves a strategic or tactical purpose.

What This Means for Your Organization

If you run an IT security team, the message is blunt: stop assuming you are too small or too boring to attract attention. Iranian threat actors are scanning the internet constantly. They look for unpatched vulnerabilities, exposed RDP ports, weak credentials, misconfigured cloud storage. When they find one, they probe. If the door opens, they walk in.

The initial access might not be used immediately. Often, attackers establish a foothold and wait — mapping the network, stealing credentials, identifying valuable data. The attack may unfold weeks or months later.

Practical Steps to Reduce Risk

Defending against this expanded threat landscape does not require a spy agency budget. It does require discipline and prioritization. Here are concrete measures every organization should take:

• Patch aggressively. Prioritize internet-facing systems. Known vulnerabilities are the #1 entry vector for Iranian groups. Automate patching where possible.
• Reduce the attack surface. Audit every public-facing service. If it does not need to be on the internet, take it offline. Use VPNs and zero-trust architectures.
• Harden authentication. Enforce multi-factor authentication everywhere, especially on remote access and administrative accounts. Iranian actors are skilled at credential theft.
• Monitor for unusual activity. Look for lateral movement, unusual outbound connections, and unexpected data transfers. Basic network monitoring catches many intrusions early.
• Have an incident response plan. Test it. Know who to call if you find a compromise. Speed matters.

The Bigger Picture

Iran’s cyber expansion is part of a global trend. State-sponsored groups everywhere are moving beyond traditional targets. The lines between espionage, crime, and sabotage are blurring. For defenders, the key takeaway is simple: you cannot rely on being overlooked. The internet is a flat battlefield. Vulnerabilities are opportunities — and adversaries are scanning for them 24/7.

Understanding Iranian cyber threat evolution is no longer niche knowledge for national security experts. It is basic risk management for any connected business. The question is not whether you will be scanned. It is whether you will be ready when the scan finds something.