Download counts aren't trust: a lesson for AI coding agents

Developers often use download counts as a quick sign that a package is safe: if millions of people install it, someone would have noticed a problem. AI coding agents use similar signals when they choose dependencies. A case published by JFrog last month shows how easily that signal can be faked, and why agents that install packages need limits that don't depend on judging a package's reputation.

The context: a copied maths library

JFrog's research team reported on 21 September a package called mathsbase, a copy of the widely used mathjs library. It recorded about six million downloads in two days, yet no other package depended on it, which suggests the downloads were inflated rather than real use.

The package worked like the original, which made it look legitimate. Hidden in it was an encrypted payload, bundled with the package and decrypted at run time. JFrog describes it as targeted: it acted only on a specific input, making it harder to catch in ordinary testing.

Why agents are exposed

Coding agents add dependencies as part of their work. When asked to parse an expression or format a date, an agent may search for a package, pick one that looks popular and install it. Several features of agent behaviour make this risky:

• Popularity signals are easy to inflate. Download counts, stars and recent activity can all be manufactured, as this case shows.
• Similar names are easy to accept. A model can confuse a copy with the original, or invent a plausible name that an attacker then registers.
• Installation runs code. Install scripts execute on the developer's computer with the developer's access.
• Nobody reads the diff closely. A one-line addition to a dependency file is easy to approve among larger changes.

Analysis: what to make of it

Reputation checks are necessary but not sufficient. Checking that a package has dependents, a history and a known publisher would have flagged this one. But each check can be gamed in turn, and agents apply them inconsistently.

Containment matters more. If a malicious install script can't reach the network or read credential files, the damage is limited even when the package gets through. The Claude Code sandbox, for example, routes command network traffic through a proxy that allows only listed domains, starting from an empty list.

Allowlists have a gap. Allowing your package registry is usually needed for builds, and it also lets a malicious package be downloaded. Network limits stop the second stage, the payload fetched from elsewhere, more reliably than the first.

Targeted payloads defeat simple testing. A package that behaves normally except in certain environments won't be caught by running it once. Static checks and registry intelligence help more than trying it out.

What it means for engineering and security leaders

Treat dependency choices made by agents like those made by a new team member: allowed, but reviewed. The controls that already protect your supply chain, such as approved registries, lockfiles and dependency scanning, need to apply to agent changes as well as human ones.

On developer computers, add containment for the commands agents run, so a bad package can't easily phone home or read keys.

Practical takeaways for this week

• Require review of any new dependency an agent adds, with the package's publisher and dependents checked.
• Use an internal registry or proxy that can block known malicious packages.
• Turn on the sandbox in your coding agents, with a network allowlist and read-deny rules for credential folders.
• Add guidance to agent instructions: prefer existing dependencies, and name approved packages for common tasks.
• Make sure dependency scanning runs on pull requests created by agents.
• Check the dependents of any new package, not just its download count: a popular package with no dependents is a warning sign.