Self-Replicating AI Swarms Are Real. Here’s What a Small Business Should Do.

Take them seriously. Just don’t panic.

There’s a lot of AI and agent content online. Very little of it is worth talking about, because so few people spend their whole day in this world.

Nate B Jones is one of the people I look up to for his commitment to the AI space. This post is a summary of his new video, Self-Replicating AI Swarms Are Real. Don’t Panic, But Take Them Seriously., plus my take.

This will probably get a lot of media hype in the coming days and weeks. It’s an important step in how agents are evolving, and as Nate says, it was predictable. This little corner of AI has really taken off in the last 60 days.

What happened

In a lab, researchers built a swarm of AI agents that copied itself from machine to machine, as many as seven generations deep. It shared the passwords it stole along the way.

Out on the open internet, a group called Swarm Chasers (think tornado chasers) followed unusual traffic to a fleet of agents on Tencent Cloud. Over 11 days, that fleet made more than 2,600 attempts to pull information from Amap, Alibaba’s mapping service, including questions about which zoo entrance people use. When blocked, they turned to outside services and exposed credentials. The researchers left a hidden note offering agents a way to get help. The agents wrote back.

Nate is careful here. There’s some indication Tencent was running an evaluation, and the fleet may be related to that. Skeptics say it was just a browsing test.

Meanwhile, Anthropic got a model running on cheap TV set-top boxes, about $55 each. It also warned about self-replicating agents in a recent paper.

Why it matters for your business

Agents are tireless. When they hit a wall, they push through. That’s what they’re trained to do. So an ordinary task, done at scale, can look like an attack.

Nate expects a two-tier internet: hardened sites like Amazon that say no to agents, and sites like Shopify that welcome them. Less-defended sites may become hotbeds for agent activity, including hidden messages only agents can read, like white text on a white page.

Your own agent is exposed, too. If your assistant meets a wild agent online, could it get prompt injected? That means tricked by hidden instructions into doing something you never asked for. Nate says there are no documented cases yet, but the ingredients are there.

When we use agents in our workflows, they might run into an agent in the wild whose intent we don’t know. Maybe it’s malicious, run by a hacking outfit. All we can do is follow best practices for a safe setup, especially for production-grade work: the stuff that has to run well because clients or profit depend on it.

It’s a balance. Agents give us amazing convenience and leverage, and this threat is the counterweight. The early internet started as a public good. Then crime and abuse on it became a real threat, and governments could use that as an excuse to clamp down. This one is about protecting our own agents and AI setup from bad encounters with wild or malicious agents.

My take: stay vigilant

Have a few agents scan your software, your code, and your agents’ output. It’s a quick way to spot anything off.

Good rogue swarms will try to be as undetectable as possible. So it’s our job to run a little counterintelligence.

That’s a new job for our agents: a bit of cybersecurity and counterintelligence, checking for anything strange in our systems. I think it’s a really cool concept.

What to do this week

• Lock down agent permissions for money and files.
• Keep audit logs so you can see what your agents did.
• Check your website forms and how you handle bots. Know what code gets deployed to your site.
• Set a safe word with your family and your team. Agents can make calls and imitate voices now.
• Make sure any agent you run has a control system and an off switch.
• Have a couple of your agents scan your software, code, and agent output for anything strange.

And be a good citizen with your own agents. Don’t send one to spam reservations across hundreds of restaurants.

The calm part

Nate says the systems that run agents have two jobs. One is making up for today’s weaker models, and that job is going away. The other is control: permissions, an off switch, an audit trail. That one stays, and it’s why companies will move slowly into swarms.

It’s the same reason Agints installs digital secretaries with permissions and audit trails built in.

As Nate puts it, “This was entirely predictable.” It’s not Terminator. It could even make the internet richer, with agents pointing people to small shops they’d never have found.

---

Drowning in busy work like inbox and research? We install a digital secretary that clears 30–70% of your busy work, even while your laptop is closed, with permissions and an audit trail built in. Talk to Gini at agints.com/free.

Get the next one in your inbox → agints.substack.com