Back

LLM penetration testing orchestrators surge after ARTEX takedown
SiTech AI Team3 min read

LLM penetration testing orchestrators surge after ARTEX takedown

Cloudflare says AI-driven penetration testing frameworks have multiplied since ARTEX was removed on October 8, with daily requests up 30x since October 1, thousands of IPs running them, and defenders struggling to keep pace.

Cloudflare says the number of Large Language Model penetration testing and orchestration frameworks has exploded in recent weeks, with over 50 framework families attempting to scan Cloudflare customers globally in the last three days. Daily requests from AI penetration testing and exploitation tools have grown by 30x since October 1, and the number of IP addresses running these frameworks has grown into the thousands. The company recommends that organizations adopt a six pillar Modern Security Model to counter the threat.

From ARTEX to thousands of forks

ARTEX, an open source agentic penetration testing system, was taken down by its creator Autumn-27 on October 8. In the month since, the ecosystem has exploded with fragmented versions, with over 12,000 repositories and several hundred unique forks containing lineage from the original repository. Hundreds of other similar agentic orchestration frameworks have also been created since October 8. As of November 12, ARTEX and its forks make up over 60% of currently known AI penetration testing orchestrators.

How the tools operate

Open source pentesting frameworks can orchestrate AI agents across stages of the traditional Cyber Kill Chain at scale. ARTEX and its forks map assets and environments onto a graph, allowing multiple independent parallel agents to enumerate, verify, prototype, and attack at the speed of inference. The tools can plug in existing Model Context Protocol and connect traditional standalone pentesting tools in novel ways, and some allow operator direction on what type of task to accomplish. Cloudflare says it is tracking more than 50 AI LLM agents, and that each of the top five orchestrators is fully autonomous and has more than 1,000 GitHub stars.

Observed attacks and incidents

Cloudforce One has been called into multiple incident response investigations involving ARTEX and ARTEX derived LLM exploitation frameworks over the last eight weeks. In multiple incidents, a threat actor gained initial access to a hyperscaler environment and pivoted directly into a customer's Cloudflare environment. Root cause analysis revealed a common credential hygiene failure: attackers repeatedly compromised static, non expiring tokens running inside CI/CD pipelines, which granted broad, cross environment access across an organization's hyperscaler infrastructure. In one investigation, ARTEX gained access to a compromised account level API token previously used for CI/CD with no IP restrictions or rate limiting, and activity exceeded 69 requests per minute across multiple unique parts of Cloudflare infrastructure, with 400 database dump requests in an hour. Cloudflare believes much of that activity was only lightly guided by the human operator.

The Modern Security Model

Cloudflare recommends six pillars: fortify the front gate with a Web Application Firewall and advanced rules such as rate limits; add anomaly detection on all web applications customers access; implement Bot Defense; implement Identity Gated Access for all high risk external and internal web applications; enforce validated schemas and explicit security controls; and implement Threat Intelligence for visibility into attacks. The company says legacy defense in depth models fall short in today's complex architectures, while a containment first approach is essential when users consistently lose their keys. If facing an active incident, Cloudflare directs readers to contact its Cloudforce One team, and says its Solutions Engineering team can help implement the model.

Sources: Cloudflare

SiTech

SiTech — AI-powered web development

We build fast, modern websites and bring AI into real business workflows. Have a project or a question? We'd love to help.