XRanges for AI offers a score for autonomous security agents

A product called XRanges for AI is being pitched as a way to measure how well autonomous security agents perform, according to an article published by The Hacker News on 23 September 2026. The pitch rests on test ranges that, the article says, 545 hackers worked through first.

The problem it claims to address is a familiar one for anyone running an AI agent against a target system. Autonomous security agents, the article says, are getting good at finding bugs, but nobody has a good way to measure how good. Point an agent at a realistic target and what comes back is a report the agent wrote about itself. That report tends to be confident prose with a list of findings, and no reliable way to tell which of those findings actually happened.

The result, according to the article, is that someone with a security background has to sit down afterwards and check every claim against the target. That person works through the list deciding which findings are real. It is slow, manual work, and it is the only current check on an agent's account of its own performance.

That gap matters because an agent's self-report is not evidence. An agent that hallucinates a vulnerability produces text that reads much like an agent that found one. Without an independent record of what happened on the target, a buyer comparing two agents is comparing two pieces of writing rather than two sets of results. The same issue affects teams trying to decide whether an agent has improved between versions.

The named figure of 545 hackers is offered as a point of reference. The suggestion is that the ranges are not purpose-built puzzles for machines but environments that human testers have already worked through, so a score produced on them can be read against human performance on the same ground.

The article, as published, does not set out how the scoring works in detail, what the ranges contain, which agents have been run against them, or what the results were. It also does not state pricing or availability. Readers should treat the claims as the vendor's own until independent results appear.

The wider point stands regardless of the product. Agentic security tooling is being sold on the basis of what it says it did. Verification of those claims is currently a human job, performed after the fact, by people with the expertise to reproduce a finding. Any benchmark that makes that check cheaper or repeatable would change how these tools are bought. Whether XRanges for AI does that is not yet something an outside reader can judge from the material published so far.