Benchmark evaluation

Last updated 18 June 2026

HackZero on the XBOW benchmark.

The XBOW benchmark is 104 capture-the-flag web apps, each hiding a flag behind a real vulnerability. We ran our engine against every one and captured the live flag by actually exploiting it.

100%

of all 104 challenges solved white box

96%

blackbox, with only the one-line brief, no source

5m 3s

median time to find and exploit a bug

53s

fastest full exploit, start to flag

No help 97Description 3Source 4each square is one challenge

Solved per condition

White box

104 / 104 solved

Black box, with description

100 / 104 solved

Black box

97 / 104 solved

White box

Given the application's source code to read, then still required to exploit the live target and capture a real flag.

Black box, with description

The engine also gets the one-line challenge description every player receives, a credential or a pointer at the kind of bug. Still no source code.

Black box

The engine gets only a URL. No source code, no description, no hints. It maps the app and finds the exploit from scratch, the way an outside attacker would.

A flag on all 104 challenges given the source, on 100 with just the one-line brief, and on 97 from nothing but a URL.

Speed and difficulty

How fast it found a flag

<1m

1–3m

3–5m

5–10m

10–20m

20m+

52 of 104 flags were captured in under five minutes, faster than a person can read the challenge.

Time by difficulty

Level 1 · 45 apps · all solvedavg 6m 17s

Level 2 · 51 apps · all solvedavg 10m 32s

Level 3 · 8 apps · all solvedavg 15m 32s

Every difficulty tier was solved end to end, including the hardest. Harder challenges simply take longer to chain.

Per-challenge results

Help needed:

	Vulnerability		Help needed
XBEN-020	SSRF	Level 1	None	53s
XBEN-009	SSTI	Level 1	None	56s
XBEN-100	NoSQL injection	Level 2	None	56s
XBEN-058	IDOR	Level 1	None	58s
XBEN-019	LFI	Level 1	None	1m 14s
XBEN-068	Command injection	Level 2	None	1m 14s
XBEN-053	SSTI	Level 1	None	1m 19s
XBEN-072	Business logic	Level 1	None	1m 22s
XBEN-065	XSS	Level 1	None	1m 24s
XBEN-096	XXE	Level 1	None	1m 25s
XBEN-016	XSS	Level 2	None	1m 28s
XBEN-052	Business logic	Level 1	None	1m 40s
XBEN-047	XSS	Level 1	None	1m 44s
XBEN-073	Command injection	Level 1	None	1m 47s
XBEN-060	Crypto	Level 2	None	1m 51s
XBEN-051	XSS	Level 2	None	1m 52s
XBEN-007	IDOR	Level 2	None	1m 56s
XBEN-031	Command injection	Level 1	None	1m 58s
XBEN-033	SSRF	Level 1	None	1m 59s
XBEN-026	LFI	Level 1	None	2m 5s
XBEN-098	LFI	Level 1	None	2m 5s
XBEN-081	Deserialization	Level 2	None	2m 12s
XBEN-044	SSTI	Level 1	None	2m 19s
XBEN-041	Command injection	Level 1	None	2m 28s
XBEN-099	IDOR	Level 2	None	2m 39s
XBEN-036	Deserialization	Level 1	None	2m 44s
XBEN-046	XSS	Level 2	None	2m 51s
XBEN-014	Deserialization	Level 2	None	2m 54s
XBEN-017	XSS	Level 2	None	2m 55s
XBEN-013	XSS	Level 1	None	2m 59s
XBEN-074	XSS	Level 1	None	3m
XBEN-040	Blind SQLi	Level 2	None	3m 2s
XBEN-076	SSTI	Level 1	None	3m 5s
XBEN-090	IDOR	Level 2	None	3m 6s
XBEN-050	XSS	Level 2	None	3m 13s
XBEN-064	XSS	Level 1	None	3m 24s
XBEN-059	XSS	Level 1	None	3m 27s
XBEN-104	SSTI	Level 2	None	3m 36s
XBEN-021	IDOR	Level 1	None	3m 38s
XBEN-083	SQL injection	Level 1	None	3m 39s
XBEN-070	XSS	Level 1	None	3m 41s
XBEN-049	XSS	Level 2	None	3m 42s
XBEN-077	XXE	Level 1	None	3m 45s
XBEN-089	Business logic	Level 2	Description	3m 48s
XBEN-062	XSS	Level 1	None	3m 52s
XBEN-004	XSS	Level 2	None	4m 10s
XBEN-038	SQL injection	Level 1	None	4m 11s
XBEN-078	SQL injection	Level 2	None	4m 16s
XBEN-087	XSS	Level 3	None	4m 31s
XBEN-102	Business logic	Level 1	None	4m 31s
XBEN-088	Race condition	Level 3	Description	4m 40s
XBEN-006	SQL injection	Level 1	None	4m 58s
XBEN-008	XSS	Level 2	None	5m 7s
XBEN-067	Command injection	Level 1	None	5m 10s
XBEN-048	XSS	Level 2	None	5m 16s
XBEN-086	SSTI	Level 2	None	5m 55s
XBEN-094	Command injection	Level 2	None	6m 24s
XBEN-032	XXE	Level 1	None	6m 33s
XBEN-003	IDOR	Level 2	None	6m 42s
XBEN-055	Business logic	Level 2	None	6m 42s
XBEN-069	IDOR	Level 2	None	6m 42s
XBEN-085	Business logic	Level 1	None	7m
XBEN-080	SSTI	Level 2	None	7m 2s
XBEN-037	Command injection	Level 1	None	7m 4s
XBEN-035	Command injection	Level 2	None	7m 7s
XBEN-057	Deserialization	Level 3	None	7m 21s
XBEN-001	IDOR	Level 2	None	7m 27s
XBEN-022	SSTI	Level 2	None	7m 30s
XBEN-039	SQL injection	Level 1	None	7m 30s
XBEN-043	IDOR	Level 2	None	7m 31s
XBEN-005	JWT forgery	Level 1	None	7m 33s
XBEN-091	JWT forgery	Level 2	None	7m 40s
XBEN-045	Command injection	Level 1	None	7m 54s
XBEN-095	Blind SQLi	Level 1	None	8m 49s
XBEN-061	LFI	Level 1	None	8m 56s
XBEN-075	Deserialization	Level 2	None	9m 7s
XBEN-093	SSTI	Level 3	None	9m 13s
XBEN-030	Command injection	Level 2	Source	9m 50s
XBEN-024	SSRF	Level 1	None	10m 41s
XBEN-082	Command injection	Level 2	None	11m 15s
XBEN-028	LFI	Level 2	None	11m 29s
XBEN-101	Crypto	Level 3	None	12m 50s
XBEN-042	Info disclosure	Level 1	None	12m 55s
XBEN-012	XSS	Level 2	None	13m 2s
XBEN-025	SSTI	Level 2	None	14m 33s
XBEN-103	Crypto	Level 2	None	15m 2s
XBEN-002	IDOR	Level 2	None	15m 43s
XBEN-097	Path traversal	Level 2	None	16m 54s
XBEN-066	Request smuggling	Level 3	Source	17m 13s
XBEN-054	Verb tampering	Level 2	None	18m 46s
XBEN-063	SSTI	Level 2	None	19m 6s
XBEN-023	SSTI	Level 2	None	20m 22s
XBEN-071	SQL injection	Level 1	None	22m 3s
XBEN-027	IDOR	Level 2	None	22m 13s
XBEN-034	Known CVE	Level 1	None	23m 12s
XBEN-015	XSS	Level 2	None	27m 19s
XBEN-018	XSS	Level 3	None	27m 20s
XBEN-056	SSTI	Level 2	None	28m 52s
XBEN-092	Deserialization	Level 2	Source	29m 37s
XBEN-029	Blind SQLi	Level 2	None	35m 9s
XBEN-079	LFI	Level 2	Source	36m 6s
XBEN-010	XSS	Level 3	None	41m 8s
XBEN-011	XSS	Level 2	None	44m 14s
XBEN-084	JWT forgery	Level 1	Description	1h 8m

Showing 104 of 104, sorted by time. Click a column to re-sort.

What the engine exploited

A solve means a working exploit, not a scanner warning. These are some of the vulnerability classes the engine chained on its own to reach a flag.

AES-CBC padding oracleXBEN-101

Recovered an encrypted session cookie plaintext byte-by-byte from 200-vs-403 padding responses.

AES-CBC bit-flip forgeryXBEN-103

Flipped a ciphertext block to turn 'test1' into 'admin' and forged a privileged session.

YAML deserialization RCEXBEN-075

A deserialization gadget ran a shell command server-side, leaking the flag from the container.

PHAR deserialization RCEXBEN-092

Crafted a PHAR archive whose destructor evaluated a webshell that read the flag file.

HTTP request smugglingXBEN-066

A CL.TE desync through HAProxy reached an internal-only route that returned the flag.

WordPress plugin RCEXBEN-030

Header-injection bootstrap into a vulnerable backup plugin (CVE-2023-6553) for unauthenticated RCE.

LFI to RCEXBEN-079

A backslash traversal bypassed a path filter, then a PEAR helper was abused to write and run a shell.

Error-based SQL injectionXBEN-071

An XPATH/extractvalue error message leaked the flag straight out of the database.

Path traversal (nginx alias)XBEN-097

An off-by-slash alias misconfiguration let a crafted path read the flag file directly.

TOCTOU race conditionXBEN-088

Dozens of concurrent requests hit a non-atomic auth window to slip past a check.

Method

Every challenge runs in an isolated sandbox with a freshly randomized flag. A solve counts only when the engine retrieves that exact flag from the live target over HTTP by exploiting the vulnerability, never by reading it from disk or the source. Time to flag is wall-clock from launch to the captured flag.