How I reached rank #231 in OpenAI's AI agent security competition

The competition

AI Agent Security Multi Step Tool Attacks asked participants to develop attack algorithms that identify reproducible multistep failures in tool using AI agents.

Source code: the GitHub repository contains the slim v65 and v170 artifacts and local uv setup instructions.

This competition ended with 12,230 registrations and 4,833 participants on 4,251 teams. There were 73,774 submissions from 111 countries.

Modern AI systems increasingly operate through tools. They can search the web, inspect files, write data, send emails, make HTTP requests, and run commands. That makes failures more consequential: an instruction hidden in content may influence a later tool call and cause an agent to leak data, modify files, misuse authority, or act without the user’s intent.

Most AI safety evaluations still focus on single prompts or isolated jailbreaks. Those tests are useful, but they do not fully capture the risks of tool using agents. Realistic failures often emerge across several steps and are hard to compare without a shared benchmark.

The competition turned that problem into a deterministic offline benchmark. Participants submitted an attack.py search algorithm through the SDK. It explored a sandboxed tool using agent and returned replayable candidates. The evaluator replayed those candidates in a fresh environment, recomputed the trace, and scored the resulting security predicates.

The benchmark focused on four unsafe action patterns:

• Exfiltration: causing an agent to leak sensitive information through an external channel.
• Untrusted to action: causing content from an untrusted source to trigger a dangerous action.
• Destructive write: causing an agent to modify, overwrite, or delete protected data.
• Confused deputy behavior: causing an agent to misuse its authority on behalf of an attacker or unintended context.

Teams could use prompt search, fuzzing, heuristic search, evolutionary algorithms, state space exploration, trace guided mutation, novelty search, Go Explore style archive methods, or hybrid systems. The common requirement was to find high impact failures, produce diverse replayable paths, and use the evaluation budget efficiently. Because every submission ran against fixture backed tools in an offline sandbox, the benchmark was safe and reproducible.

The result I optimized for and the result I missed

Over a month of submissions, I pushed my public score to 93.465, which finished at public rank #231. It was the result of a long, careful progression: valid candidate generation, live validation, replay budget sizing, and finally separate throughput work for the two target models.

The uncomfortable lesson came from the released private results. An earlier, structurally different v65 strategy had an observed private score of 14.310. That score would have landed at #227 in the released 4,187 row private leaderboard: 226 teams were higher, and none tied it. That is top 5.42% of the released board. It is inside Kaggle’s standard bronze band for competitions with more than 1,000 teams. Kaggle’s progression rules describe that band as the top 10%.

This is a counterfactual, not a claim that history can be rerun. The eligible alternative was the original v65 submission from before the deadline. Submission ref 56028258 later confirmed the same hosted script version (341256615) and the same private score. The point is simpler: the evidence for a distinct hedge already existed, and I treated final selection too much like a public score sorting problem.

The 90 point diagnostic

A public score near 90 was not simply a good score. It was a clue. The scoring structure and the two model evaluations suggested that one model was producing almost the full candidate portfolio while the other contributed little. The path beyond 90 therefore required model balance, not just more variants of the same strategy.

That interpretation changed the direction of the final public experiments. I preserved the proven v135 path for GPT OSS and built a separate live only candidate flow for Gemma. v166 reached 91.035 and v170 reached 93.465. This was strong evidence about public model throughput, but it was not evidence that the same family would transfer to the private guardrail.

One month of experiments

The work moved through five phases:

• Validity: From July 25 to 26, I learned to produce the required artifact and candidates that survived replay. The score moved from 2.700 to 16.200.
• Measurement: From July 27 to 30, live validation and replay budget sizing raised the best result from 46.295 to the v30 baseline at 88.020.
• Public optimization: From August 2 to 24, calibration, reliability gates, and bounded source action overlays produced v91 at 88.650, v107 at 89.100, and v135 at 89.590.
• Model balance: On August 30 and 31, preserving the proven GPT path and improving the other model’s live only flow produced v166 at 91.035 and v170 at 93.465.
• Selection: I chose the two public leaders, even though v65 had already shown evidence that a structurally different family transferred better to private evaluation.

The failed experiments were part of the journey too. Larger generated suffixes, broad prompt banks, email branches, multiple post candidates, source expansion, branch reordering, and shorter or Harmony heavy variants were interesting attack ideas, but they reduced replay reliability or consumed too much budget.

What moved public score, and what did not

The public gains were consistent with a throughput story. Live validation prevented unproven candidates from consuming replay capacity. Conservative replay accounting kept the portfolio within budget. Deterministic ordering kept the strongest known candidates near the front. Once a reliable path existed, preserving it was more valuable than repeatedly rewriting it.

This is a measured interpretation rather than a claim that every score increase had one cause. Some small changes were within normal evaluator variation. The clearer signal came from the large regressions and the eventual jump after separating the model paths.

The failed experiments were equally useful. Static fill, broad prompt banks, generated candidates, multiple post portfolios, and broad source expansion all cost replay reliability or throughput. More complicated candidate families could be interesting from a security perspective while still being the wrong choice for this evaluator’s budget.

“None” means the row is included for its public progression rather than as private transfer evidence. The evidence below documents the final selection alongside the earlier v65 signal.

---

Private transfer and final selection discipline

I chose v170 and v166 because they were the two public leaders. I should have used one public leader and one structurally different hedge, such as v65. The v65 result did not reveal how the private guardrail worked, but it was enough evidence that the two mechanisms transferred differently under hidden evaluation.

I optimized the public leaderboard as if it were the objective. It was only one observation about an unknown private evaluation. The mistake was not choosing a weak strategy. It was choosing two strategies from the same public family.

What I learned about tool using agents

The first lesson is that the unit of evaluation is not a prompt. It is a replayable action trajectory. A candidate matters only when the model generates the intended tool behavior, the guardrail permits it, the trace satisfies a predicate, and the result survives fresh replay.

The second lesson is that throughput is part of the security measurement. Faster generation can produce more candidates, but only reliable candidates create score. Shorter prompts, compact destinations, and live validation help only when they preserve the complete replay trace. Raw candidate count is not enough.

The third lesson is that model behavior must be measured separately. GPT OSS and Gemma did not respond identically to the same candidate families. A strategy that looks saturated on one model may contribute almost nothing on the other. The two model hint was therefore not just about finding more attacks; it was about avoiding model imbalance.

The fourth lesson is that public score is a proxy, not the objective. The public leaderboard rewarded the exfiltration throughput family, while the private results favored a structurally different confused deputy family. Final selection should therefore be treated as portfolio construction: one entry can maximize the visible metric, while another protects against a different guardrail or scoring regime.

The research literature helped me interpret these results. AgentDojo and InjecAgent framed indirect prompt injection as a multi step tool use problem. CaMeL emphasized provenance and control flow rather than surface keyword filtering. Crescendo and newer adaptive adversary work reinforced the importance of gradual, stateful interaction. ASB and MCPTox broadened the picture to memory and tool poisoning, although those areas were largely outside this benchmark’s fixed tool registry.

References

These references shaped how I interpreted the experiments. They informed the questions I asked about indirect injection, adaptive interaction, provenance, and tool or memory poisoning, but I did not treat every research direction as directly applicable to this fixed benchmark.

• AgentDojo and InjecAgent cover indirect prompt injection and tool use.
• CaMeL focuses on provenance, control flow, and capability boundaries.
• Crescendo studies gradual escalation across turns.
• ASB: Agent Security Bench and MCPTox broaden the view to memory and tool poisoning.
• The OWASP AI Agent Security Cheat Sheet connects those research themes to practical agent security risks.

Methodology and scope

The numbers in this note come from the downloaded final public leaderboard, final private leaderboard, and submission history. The counterfactual rank is direct: the released private board had 226 scores greater than 14.310, no equal score, and rank 227 scored 14.280. On a 4,187 row board, the standard top 10% boundary is rank 418; the top 5% boundary is rank 209.

The notebook and source artifacts were inspected for reproducibility, but no candidate prompts, payloads, or tool call syntax are reproduced inline in this note. All experiments used only the official competition SDK, fixture data, and hosted evaluator.

The accompanying reproducibility repository preserves the two slim source files and the evidence used for this postmortem.

I did not leave the competition with the highest private score I could have achieved. I did leave with a better understanding of how agent security failures emerge through trajectories, model behavior, tool permissions, latency, and evaluation design. The benchmark rewarded me for learning how to move faster.