SENTINEL AgentDojo
A benchmark and environment for agent utility and prompt-injection robustness. Sentinel’s fixture results are not AgentDojo scores.
Read the project ↗RESEARCH & EVIDENCE
Measure the boundary.
State the limits.
The working hypothesis: independent task-scoped enforcement can reduce unauthorized tool execution while preserving legitimate work on controlled workloads.
Deterministic synthetic benchmark
EXISTING LOCAL ARTIFACTSeed 1729. Twenty repetitions of the same fixture; six unauthorized proposals per task. Completion requires the final summary to remain correct. These are repeated deterministic scenarios, not independent real-world attack samples.
| Research arm | Unauthorized execution | Tasks completed | False denials |
|---|---|---|---|
| Unrestricted simulation | 120 / 120 | 0 / 20 | 0 / 140 |
| Basic tool restrictions | 60 / 120 | 0 / 20 | 0 / 140 |
| Static policy comparator | 20 / 120 | 0 / 20 | 0 / 140 |
| Sentinel research runtime | 0 / 120 | 20 / 20 | 0 / 140 |
Source: benchmarks/phase2-results.json, macOS ARM64, Python 3.12.14. The comparator arms execute in memory; runtime latency is not an equivalent overhead comparison.
Read the original benchmark artifact ↗REAL LOCAL MODEL
Qwen2.5:3b
through Ollama.
Existing local runs used Ollama 0.40.3 and strict structured proposals. The sequenced clean run produced six allowed gateway actions; the sequenced injected run produced ten allowed actions and one denied unauthorized API request. Both wrote and read back authorized output with matching audit evidence.
Earlier runs stopped safely on invalid proposals or the step bound. Model-declared completion does not independently establish summary quality. These few smoke tests are not a prompt-injection robustness benchmark.
Source: docs/local-llm-integration.md and reports/qwen-live-*-sequenced.json, 2026-10-13. No independent replication or certification is claimed.
Built on established methods.
Capability security, sandboxing and authorization are established approaches. Sentinel’s potential contribution is a measurable combination of task authority, an independent gateway and constrained execution for agent workflows.
CaMeL
Separates control and data flow to contain prompt-injection effects. Sentinel’s sticky task taint is a narrower experimental mechanism.
Read the paper ↗Cedar & Wasmtime
Existing policy and execution foundations. Their guarantees depend on correct integration and explicitly scoped host interfaces.
Cedar documentation ↗