Methodology
Measured numbers only
Every number on this page comes from a real run of the full engine on public datasets, with date and version. If it isn't measured, it isn't published.
- Engine version
adarve-l1@2026.10.2+l2-onnx[deberta-pi-v2+horizon-pi+e5-small-cmknn4496]- Date
- 12 October 2026
| Evaluation set | threshold 0.8 | threshold 0.5 | ||
|---|---|---|---|---|
| Recall | False positives | Recall | False positives | |
| Adarve curated ES/EN80 texts · CC0-1.0 | 90% | 2.5% | 100% | 5% |
| deepset/prompt-injections (test)116 texts · Apache-2.0 | 25% | 0% | 58.3% | 0% |
| Lakera/gandalf_ignore_instructions (test)112 texts · MIT | 98.2% | — | 99.1% | — |
| jackhhao/jailbreak-classification (test)262 texts · Apache-2.0 | 86.3% | 0.8% | 91.4% | 1.6% |
| SPML Chatbot Prompt Injection (1k slice)999 texts · MIT | 36% | 0% | 80% | 1% |
How to read the table
- Recall: share of attacks scoring above the threshold. False positives: share of benign texts scoring above it.
- 0.5 is the recommended operating point: false positives stay at 5% or less on every set. 0.8 flags only near-certain attacks: 2.5% or less false positives, with lower recall.
- deepset is the hard case: subtle attacks, often in German. It is our main improvement track, and the Engine page shows the gap honestly next to the public baseline.
Honest limitations
- ProtectAI's classifier was trained on public datasets that may overlap these test sets. The attack memory uses only training splits and excludes any text present in the evaluation.
- The ES/EN set was written by the same author as the layer-1 signatures: its layer-1 recall is optimistic.
- We don't publish latencies yet: they haven't been measured on production hardware.
- No detector is immune to adaptive attacks. Adarve returns a score, not a guarantee.
Engine components
- Normalisation, bounded de-obfuscation, multilingual signatureslayer 1deterministic
- Transformer prompt-injection classifier ensemble (ONNX, CPU)layer 2sliding windows
- Multilingual embeddings + labelled attack memory (7 languages)layer 2kNN margin
Reproduce it
The evaluation tool runs the production layer 1 and layer 2 code and fuses them exactly like the API. Every published number names its dataset, date and engine version.