Benchmarks
GenTel-Bench comparison for the local defend classifier versus other guard models; methodology and caveats.
When provider is defend, input evaluation uses Defend’s local pipeline (fine-tuned classifier plus heuristics), not a third-party LLM. The table below summarizes results reported for that path on GenTel-Bench alongside other listed models. The weights ship from Adaxer/defend on Hugging Face.
How to read this
Benchmarks reflect a specific benchmark suite and snapshot (subset of jailbreak, goal-hijacking, and prompt-leaking scenarios). Leaderboards and competitor numbers can change. High scores do not guarantee safety in your application, treat this as orientation when choosing defend versus LLM-backed providers, not as a compliance claim.
GenTel-Bench comparison
| Model | Accuracy | Precision | Recall | F1 |
|---|---|---|---|---|
| Defend (this repo) | 95.96 | 94.83 | 97.10 | 95.94 |
| GenTel-Shield | 97.45 | 98.97 | 95.98 | 97.44 |
| ProtectAI | 91.55 | 99.72 | 83.56 | 90.88 |
| Lakera AI | 85.96 | 91.27 | 79.51 | 84.11 |
| Prompt Guard | 50.59 | 50.59 | 98.96 | 66.95 |
| Deepset | 63.63 | 58.54 | 98.36 | 73.39 |
See also
- Actions and providers - when to use
defendversusclaude/openai. - Pipeline - what runs before the classifier decision.
- Security and privacy - limitations and third-party evaluation.