POCKET-Darwin-180B Takes #1 on Hugging Face's Official Benchmark for Protecting AI Agent Tool Calls

2026-11-19 · VIDRAFT AI Media

aisecuritybenchmarkllm

POCKET-Darwin-180B Takes #1 on Hugging Face's Official Benchmark for Protecting AI Agent Tool Calls

TL;DR

On November 19, 2026, VIDRAFT's POCKET-Darwin-180B took #1 on the Category Macro F1 leaderboard of safeagents judge bench, a Hugging Face official benchmark that tests whether a model can protect an AI agent's tool calls. The score is 0.599. Second place on the leaderboard is Cloudflare clef at 0.563. VIDRAFT took on this benchmark because it maps directly to ViKernel, its security kernel that checks an agent's action right before it runs. It is the sixteenth #1 record the Darwin family has set on Hugging Face official benchmark leaderboards.

Leaderboard: safeagents judge bench, Category Macro F1 · Model: POCKET-Darwin-180B

Top five of the Category Macro F1 leaderboard on safeagents judge bench: POCKET-Darwin-180B 0.599, Cloudflare clef 0.563, vllm-sr d3 0.531

What does safeagents judge bench measure?

A coding agent calls tools: it runs shell commands, reads files and sends requests. Hugging Face's safeagents is a local proxy that checks every tool call before it runs. Calls that its scanner finds suspicious go to a decision model, which answers allow or block. safeagents judge bench measures how well a model makes that decision, and whether it can tell what kind of misbehaviour a call is.

Item Value
Status Hugging Face official benchmark, registered October 30, 2026
Data 661 tool calls from 122 real agent trajectories over 55 tasks
What the judge sees The user's task, the call and the scanner findings
Kinds of misbehaviour Reward hacking, harness introspection, credential misuse, sandbox escape
Legitimate calls 125 calls that serve the task and must be allowed

Who is on the Category Macro F1 leaderboard?

Nineteen models are listed. The top five, read from the Hugging Face leaderboard on November 19, 2026:

Rank Model Category Macro F1
1 FINAL-Bench/POCKET-Darwin-180B-GGUF 0.599
2 Cloudflare/clef 0.563
3 vllm-sr/d3 0.531
4 PromptforceAI/niti-gate-vegaml-4b 0.497
5 vllm-sr/vela3-turbo 0.491

How was POCKET-Darwin-180B measured?

VIDRAFT ran the benchmark's official scoring code (bench/run.py) on all 661 calls of the test split, with the benchmark's own category prompt and JSON schema at temperature 0. The model is the 4-bit GGUF build of a 180B model, and it ran on a single NVIDIA DGX Spark, a desktop machine. In the same run it named the right category for 86.2 percent of the calls. On the allow-or-block question it blocked 97.6 percent of the calls that should be blocked and caught 77 of 82 misbehaving trajectories, with a verdict returned for every call.

Why did VIDRAFT take on this benchmark?

Because it maps directly to ViKernel. ViKernel is VIDRAFT's security kernel placed inside the weights of a model. It blocks a dangerous tool call right before it runs, records every decision and has a kill switch. safeagents judge bench asks a model the same question that ViKernel answers in production: should this action run, and if not, what is wrong with it?

VIDRAFT also publishes ViKernel Scanner, a free tool that grades any model from A to F in about a minute, with its source on GitHub under Apache-2.0, and runs an open AI Model Hacking Hackathon where anyone can attack the security kernel.

  • Leaderboard (Category Macro F1): https://huggingface.co/datasets/harness-hub/safeagents-judge-bench?leaderboard_task_id=category_macro_f1
  • POCKET-Darwin-180B: https://huggingface.co/FINAL-Bench/POCKET-Darwin-180B-GGUF
  • ViKernel Scanner (free): https://huggingface.co/spaces/FINAL-Bench/ViKernel-Scanner
  • Scanner source code (Apache-2.0): https://github.com/final-bench/vikernel-scanner
  • AI Model Hacking Hackathon: https://huggingface.co/spaces/FINAL-Bench/ViKernel-RedTeam
  • VIDRAFT: https://vidraft.net

FAQ

Which model is #1 on the Category Macro F1 leaderboard of safeagents judge bench?

POCKET-Darwin-180B from VIDRAFT, with 0.599, as of November 19, 2026. Cloudflare clef is second with 0.563.

What is safeagents judge bench?

A Hugging Face official benchmark of 661 suspicious tool calls taken from real AI agent trajectories. A model must decide whether to allow or block each call and name the kind of misbehaviour.

What hardware did the run use?

A single NVIDIA DGX Spark, running the 4-bit GGUF build of POCKET-Darwin-180B.

ViKernel blocks a dangerous tool call right before it runs. The benchmark tests the same judgement, so VIDRAFT took it on.