+47% Coding Ability at Q4: POCKET-27B-Pi Solves 58.3% of Coding Tasks, the Original Qwen3.8-27B 39.6%

2026-11-20 · VIDRAFT AI Media

aillmcodingopensource

+47% Coding Ability at Q4: POCKET-27B-Pi Solves 58.3% of Coding Tasks, the Original Qwen3.8-27B 39.6%

Coding ability on Terminal-Bench 2.1: original Qwen3.8-27B 39.6%, pi model at full precision 43.8%, POCKET-27B-Pi 58.3%

VIDRAFT has released POCKET-27B-Pi, the coding-specialist model of the POCKET family. It is a coding agent in one 16.5 GB file, quantized to Q4. On the same GPU, the same tasks and the same time limit, it solved 47% more coding tasks than the original Qwen3.8-27B, and 33% more than the 53.8 GB full-precision pi model it is built from.

Model: https://huggingface.co/FINAL-Bench/POCKET-27B-Pi-GGUF

The comparison

We put four builds on the same machine and gave them the same job: Terminal-Bench 2.1 under the pi coding agent, thinking off, one attempt per task, one hour per task, eight agent sessions in parallel on one GPU. The table counts the 48 tasks that all four builds completed.

Model File size Tasks solved Score
Original model: Qwen3.8-27B, original weights (POCKET-27B) 22.4 GB 19 of 48 39.6%
Pi model: Qwen3.8-27B-pi-adapted, full precision 53.8 GB 21 of 48 43.8%
Our model: POCKET-27B-Pi Q4_K_M 16.5 GB 28 of 48 58.3%
Our model: POCKET-27B-Pi Q6_K 22.1 GB 27 of 48 56.3%
  • +33% over the full-precision pi model (58.3% against 43.8%, 14.5 points), in a file 3.3 times smaller.
  • +47% over the original Qwen3.8-27B (58.3% against 39.6%, 18.7 points), at Q4 and in a smaller file.
  • Head to head on 50 shared tasks, POCKET-27B-Pi Q4_K_M solved 30 and the full-precision original solved 23.

Why the small build wins

A coding agent works against the clock. Every task has a time budget, and every turn of the agent waits for the model to answer. The 16.5 GB build answers faster on the same GPU than the 53.8 GB build, so it completes more turns, and more whole tasks, before the budget runs out. The quality of each answer holds: Q4_K_M and Q6_K solved the same number of tasks, 39 and 39, on 62 head-to-head tasks.

That is the POCKET idea applied to coding agents: make the model small enough to run fast on the hardware people actually have, and measure that it still does the job.

It finishes what it starts

Same 50 tasks POCKET-27B-Pi POCKET-27B
Tasks solved 29 19
Runs that reached the 100-turn cap 0 14
Median agent turns per task 15 36.5

POCKET-27B-Pi plans, acts and stops. It needs fewer than half the turns, which means less waiting and fewer generated tokens for every job.

Thinking-level results, without thinking

POCKET-27B-Pi carries the pi coding adapter, trained with reinforcement learning on verifier rewards, merged into its weights. The adapter's authors report what it does at full precision under the pi agent: SWE-bench Pro (hard 51) rises from 39.2% to 64.7%, and Terminal-Bench 2.1 from 39.5% to 55.8%. With thinking off it is level with the base model thinking at maximum effort on Terminal-Bench 2.1, while generating about 4.5 times fewer tokens per task.

Median per task With the pi adapter Base model with thinking
Generated tokens, SWE-bench Pro 9.4k 48.8k
Generated tokens, Terminal-Bench 2.1 3.7k 16.5k
Model calls, SWE-bench Pro 21 61

Our measurement lines up with theirs: POCKET-27B, which carries the unchanged base weights, solved 38.0% in our run, next to the 39.5% they report for the base model.

Your code stays on your machine

Source code is one of the assets a company is least willing to send outside. POCKET-27B-Pi runs inside your own PC or server, so the code never leaves. There is no per-token bill, and once the file is downloaded it works with no network at all. A single 24 GB GPU is enough.

How it is built

POCKET-27B-Pi combines the open Qwen3.8-27B model with the pi coding-agent adapter released by the Hugging Face harness team. We merged the adapter into the base weights so there is one file to load and no adapter at serving time, quantized it to PC size, and then tested every build on real agent tasks to choose the best size. Every ingredient and the result are Apache-2.0.

POCKET-27B-Pi: the coding-specialist POCKET model

The POCKET family

POCKET is VIDRAFT's line of large models packaged for personal machines: POCKET-Darwin-180B for math and science reasoning, POCKET-35B for laptops and phones, POCKET-26B for multilingual chat, and now POCKET-27B-Pi for coding.

  • POCKET-27B-Pi: https://huggingface.co/FINAL-Bench/POCKET-27B-Pi-GGUF
  • POCKET-Darwin-180B: https://huggingface.co/FINAL-Bench/POCKET-Darwin-180B-GGUF
  • POCKET-35B: https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF
  • POCKET-26B: https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF

Try it

llama-server -hf FINAL-Bench/POCKET-27B-Pi-GGUF:Q4_K_M \
  -ngl 999 -c 65536 --port 8080 --alias pocket-27b-pi \
  --temp 0.8 --top-p 1.0 --top-k 0 --min-p 0 \
  --chat-template-kwargs '{"enable_thinking":false}'

Then point the pi coding agent at it. Add this to ~/.pi/agent/models.json:

{
  "providers": {
    "local": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "dummy",
      "models": [{ "id": "pocket-27b-pi" }]
    }
  }
}
pi --provider local --model pocket-27b-pi --print "Find out why the tests in tests/test_parser.py fail and fix the bug."