Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads

2026-11-01 · VIDRAFT AI Media

aillmedge-aigguf

TL;DR

VIDRAFT POCKET is a family of GGUF models that run large language models on hardware people already own: a PC with no GPU, a laptop, and (with the phone builds) an iPhone or Android device. It works with stock llama.cpp, Ollama, LM Studio and PocketPal. No fork, no CUDA, no cloud.

As measured on the Hugging Face public API on 1 November 2026, the three flagship POCKET repositories were downloaded 1,113,164 times in the last 30 days and 3,154,913 times in total.

POCKET-Darwin-180B was published less than 30 days ago, so its 30-day and all-time counts are the same.

What is VIDRAFT POCKET?

POCKET is our answer to a simple question: how large a model can a normal person run, privately, on the device in front of them?

Most "small" models still assume a datacenter GPU somewhere. POCKET takes the opposite starting point. Each repository ships ready-to-run GGUF files, quantized so that the model fits in the RAM of a consumer machine, and tested on the tools developers already use.

Three properties define the family:

1. Stock runtimes only. Every file loads in upstream llama.cpp and the apps built on it. We do not ask you to install a custom fork.
2. Sparse Mixture-of-Experts models. Only a small part of the network is active for each token, which is why a 35B or 180B model can generate at usable speed on a CPU.
3. Measured, not estimated. Every speed and quality number on the model cards comes from a run we did ourselves, on named hardware. Where we have not measured something (for example iPhone throughput for every build), the card says so.

Which POCKET model should I download?

A quick rule: start with POCKET-35B Q2_K on a PC, POCKET-26B if your app does not yet support the newest architectures, and POCKET-Darwin-180B if you want the strongest model and have the disk space.

How fast is a 35B model on a CPU?

We compare against Bonsai-27B because it is the closest model in size that targets the same "large model, small machine" niche. Both run in the same stock llama.cpp build on the same machine.

On a MacBook M3 Pro with 18 GB (measured):

On a 16-thread Xeon CPU, POCKET-35B IQ1_M generates at 27.0 tok/s against 10.1 for Bonsai-27B (2.69x). On an H100, generation is 197 against 89 tok/s. We also publish where we lose: on the H100, Bonsai is faster at prompt processing (1816 against 753 tok/s). On HellaSwag (400 questions) the two models tie within the confidence interval (61.0% against 60.0%).

The reason POCKET wins generation is the Mixture-of-Experts design. A dense 27B model has to read every weight for every token. A sparse 35B model reads only the active experts, so memory bandwidth, the real limit on a CPU, goes much further.

What does quantization cost in quality?

We measure reasoning quality on GPQA Diamond (198 questions, greedy decoding):

POCKET-26B keeps almost all of its base model's GPQA score at 2-bit because our mixed-precision quantization keeps the most quality-critical weights at higher precision. For POCKET-Darwin-180B, the 4-bit build matches the BF16 model on our quality check (87.65%).

How do I run POCKET with llama.cpp?

Any recent llama.cpp from Homebrew, winget or apt works. No fork is needed.

# download one file from Hugging Face
huggingface-cli download FINAL-Bench/POCKET-35B-GGUF POCKET-35B-Q2_K.gguf --local-dir .

# chat on the CPU only (-ngl 0), use your physical core count for -t
llama-cli -m POCKET-35B-Q2_K.gguf -p "Explain mixture-of-experts in two sentences." -ngl 0 -t 8

# reproduce our CPU speed numbers
llama-bench -m POCKET-35B-IQ1_M.gguf -p 128 -n 64 -ngl 0 -t 16

Two practical notes from our own runs:

• Set -t to the number of physical cores, not logical threads. Passing every hardware thread can slow generation down sharply.
• On an 18 GB Mac, run Q2_K on the CPU (-ngl 0). We measured 19.5 tok/s for POCKET-35B Q2_K on the M3 Pro CPU.

If you prefer an app, the same GGUF files load in Ollama, LM Studio and PocketPal.

Why are developers downloading POCKET?

We read the download curve as a signal about what people want from local AI:

• Privacy by default. The model runs where the data is. Nothing leaves the device.
• Offline by design. A laptop on a plane, a factory floor, an air-gapped office: none of them need a network.
• No new hardware. The target is the machine you already have, not a GPU you have to buy.

The ecosystem around the family also grows on its own. Independent quantizers in the community publish their own conversions of our Darwin models, and the POCKET collection keeps adding builds for new devices.

How did we measure the download numbers?

All counts come from the Hugging Face public API (/api/models with expand=downloads for the last 30 days and downloadsAllTime for the total), read on 1 November 2026. Anyone can repeat the query. The full measurement page, including the method and a same-method comparison of models published in 2026, is here: vidraft.net/hf-usage.html.

FAQ

Can I run a 35B model without a GPU? Yes. POCKET-35B Q2_K (13 GB) runs on a CPU-only PC with stock llama.cpp. On a MacBook M3 Pro CPU we measured 19.5 tok/s.

Does POCKET need a custom llama.cpp fork? No. Every POCKET file loads in upstream llama.cpp, Ollama, LM Studio and PocketPal.

Can a 180B model really run on a laptop? POCKET-Darwin-180B (4-bit, 111 GB) runs on a laptop with 8 GB of VRAM and 32 GB of RAM at a measured 4.17 tok/s, streaming weights from the SSD. A 128 GB RAM mini-PC runs it without a GPU.

Which POCKET model is best for Korean? POCKET-26B is tuned for on-device Korean, and POCKET-KR-MLX / POCKET-KR-GGUF are the Korean phone builds.

How many times has POCKET been downloaded? 1,113,164 times in the last 30 days and 3,154,913 times in total across the three flagship repositories (Hugging Face, 1 November 2026).

Where can I try it without installing anything? The POCKET collection lists every build, and POCKET Image Studio runs in the browser.

Follow VIDRAFT: vidraft.net · Hugging Face · X @VIDRAFT_ai · Threads @vidraft_lab · LinkedIn