Pretraining From Scratch on an Inference-Only NPU: VKORE-46M-NPU
aimachinelearningnpullm
TL;DR
VIDRAFT pretrained a small language model from scratch on an inference-only NPU, running real backpropagation on a chip the industry considers inference-only. The result, VKORE-46M-NPU, is published on Hugging Face with weights, a model card, and comparison data. Trained under identical conditions on a GPU, the loss trajectories correlate at 0.99.
Why training on an inference-only NPU is hard
Inference-only NPUs are built for the forward path: static graphs, low precision, weights as constants. Training needs the opposite, a backward graph with saved activations, optimizer state, numerical stability for small gradients, and multi-chip gradient synchronization. Those paths usually do not exist in an inference stack. That is why "NPUs cannot do backprop, so they cannot train" became the default view. This work runs backprop directly on such a chip and trains a model from scratch, with no pretrained weights.
The evidence: same model, two kinds of hardware
The most convincing part is the methodology. The same model was trained from scratch under identical initialization, data, and schedule on a 32-chip NPU and on a GPU.
- The two loss trajectories correlate at 0.99.
- Throughput (tokens per second) converged to a comparable level.
A single falling-loss curve proves nothing on its own. If low-precision kernels mangle gradients, the curve drifts away from a GPU run. A 0.99 correlation with the GPU reference under identical conditions is strong evidence that backprop and optimizer updates are doing numerically the same work as the reference.
It worked on an older chip
This was done on a first-generation inference-only NPU released in 2025 (32 chips), not the newest silicon. If older hardware can do it, next-generation NPUs leave clear headroom.
One NPU now covers the whole loop
Pretraining from scratch, LoRA fine-tuning, inference acceleration (confirmed on general open models too), and quantized-model execution and speedup. VIDRAFT is consolidating this into an NPU software layer called VKORE. Just as Nvidia built dominance through the CUDA software ecosystem rather than the GPU alone, VKORE aims to be that foundation for NPUs, a common acceleration layer that lowers dependence on a specific GPU and its software ecosystem. The company describes it as "CUDA for NPUs."
Why it matters
Top GPU servers are expensive, back-ordered, and power-hungry. A cheaper, low-power NPU that can also train lets a company grow its own model on-site, installed like a desktop in the office, without waiting on GPUs and without sending sensitive data outside.
The released model is trained on a public dataset and generates short English stories. The core training technique is proprietary and patent-pending, with details to follow in a paper; the core training code is kept private for IP protection.
Where to look
Model card: https://huggingface.co/FINAL-Bench/VKORE-46M-NPU