handy models

speech-to-text benchmarks for transcribe.cpp, the engine behind Handy

best model by language

Benchmarked on the FLEURS test split at Q8_0. Error is WER — lower is better. Speed is × realtime on Handy target hardware.

accuracy vs speed

Ukrainian · 8 models · left is slower, down is better · log speed axis

AMD Ryzen 4750U GPU (Vulkan)
pareto frontier (no model is both faster and more accurate) other models
1 whisper-large-v3most accurate 6.285.74–6.83 2.1× 0.6× 1.55 GB Apache-2.0
2 parakeet-tdt-0.6b-v3overall pickfast pick 6.846.25–7.46 12.5× 7.5× 0.72 GB CC-BY-4.0
3 whisper-large-v3-turbo 7.316.72–7.87 3.4× 0.8× 0.83 GB Apache-2.0
4 parakeet-primeline 8.117.55–8.70 12.5× 7.5× 0.72 GB CC-BY-4.0
5 canary-1b-v2! 10.589.87–11.27 13.2× 6.7× 1.10 GB CC-BY-4.0
6 whisper-medium 11.5910.77–12.39 4.3× 1.1× 0.77 GB Apache-2.0
7 nemotron-3.5-asr-streaming-0.6b 14.8814.02–15.72 14.5× 7.5× 0.70 GB OpenMDW-1.1
8 whisper-small 20.4219.43–21.41 12.1× 3.4× 0.25 GB Apache-2.0