handy models

speech-to-text benchmarks for transcribe.cpp, the engine behind Handy

model detail

Where this model stands against the best model in each language at Q8_0, its speed on an AMD Ryzen 4750U laptop, and what the card says. Add a second model to compare them head to head.

model Breeze-ASR-25
AMD 4750U GPU2.0×
AMD 4750U CPU1.0×
Q8_0 size1.55 GB
frontier · AMD 4750U GPU0 of 2

verdict: Never first, but tied with the leader in 1 of 2 languages (overlapping 95% CIs). Fast enough for realtime on the AMD 4750U GPU (2.0×), not on the AMD 4750U CPU (1.0×).

license: Apache-2.0 · card languages: 2 · translatelang detecttimestamps (segment)

standing by language · 2 benchmarked

Each bar compares this model with the best other model in that language, on a log scale: left of center it is ahead, right of center behind, and the other model is named on the row. The two error rates use the legend colors: this model first, the other model second. Faded bars are ties, where the 95% confidence intervals overlap. Sorted strongest standing first. Hover a row for full details.

Breeze-ASR-25 error best other model error tie: the 95% CIs overlap

quantization · 2 usable languages at BF16, F16, Q6_K, Q5_K_M, Q4_K_M

BF16 is within noise on the one usable language: 186% of the Q8_0 file for no measurable accuracy cost. F16 is within noise on the one usable language: 186% of the Q8_0 file for no measurable accuracy cost. Q6_K is within noise on the one usable language: 78% of the Q8_0 file for no measurable accuracy cost. Q5_K_M is within noise on every one of 2 usable languages: 70% of the Q8_0 file for no measurable accuracy cost. Q4_K_M is within noise on the one usable language: 60% of the Q8_0 file for no measurable accuracy cost.

BF16 file186% of Q8_02.88 GB against 1.55 GB
median error change+0.2%over 1 usable language
measurably worse0 of 1above the Q8_0 95% confidence interval
worst usable languageMandarin Chinese (CER)8.10 → 8.12 (+0.02 pts)
F16 file186% of Q8_02.89 GB against 1.55 GB
median error change+0.1%over 1 usable language
measurably worse0 of 1above the Q8_0 95% confidence interval
worst usable languageMandarin Chinese (CER)8.10 → 8.11 (+0.01 pts)
Q6_K file78% of Q8_01.21 GB against 1.55 GB
median error change+0.2%over 1 usable language
measurably worse0 of 1above the Q8_0 95% confidence interval
worst usable languageMandarin Chinese (CER)8.10 → 8.12 (+0.02 pts)
Q5_K_M file70% of Q8_01.08 GB against 1.55 GB
median error change+0.1%over 2 usable languages
measurably worse0 of 2above the Q8_0 95% confidence interval
worst usable languageEnglish4.14 → 4.16 (+0.02 pts)
Q4_K_M file60% of Q8_00.93 GB against 1.55 GB
median error change−0.2%over 1 usable language
measurably worse0 of 1above the Q8_0 95% confidence interval
worst usable languageMandarin Chinese (CER)8.10 → 8.08 (−0.02 pts)
measurably worse than Q8_0: above its 95% confidence interval within noise typical noise of one measurement
per-language numbers (2)
English 4.143.70–4.61 ———4.16+0.02— +0.5%
Mandarin Chinese (CER) 8.107.28–8.94 8.12+0.028.11+0.018.12+0.028.07−0.038.08−0.02 +0.2%

speed · all 73 models on the AMD 4750U GPU

× realtime, log scale. Left of the realtime marker, transcribing takes longer than the recording. Hover a dot for the model.

compare with

Qwen3-ASR-1.7Bahead of it in 2 of 2 shared languagesMOSS-Transcribe-Diarizeclosest coverage, 2 languages in common