Building a Production Greek-English Speech Recognizer
Christos Petrocheilos, Cleopatra Papadopoulou, Chris Porikis, Ioakeim Perros, Ayoub Kirouane, Themistoklis Nikolis
eess.AS, cs.AI, cs.SD
2026-09-12
A 23-run Greek-English ASR program finds a 6× step clash between noisy Greek WER and English LID; a three-model ROVER ensemble is what clears all nine gates.
Greek has about eleven million native speakers. English lands inside the same sentence. Most of the audio is a phone line or a crowded meeting room, not a studio. KIEFER's production ASR, Sophea, has to clear word-error-rate ceilings on Greek and English read speech, conversation, business meetings, and noisy rooms, plus a 95% language-identification floor and zero hallucinated text on non-speech audio. Each of the nine gates is a failure a customer would notice. Passing all nine is the ship criterion.
Seven versions of a 1.7B bilingual model never cleared them together. The paper shows this is not a data-quantity problem.
Two lines run in parallel. The compact line is a 1.7B encoder-decoder in the Qwen-Audio family. The large and turbo lines are Whisper; turbo keeps four decoder layers. Training audio goes through six filters: quality score, words-per-minute plausibility, transcription confidence, multi-teacher cross-check, forced alignment, and language parity. One representative build assembled 1,169,565 rows and finished at 1,073,294 Greek versus 1,079,148 English, a 0.55% parity gap.
Quality scoring uses UTMOS. The textbook cutoff of 3.0, built for English studio TTS, would have dropped 98.7% of scored Greek audio, including clean FLEURS Greek. Recalibrating on in-domain clean anchors moved the threshold to 1.30 and the discard rate to 10.6%. Forced-alignment cutoffs are likewise relative to each language's clean-anchor distribution.
Once a single mix could not hit all nine gates, production became a small family plus a router: calls and meetings go to a specialized checkpoint by expected language and domain, Greek-heavy traffic uses forced-language decoding, everything else uses implicit LID. Decode-time patches repeatedly beat retraining. A VAD plus two confidence thresholds brought silence hallucination under 2%. Dropping "ok" and "okay" from a phrase filter took boilerplate from 26 hits to 0, with every other metric unchanged to two decimals.
Two ensembles. One votes word-by-word with ROVER over the production bilingual model, a Greek Canary fine-tune, and a Whisper large-v3 LoRA. The other, Sophea ASR K1, lets a gradient-boosted classifier pick one of two models per clip and serves offline batch traffic only.
A controlled run held the checkpoint fixed and varied only dense Greek noisy-environment exposure. The WER ≤25.25 gate needs about 1,500 steps (26.8 to 25.21 by step 1,750). English LID ≥95 survives at most about 250 steps on the baseline mix, dropping roughly 3.5 points per 250 steps; a rebalanced mix stretches tolerance to about 1,250 steps and stalls Greek near WER 26, about 0.75 points short of its own gate. The two budgets differ by nearly 6×. The joint operating point sits about one WER point outside the measured frontier. Both curves are single runs, no repeated seeds.
The fight is acoustic neighborhood, not language volume. Adding 835 hours of clean studio English did not protect noisy English. Adding 577 hours of noisy, overlapping, meeting-style English, about a third as much audio, held accuracy over the same step count.
The compact line's best ship, C6, swapped machine Greek call labels for about 230 hours of multi-engine consensus labels and reached seven of nine gates, missing meetings and noisy rooms. C7's best noisy checkpoint hit 26.32% and still missed 25.25.
| Surface | In-house | Canary FT | Whisper FT | Three-model vote |
| Greek FLEURS | 8.99 | 8.63 | 5.23 | 3.98 |
| Greek noisy | 26.63 | 22.89 | 24.99 | 21.56 |
| English meeting LID | 99.3 | 100.0 | 74.7 | 99.7 |
| Code-switch WER | 59.74 | 42.83 | 45.40 | 23.84 |
| Non-speech hallucination | 0.0 | 1.8 | 100.0 | 1.0 |
Single models passed four to seven gates. The three-model vote plus a post-hoc silence-suppression rule reached nine of nine. Overlap-only WER fell from 53.35% to 37.87% (29% relative, bootstrap 95% CI 11.0–19.5 absolute points across 18 meetings). K1 sits at 4.26% average WER on the Open ASR Leaderboard's eight public English sets, 5.03 and 11th in the default view that adds two private sets (11 September 2026). Live Greek noisy traffic scored 25.88% (n=1,325, CI 24.44–27.19), the first single served model under the 26% gate. The large model's L4 checkpoint still leads that same set at 24.63%; K1's edge is English and far-field, not Greek.
For production ASR on a mid-resource language, this paper separates "try another mix" from "you are on a capacity frontier." Mixes slide along the frontier. An off-domain quality threshold can delete nine-tenths of real data; score known-good in-domain anchors first. Silence hallucination, code-switching, and boilerplate were each attacked with retraining and each closed by a decode-time change, which is now their default first move. Seven pre-registered negatives are in the paper: text-only correction, audio-conditioned deliberation, and reading a mel-spectrogram as an image all net-regressed.
Weights and training data stay closed. K1 has a public API.
The impossibility result is for this 1.7B scale and training setup. A larger model or an auxiliary LID loss might move the frontier; that was not tested. Meeting and noisy-environment numbers are mostly proprietary. Tables 3 and 4 are single-run point estimates; gaps under about one WER point are not treated as real. The overlap result is English only, Canary has AMI pretraining exposure, and Sortformer's overlap-localization precision peaked at 0.30 to 0.39 against a 0.85 requirement, so fusion never entered streaming production. Overlap is about 3.2% of real meeting audio, so even a perfect detector would move whole-meeting WER by under 1% relative. The Whisper member in ROVER and K1 is an earlier large-line checkpoint, not the shipped L4; swapping L4 was not measured. Internal instruments lied five times, including a 32× mis-logged loss from gradient accumulation and a six-fold loss jump from curriculum order that was misdiagnosed across three restarts.