Task-Conditional Flow Matching for Balanced Multilingual Text Embedding Adaptation
Tirth Bhatt, Naren Kumar S, Mayank Singh
cs.CL, cs.AI
2026-08-06
Matching each task to its own objective (flow matching for translation, contrastive and margin for retrieval) plus a 3-stage curriculum lifts several embedding models 1.6 to 4.4 points to a new SOTA on 22-language Indic MTEB.
Adapting a multilingual text embedding model usually means training every task with one contrastive objective: retrieval, classification, pair classification, translation, clustering, and semantic similarity all share the same loss. The problem is that these tasks have fundamentally different optimization dynamics, so a single objective produces conflicting signals. Contrastive learning also has a long-standing false-negative problem, pushing apart sentences that are semantically close and fragmenting a representation space that should stay continuous.
The authors, from IIT Gandhinagar, focus on Indian languages. The 21 scheduled languages beyond Hindi are chronically underserved and lack high-quality multilingual embeddings. The question is how to adapt embeddings for low-resource languages without the training objectives fighting each other.
The core idea is simple: match the objective to the task. TCFM sorts data into four task families and routes each to its own loss.
For Flow Matching the model linearly interpolates an intermediate state between source and target representations and trains a small network to predict the direction of that path. The loss is the cosine distance between the predicted velocity and a teacher-guided target direction, which folds in the difference between the teacher's source and target representations.
Training follows a three-stage curriculum. Stage 1 does only translation-style cross-lingual alignment with flow and teacher preservation. Stage 2 adds classification and natural-language-inference data while replaying about 40k translation pairs to prevent forgetting. Stage 3 adds retrieval and a SimCSE-style monolingual consistency regularizer to preserve within-language representations.
The key design choice is that Flow Matching is applied only to translation. The ablation shows extending it to all tasks is slightly worse (a 1.63 gain versus 1.67). The hypothesis is that translation has clean one-to-one correspondence that Flow Matching can exploit and other tasks lack.
Evaluated on Indic MTEB v3 across 22 languages and five embedding model families, every model improves to a new state of the art.
| Model | Base | TCFM | Gain |
| Harrier-0.6B | 65.87 | 69.46 | +3.59 |
| Qwen3-Embedding-8B | 72.89 | 74.87 | +1.97 |
| Harrier-270M | 63.94 | 68.33 | +4.40 |
| EmbeddingGemma-300M | 64.34 | 66.68 | +2.34 |
| Qwen-Embedding-4B | 70.94 | 72.59 | +1.65 |
The largest single jump is on clustering, where Harrier-0.6B climbs from 29.09 to 50.12, up 21 points. The ablation confirms each component pulls weight: drop the teacher and keep only flow plus contrastive, and retrieval falls 12.48 points and STS 8.99, showing the teacher term does the heavy lifting on those tasks. The staged curriculum accumulates gains, though not smoothly. Clustering on Qwen3-8B regresses 2.6 points at Stage 2 before surging 15 points at Stage 3.
This is an architecture-free recipe for low-resource embedding adaptation that works from a 300M model up to 8B. Anyone building non-English retrieval or RAG, especially for languages where mainstream embedding models underperform, can pick it up directly.
The broader takeaway transfers beyond Indic languages: one objective cannot serve every task. Translation, retrieval, and classification ask different things of a representation space, and forcing a single contrastive loss on all of them lets them drag each other down. The finding that Flow Matching helps only translation is a concrete, falsifiable design lesson, not an Indian-language curiosity.
Evaluation covers only 22 Indian languages on Indic MTEB; generalization to other language families is untouched. The method depends on high-quality parallel translation corpora, so genuinely zero-resource languages are out of reach.
The claim that Flow Matching improves embedding-space uniformity rests on empirical evidence only; the authors concede there is no formal geometric analysis and that adding an explicit repulsive regularizer to Flow Matching goes unexplored. The cost is non-trivial: a frozen teacher model, an extra velocity-prediction network, and a three-stage curriculum.
The results are not a clean sweep. Retrieval on Qwen3-8B dips 0.28 points, and the clustering gains are volatile, regressing then surging across stages, which suggests the metric is highly sensitive to representation geometry in a way the method does not fully control.