GEM: A Generative Embedding Model Bridging Reasoning and Retrieval
Zhili Shen, Craig Macdonald
cs.CL, cs.AI, cs.IR
2026-08-13
GEM unifies reasoning and embedding in one 4B model, generating an intent analysis before encoding, reaching nDCG@10 of 29.1 on BRIGHT to beat the 8B ReasonIR, and matching larger setups with GPT-4 reasoning.
Users now hand AI a single natural-language sentence with a complex need, such as asking for the name of a Python syntax with simple examples. Retrievers still work on keywords and surface similarity, so recall collapses when a query and its relevant document share no obvious wording. Two existing fixes each fall short. One runs an upstream LLM to reason over and expand the query before retrieval, which keeps two models in play and cannot tell whether gains come from real understanding or just more lexical overlap. The other asks users to spell out relevance criteria themselves, but users rarely can. GEM puts the reasoning inside the retriever itself, so it understands intent before it embeds.
GEM is built on Qwen3-4B-Instruct and follows a generate-then-encode flow. Given a query, the model first reasons about user intent and relevance criteria under a meta-instruction, producing a reasoning passage; an embedding token is then appended to that prompt-response pair to encode the enriched context into a vector for retrieval. Documents are encoded only, never generated.
Keeping generation and embedding alive together is the hard part. Once an LLM is trained as a bi-encoder, its generative ability tends to degrade through catastrophic forgetting or bidirectional attention. GEM holds both with joint training: a causal language modeling loss (lambda gen 0.1) preserves generation, while a contrastive InfoNCE loss (lambda emb 1.0) learns the embedding, at temperature 0.02.
Data alignment is the other piece of work. Reasoning can drift, so for each query the authors sample 8 reasoning passages, filter out those that contradict the positive document with a relevance classifier, and keep only the consistent ones. Conditioned on the surviving reasoning, Llama-3.1-8B then generates positives and hard negatives that share a topic but subtly contradict the intent, which stops the model from coasting on surface matching.
On the reasoning-intensive BRIGHT benchmark (single-model setting):
| Model | Avg nDCG@10 |
| BM25 | 14.8 |
| Promptriever (7B) | 20.0 |
| Qwen3-4B-Instruct (embedding only) | 21.4 |
| ReasonIR-8B | 24.4 |
| GEM (4B) | 29.1 |
On the same 4B backbone, the embedding-only variant scores 21.4, and adding the model's own reasoning lifts it to 29.1, with theorem-based tasks climbing from 19.8 to 32.0. Paired with GPT-4-generated reasoning (the pipeline setting), GEM reaches 30.0, on par with the 8B ReasonIR (29.9) and Rank1-32B (29.4) at a quarter of the size.
On instruction-following retrieval (FollowIR), GEM scores p-MRR of +11.7, matching the much larger Promptriever (+11.2) and pulling well ahead of the same backbone used for embedding only (+6.8). It also scales test-time compute like a normal instruction model: nDCG@10 peaks at 30.1 when prompted to generate about 1024 words, then saturates.
GEM shows the generative ability already inside a retrieval model is not wasted; tying it to the embedding objective helps both sides. For anyone building RAG, that removes the need to maintain a separate reasoning-plus-retrieval pipeline, since one 4B model matches an 8B-plus-GPT-4 stack on hard queries. The test-time scaling knob also lets you spend more on a hard question when you need to.
Compute limits meant the authors could not replicate the work at 7B or larger, so all backbone experiments stay at 4B. Hallucinations during data generation are mitigated by filtering, not eliminated. On the Pony programming-language task, GEM actually trails its embedding-only variant, which the authors attribute to Pony being absent from training data. The most revealing result is RQ4: pasting GEM's own reasoning onto other models for query expansion hurts their p-MRR, so the gain comes from joint training rather than from the reasoning text itself.