There and Back Again: Bidirectional Diffusion Bridges for Multimodality Translation
Gabe Guo, Elon Litman, Thanawat Sornwanee, Jose Blanchet, Stefano Ermon
cs.LG
2026-08-28
BIT interpolates text embeddings into images via a reversible SDE bridge. A 1B-param model hits FID 2.54 on GPIC; noise-to-data diffusion is close, a bare flow baseline hits 279.
Mainstream text-to-image models walk from noise to data. The image starts as Gaussian noise; text enters as sidecar conditioning, usually via cross-attention. Intermediate states share no mutual information with the prompt, and the finished trajectory cannot be inverted into a caption. Image-to-text is a separate autoregressive VLM.
Editing methods such as SDEdit inherit the same hole. The farther the state walks toward noise, the more of the reference image disappears. Without the original condition, semantically related variants are hard to recover.
BIT wants four things: a joint distribution over modalities (one prompt, many images, so a deterministic map is the wrong object), a path whose intermediate states still carry the source, reversibility at any time, and one framework for both T2I and I2T.
BIT is Bidirectional Image-Text Diffusion Bridges, from Stanford. Generation does not start from noise. It interpolates between a text representation and an image representation.
The base process is scaled Brownian motion, independent of the data. Girsanov's theorem and Doob's h-transform pin the path onto real image-caption pairs. A forward SDE walks text to image. The reverse SDE is that same process run backward, image to text. Both drifts are neural nets, trained with a weighted L2 loss against a closed-form score of the base process. The terminal point is a continuous extension of the path; the SDE used in simulation lives on finite horizons before time 1.
Text is discrete, and an SDE wants a continuous state. Each token is sent independently through Qwen3-Embedding-8B into a frozen lookup table, with no attention across the caption. Contextual hidden states would map one token onto too many vectors and make decoding messy. Matryoshka truncation cuts each token to 64 dimensions. Captions are padded or truncated to 64 tokens, so 64×64=4096 matches the Stable Diffusion VAE latent of a 256×256 image (4×32×32). A small MLP maps embeddings back to tokens with cross-entropy; accuracy stays above 99% even on noisy embeddings.
Sampling can mix the endpoint-conditioned drift with the unconditioned one, in the style of classifier-free guidance. The paper calls this an empirical heuristic and does not claim it samples from a named guided law.
Theorem 3 shows that near the source endpoint, a data-to-data bridge keeps more mutual information about the target than an unconditioned noise-to-data bridge. The comparison applies when the state itself has to remember the source. A standard conditional T2I sampler that keeps the prompt as side information throughout is outside the theorem.
A deterministic flow-matching ablation sits in the same table. An ODE has no noise, so it cannot represent a multi-peaked conditional. FlowTok and CrossFlow restore stochasticity with variational encoders; BIT gets it from the bridge SDE.
The controlled comparison uses about 100 million GPIC image-caption pairs, images as SD VAE latents, text as Qwen embeddings. Volatility schedules are matched. Table 1 is ω=0, 500 SDE steps, 50k T2I samples and 10k I2T samples:
| Method | FID↓ | CLIP T2I↑ | Gen PPL↓ | CLIP I2T↑ |
| BIT | 6.23 | 27.10 | 123.4 | 26.99 |
| Noise-to-data diffusion | 5.73 | 27.13 | 178.2 | 26.42 |
| Flow (no stochasticity) | 279.13 | 20.13 | 169.2 | 27.25 |
On T2I, BIT sits next to noise-to-data diffusion. The noise baseline is slightly better on FID and CLIP, reported as significant at 95%. On I2T, BIT's generative perplexity is clearly better (123.4 vs 178.2). The bare flow's FID is 279.13 and CLIP T2I falls to 20.13, which is what a deterministic map should do on a one-to-many task. That same flow posts the best I2T CLIP of the three (27.25); captioning on GPIC may be fairly low-entropy given the image, but its perplexity is the worst, so syntax is not under control.
A 1.05B DiT-XL/2 (1,054,621,840 parameters, 122,500 steps, batch 1792, one net for both directions) plus CFG:
| Method | FID↓ | CLIP T2I↑ | Gen PPL↓ | CLIP I2T↑ |
| BIT ω=0.0 | 4.39 | 27.75 | 78.1 | 29.4 |
| BIT ω=0.5 | 2.54 | 28.80 | 61.1 | 29.5 |
| Qwen-3-VL-4B-Instruct | n/a | n/a | 37.7 | 31.9 |
| Stable Diffusion 1.5 | 9.92 | 33.15 | n/a | n/a |
At ω=0.5, FID is 2.54 versus Stable Diffusion 1.5's 9.92 on the same GPIC holdout, while CLIP 28.80 still trails SD 1.5's 33.15. I2T perplexity 61.1 and CLIP 29.5 still trail Qwen-3-VL-4B-Instruct (37.7 / 31.9), the model that wrote the GPIC captions. Those external numbers are anchors, not matched ablations. Even ω=0 already beats every entry in Table 1, so scale and engineering still move this bridge.
On round-trip variation, BIT's fidelity falls more slowly at high corruption. Noise-to-data is more diverse, because its noise endpoint wipes the source state.
On the LARRY lineage-tracing set, the task is to map day-2 progenitors to day-6 descendants. The cosine-volatility shared-network bridge has the best average rank (2.67) and the best fate kNN (0.445). Endpoint regression has the lowest pointwise MSE (forward 1.195) and the worst structure metrics (fate kNN 0.336, clone kNN 0.047, MMD 0.0847). Rectified flow looks fine on MMD and cycle error, then posts forward MSE 2.444: the paired state-fate map is not preserved. Ranks are from a single run, with no uncertainty estimates.
This is not a new SoTA text-to-image model. In the controlled table, noise-to-data still wins FID by a little. The change is the starting point: T2I does not have to begin at Gaussian noise. A text representation can be one end of an SDE.
Two practical consequences follow. One network does T2I and I2T, without a separate captioner. Intermediate states keep source information, which is the setting SDEdit-style editors actually need. The same bridge also runs on cell-fate translation, so the construction is not glued to pixels and tokens.
It is not a drop-in replacement for Flux or Stable Diffusion. CLIP trails SD 1.5, captions trail a dedicated VLM, and resolution stays at 256. Read it as a controlled test of whether the Gaussian source is special. In these experiments, it is not.
Theorem 3 is local near the source endpoint, and it does not rank against a sampler that keeps the prompt as side information. The CFG mix has no analytic guarantee. Finite-sample networks do not inherit the path-law claims of the population oracle. Those caveats are in the theorems.
The experiments have soft spots. Table 1 never lets BIT beat noise-to-data on T2I FID, only match it. The 1B external comparisons are not like-for-like. I2T still trails Qwen-3-VL-4B. Text is clipped to 64 tokens at 64 dimensions, so long captions get cut. Independent token embeddings throw away context and leave syntax to the bridge, which is a plausible reason perplexity still lags a dedicated VLM. LARRY ranks have no error bars. FlowTok and CrossFlow, which add variational encoders on top of data-to-data transport, are discussed but not compared head-to-head; the ablation only knocks out a bare ODE.
The paper flags two open directions: text-image-audio translation, and a systematic study of how endpoint geometry changes bridge quality. Neither is in this work.