CoBit paper: one bitstream diffusion framework co-generates text, speech, images, proteins
LucaAmb · x · 2026-10-15
Luca Ambrogioni and coauthors present "Multimodal Co-Generation With Continuous Bitstream Diffusion" (arXiv:2610.14349), extending continuous bitstream diffusion from language modeling into a unified multimodal framework.
Key points:
- Image-text, speech-text, and protein sequence-structure models each train from scratch but share one protocol: single input projection, transformer backbone, bitwise prediction head, and binary score-matching objective—no modality-specific calibration
- A single checkpoint supports joint generation and all conditional directions by masking parts of the bitstream
- Near-perfect conditional accuracy on MNIST-Sum without learned tokenizers
- Speech-text model (no text pretraining) does synthesis, recognition, continuation, and co-generation with 3.5% WER between generated text and transcriptions of generated speech
- One protein checkpoint handles forward folding, inverse folding, and joint generation
Paper and code are public.
Related event: Continuous Bitstream Diffusion Unifies Multimodal Co-Generation(2 posts)→
More from Multimodal
- Inside 'The Very Next Day': 2D keyframes plus a custom IC-Lora by LTX — paultrillo · 2026-10-16
- fal Worlds: designer unveils brand identity for fal's generative world-models platform — adamho · 2026-10-16
- Slopcannon picks Muse Wonder as default AI director, adds frontier model picker — basedjensen · 2026-10-16
- Developer mass-produces 200 videos in hours using Claude Code + Hyperframes — mcraddock · 2026-10-16
- Creator showcases V2 of AI-generated artwork 'Alien Rebel' — creatoroff · 2026-10-16
- Mirage Tesseract lets Claude and ChatGPT create full HDR videos and motion graphics — LinusEkenstam · 2026-10-16