GTE-MLM sparse embedding models now natively supported in Transformers v5

tomaarsen · x · 2026-11-20

Transformers maintainer tomaarsen announces native integration of the GTE-MLM sparse embedding model line, to be maintained across all future transformers v5. releases. The original repos require trustremotecode=True and no longer work with the latest transformers, so he built native copies (e.g. tomaarsen/gte-en-mlm-large-native, from Alibaba-NLP/gte-en-mlm-large; unchanged weights/tokenizer, requires transformers>=5.18.0).

They work out of the box via SparseEncoder("tomaarsen/gte-en-mlm-large-native") like bert-base-uncased, ready for finetuning, especially retrieval fine-tuning with sentencetransformers.SparseEncoder; three models released.

Related event: GTE-MLM Sparse Embedding Models Get Native Transformers Support(2 posts)→

Original post →

More from coding & agent

coding & agent channel →