NVIDIA Nemotron rerankers
NVIDIA offers four rerankers for commercial use, three of them as open weights. The newest, Nemotron-3.5-Rerank-8B (20 October 2026), scores text, page images or both with one 8.38B-parameter model and a 32,768-token context. The older llama-nemotron family is much smaller: a 1.7B vision-language reranker and 1B and 500M text rerankers, all three served as NVIDIA NIM containers.
The four models
| Model | Size | Input | Context | Licence |
|---|---|---|---|---|
nvidia/Nemotron-3.5-Rerank-8B-BF16 | 8.38B, on Ministral-3-8B | Text, page image, or both | 32,768 tokens | OpenMDW-1.1 |
nvidia/llama-nemotron-rerank-vl-1b-v2 | About 1.7B, on Llama 3.2 1B with a SigLIP 2 vision encoder | Text, page image, or both | 8,192 tokens in NIM | NVIDIA Open Model License |
nvidia/llama-nemotron-rerank-1b-v2 | 1B, on Llama 3.2 1B | Text, 26 languages evaluated | 8,192 tokens | OpenMDW-1.1 |
nvidia/llama-nemotron-rerank-500m-v2 | 500M | Text | 8,192 tokens in NIM | NIM only; no Hugging Face card |
All four are cross-encoders that return a raw logit per query–document pair: higher means more relevant, and it isn’t a probability. The OpenMDW licence allows commercial use; the vision-language model’s NVIDIA Open Model License does too, with the Llama 3.2 licence on top. The 500M has no Hugging Face card; it is available only as a NIM. NVIDIA also sells the 8B through Baseten, DeepInfra, FriendliAI and OpenRouter, and since NIM release 2.4 as a NIM container, nemotron-rerank. That NIM caps each query–passage pair at 10,240 tokens by default; set NIM_PIPELINE_MAX_SEQ_LEN to allow the model’s full 32,768. The three llama-nemotron models stay on NIM release 2.3.
Scores, and whose they are
Nemotron-3.5-Rerank-8B
| Model | ViDoRe V3, page image | ViDoRe V3, text | ViDoRe V3, image + text | MIRACL-VISION |
|---|---|---|---|---|
| First stage only (Nemotron-3.5-Embed-1B) | 58.02 | 57.80 | 58.82 | 68.93 |
| Qwen3-VL-Reranker-8B | 66.02 | 64.88 | 66.03 | 75.06 |
| Nemotron-3.5-Rerank-8B | 67.22 | 66.89 | 67.56 | 78.69 |
NVIDIA’s own run, from the model card: NDCG@10, with both rerankers reordering the top 50 candidates from NVIDIA’s Nemotron-3.5-Embed-1B. MIRACL-VISION is page images in 18 languages. NVIDIA’s blog also reports the new ViDoRe v3.1 reranking leaderboard, a different setup, where the 8B scores 60.49 on image-and-text inputs against Qwen3-VL-Reranker-8B’s 59.17.
The lead over Qwen3-VL-Reranker-8B is one to two points on ViDoRe V3 and nearly four on the multilingual MIRACL-VISION. Put the Qwen figure next to the others before reading much into it: Qwen’s own card gives the same model 66.7* on ViDoRe V3, and LightOn’s run 64.23*, each with a different first stage and settings (see the Qwen3-VL-Reranker page). What the table shows clearly is the gain over the first stage: about nine points on ViDoRe V3 and ten on MIRACL-VISION.
llama-nemotron-rerank-vl-1b-v2
| Model | Text | Page image | Image + text |
|---|---|---|---|
| First stage only (llama-nemotron-embed-vl-1b-v2) | 71.04% | 71.20% | 73.24% |
| jina-reranker-m0 | 69.31% | 78.33% | — |
| llama-nemotron-rerank-vl-1b-v2 | 76.12% | 76.12% | 77.64% |
NVIDIA’s own run, from the model card: average Recall@5 over ViDoRe V1, V2 and V3 and two NVIDIA datasets (DigitalCorpora-10k and Earnings V2), reranking the candidates from llama-nemotron-embed-vl-1b-v2. NVIDIA ran jina-reranker-m0 on text and images only, as it doesn’t take both together.
On page images alone, jina-reranker-m0 scores higher, but its weights are non-commercial; the NVIDIA model’s advantage is the licence and the combined image-and-text input.
llama-nemotron-rerank-1b-v2
NVIDIA report this one only as part of their own embed-and-rerank pipeline, with Recall@5 rather than NDCG. Reranking llama-nemotron-embed-1b-v2’s candidates lifts the average over NQ, HotpotQA, FiQA and TechQA from 68.60% to 73.64%. The same pipeline reaches 65.80% on NVIDIA’s subsampled multilingual MIRACL and 86.83% on cross-lingual MLQA. None of these can be compared with BEIR NDCG@10 figures, so the models table shows none.
Quick start
Nemotron-3.5-Rerank-8B with sentence-transformers
pip install "sentence-transformers[image]" "transformers==5.20.0"
from sentence_transformers import CrossEncoder
model = CrossEncoder(
"nvidia/Nemotron-3.5-Rerank-8B-BF16",
device="cuda",
model_kwargs={"dtype": "bfloat16", "attn_implementation": "flash_attention_2"},
)
query = "What is NVIDIA NeMo Framework?"
documents = [
{"image": "https://example.com/page_3.png", "text": "NeMo Framework offers ..."},
"NeMo Framework offers comprehensive functionalities for developing LLMs.",
{"image": "https://example.com/page_3.png"},
]
rankings = model.rank(query, documents, batch_size=1)
A document can be text, one page image, or both. The example needs Flash Attention 2 installed. NVIDIA’s card tests it with Transformers 5.20.0 and warns that Flash Attention 2 can give wrong results on image inputs in 5.17 to 5.19. It needs a CUDA GPU; NVIDIA list A100 and H100 80GB and GB200.
Serving the 8B with vLLM
The card serves it with vllm serve --runner pooling and the chat template that ships with the checkpoint, and scores each pair through the /classify endpoint. Don’t use /score or /rerank: NVIDIA note they can mishandle the model’s image placeholders. Each document takes at most one image.
The llama-nemotron models through NIM
Both open llama-nemotron models load from Hugging Face with transformers, and the vision-language one also with sentence-transformers’ CrossEncoder. For production, NVIDIA package them, and the 500M, as NIM containers (release 2.3), supported on A100, H100, B200, RTX PRO 6000 Blackwell and DGX Spark. The 1B is also callable as a hosted API on build.nvidia.com.
When to pick NVIDIA
- Yes: you rerank PDF pages, slides or scans, possibly with their extracted text, and need a commercial licence: Nemotron-3.5-Rerank-8B if you have the GPU, llama-nemotron-rerank-vl-1b-v2 if you don’t.
- Yes: you already run NVIDIA NIM, and want a reranker that drops into the same stack.
- Yes: multilingual text with a small model: the 1B text reranker covers 26 evaluated languages at 8,192 tokens.
- Maybe not: CPU serving, or a comparison against BEIR scores. Look at bge-reranker or the GTE rerankers.
Pros and cons
Pros
- Open weights licensed for commercial use
- Text, page images, or both, in one model
- The 8B leads NVIDIA’s ViDoRe V3 and MIRACL-VISION comparison
- NIM containers and hosted APIs for production
Cons
- Published scores are NVIDIA’s own, mostly on its own pipelines
- The 8B needs a large GPU; none run on CPU in practice
- The 8B’s vLLM serving needs NVIDIA’s exact setup
- Too large for our in-browser demo
Choosing for page images?
The scenario guide compares NVIDIA, Qwen3-VL-Reranker, LightOn-rerank and the hosted options for visual documents.
Choose by scenario → All models →