NDCG and MRR calculator
Did the reranker help? Paste the relevance labels of each query’s candidates in the order they were returned, once before reranking and once after, and read off NDCG@k, MRR and Hit@1. It runs in your browser: nothing you type is sent anywhere. The evaluation guide explains how to build the labels.
Your rankings
Results
| Metric | Before | After | Change |
|---|
Per query
| Query | NDCG before | NDCG after | Rank of first relevant |
|---|
How to enter the labels
Each line is one query. Write one number per candidate, in ranked order, separated by spaces or commas: 0 for not relevant, 1 for relevant, and 2 or 3 if you grade relevance. “Before” is the order your retriever returned; “after” is the same candidates in the order the reranker put them. The two lines for a query must hold the same labels, just in a different order.
What the numbers mean
- NDCG@k — discounted cumulative gain of the top k results, divided by the best score those same candidates could have reached. 1.0 means the best candidates are on top. Gain is the label itself and the discount is 1 / log2(rank + 1).
- MRR — the average of 1 / rank of the first relevant result within the top k, counting 0 when there is none.
- Hit@1 — the share of queries whose top result is relevant.
Queries with no relevant candidate at all are left out of the averages, because no reranker can fix them; if there are many, the problem is retrieval, not reranking. Only the candidates you list are scored, so this measures reordering, not recall. With 30 queries a small difference can be noise: treat gains of a point or two as a tie and look at the per-query table before trusting them.
Need candidates to label?
The live demo shows how a cross-encoder reorders a shortlist — a quick way to see which ranking is better before you label a real set.
Open the live demo →