The Ranking and Relevance Catalog

BM25 family in production depth #

Source: Robertson et al., "Okapi at TREC-3" (1995); Robertson and Zaragoza, "The Probabilistic Relevance Framework: BM25 and Beyond" (2009); production implementations in Lucene, Elasticsearch, OpenSearch, Solr

Classification — Scoring function family that remains foundational to production ranking, used both as first-stage retrieval scoring and as features in learned ranking models.

Apply BM25 and its production variants to score query-document pairs in ways that work as standalone first-stage retrieval scoring and as input features to learning-to-rank models.

Volume 1 introduced BM25 as a retrieval pattern; this entry covers the variants production teams use in ranking contexts and the parameter tuning that makes BM25 effective as both standalone scoring and as LTR features. The variants address specific weaknesses of vanilla BM25 (multi-field documents, very long documents, length-normalization artifacts) and the parameter tuning addresses the workload-specific calibration that defaults don't handle.

Vanilla BM25 recap. The formula computes per-term scores that combine term frequency (saturating, controlled by k1), inverse document frequency (rare terms weighted higher), and document length normalization (controlled by b, ratio to average document length). Per-term scores sum to produce the document's BM25 score for the query. The k1 parameter (typical 1.2–2.0) controls how fast term frequency saturates; the b parameter (typical 0.5–1.0, default 0.75) controls how aggressively length is normalized.

BM25F (multi-field). Vanilla BM25 treats a document as a single bag of words. Real documents have fields: title, body, brand, category, tags. BM25F extends BM25 to weight per-field contributions: a term match in the title contributes more than a match in the body, with per-field b (length normalization) and per-field weight parameters. The configuration is per-field weight (how much each field matters) and per-field b (how length normalization applies within each field). In LTR contexts, BM25F per-field scores often appear as separate features rather than combined into one BM25F score; the model learns the right combination.

BM25+ (long-document correction). Vanilla BM25 has a known weakness with very long documents: even with length normalization, very long documents can fail to score competitively against shorter relevant documents due to score saturation artifacts. BM25+ adds a small constant (delta, typically 1.0) to the term score that addresses this. The variant is appropriate when document length distribution is heavily skewed; for most production workloads the difference is marginal.

Parameter tuning. The default BM25 parameters (k1=1.2, b=0.75) come from the Okapi system's TREC experiments in the 1990s. They're reasonable defaults but not optimal for every workload. Production teams tune the parameters per workload: grid search over k1 and b ranges, with NDCG@10 (Volume 5) as the optimization target on the team's judgment list. The tuning often produces 2–5% NDCG improvement over defaults; the marginal benefit per hour of tuning effort is high for the first round and diminishes after.

BM25 as LTR features. In production LTR systems, BM25-derived scores are typically among the most important features. Common feature decompositions: overall BM25 score; per-field BM25F scores (title BM25, body BM25, brand BM25, etc.); BM25 against different analyzers (BM25 against stemmed text vs. raw text); BM25 with different parameter settings (k1=1.2 BM25 and k1=2.0 BM25 as separate features). The LTR model learns to weight these features per query class; the decomposition lets the model use lexical signals more flexibly than a single combined BM25 score would allow.

Caching considerations. Computing BM25 over millions of documents at query time is expensive; production retrieval typically precomputes term-level statistics (document frequency, total tokens) at index time, then BM25 is computed only for the candidate documents that match the query terms. Lucene-based engines (Elasticsearch, OpenSearch, Solr) implement this efficiently through inverted-index skip lists; custom implementations need to handle this correctly.

Every production search system uses BM25-derived scoring somewhere. As first-stage retrieval scoring when lexical match is appropriate. As features in LTR models. As reference scoring for hybrid retrieval (Volume 1 Section C). The pattern is foundational; the question isn't whether to use it but how to integrate it.

Alternatives — pure vector scoring (next entry) for cases where lexical matching is less appropriate (very short queries, heavily synonymized domains). Hybrid retrieval (Volume 1 Section C) for the dominant production pattern combining both. BM25-only retrieval is becoming less common in production as hybrid patterns mature; BM25 as one of multiple paths remains the working pattern.

• Robertson and Zaragoza, "The Probabilistic Relevance Framework: BM25 and Beyond" (2009)
• Manning, Raghavan, Schütze, Introduction to Information Retrieval, ch. 11
• Elasticsearch / OpenSearch / Solr documentation

Code

// Elasticsearch / OpenSearch: per-field BM25 with custom parameters
// This produces BM25F-style scoring with per-field weights

PUT /products
{
"settings": {
"index": {
"similarity": {
// Custom BM25 parameters - tuned per workload via judgment-list
evaluation
"bm25_tuned": {
"type": "BM25",
"k1": 1.5,
"b": 0.7
}
}
}
},
"mappings": {
"properties": {
"title": { "type": "text", "similarity": "bm25_tuned" },
"description": { "type": "text", "similarity": "bm25_tuned"
},
"brand": { "type": "text", "similarity": "bm25_tuned" },
"category": { "type": "keyword" }
}
}
}

// Query with per-field weighting (BM25F-style)
GET /products/_search
{
"query": {
"multi_match": {
"query": "trail running shoes",
"fields": [
"title^4", // Title 4x weight
"brand^2", // Brand 2x weight
"description^1" // Body baseline weight
],
"type": "best_fields"
}
}
}

// For LTR features: extract per-field BM25 scores separately
// using Elasticsearch\'s "explain" or separate queries per field
// Each becomes a feature in the LTR model

GET /products/_search?explain=true
{
"query": {
"bool": {
"should": [
{ "match": { "title": "trail running shoes" }},
{ "match": { "brand": "trail running shoes" }},
{ "match": { "description": "trail running shoes" }}
]
}
},
"size": 100
}
// Each clause\'s score becomes a feature; the LTR model combines
them

Vector similarity scoring #

Source: Karpukhin et al., "Dense Passage Retrieval" (2020); Reimers and Gurevych, sentence-transformers; production implementations in Pinecone, Weaviate, Qdrant, Elasticsearch k-NN, Vespa

Classification — Scoring function for dense vector retrieval and ranking — cosine similarity, dot product, learned similarity functions.

Score query-document pairs by similarity in a learned embedding space, where queries and documents are encoded as dense vectors and similarity captures semantic relationships beyond lexical overlap.

BM25 family captures lexical match but misses semantic similarity. A query for "pain reliever" doesn't lexically match documents about "analgesic"; a BM25-only system misses the relationship. Vector similarity captures semantic relationships by encoding queries and documents in a learned space where conceptually-related inputs produce similar vectors. The pattern is the foundation of dense retrieval (Volume 1 Section B) and one of the standard feature inputs for modern LTR models.

Cosine similarity. The most common vector similarity function. cos(q, d) = (q · d) / (||q|| × ||d||). The dot product divided by the product of magnitudes produces a value in [-1, 1] where 1 is identical direction and -1 is opposite. For normalized vectors (unit length), cosine equals dot product, which is computationally cheaper. Most production embedding models produce normalized vectors and use dot product as the similarity measure; cosine and dot product are interchangeable for normalized vectors.

Dot product. q · d = sum of (q_i × d_i) over all dimensions. The simplest and fastest similarity function. For normalized vectors, dot product equals cosine; for unnormalized vectors, dot product is sensitive to vector magnitudes. Most production deployments use normalized vectors and dot product; the choice between explicit cosine and dot product is implementation detail rather than fundamental design.

Euclidean distance. The geometric distance between vectors: ||q - d||. Smaller distance means more similar; conventions vary on whether to invert this to produce a similarity score. Less common in production than cosine/dot product because cosine's magnitude invariance is typically more aligned with what "similarity" should mean in semantic space.

Learned similarity functions. Beyond geometric similarity, learned functions can produce better quality in some contexts. The most common pattern: project query and document embeddings through a learned linear layer or small neural network before comparing. The projection can be tuned on labeled relevance data. The pattern is less common in retrieval (where simple cosine/dot product allows ANN indexing) and more common in ranking (where the projection can be applied to the small candidate set without retrieval-scale constraints).

Vector similarity as LTR features. In LTR pipelines, vector similarity scores are typically among the more important features. Common feature decompositions: similarity between query and full document embedding; similarity between query and per-section document embeddings (title embedding, body embedding); similarity using different embedding models (OpenAI text-embedding-3 similarity AND BGE similarity as separate features). The decomposition lets the LTR model weight the embedding signals per query class.

Embedding model selection trade-offs. The quality of vector similarity depends primarily on the embedding model. General-purpose models (OpenAI text-embedding-3-large, BGE-large, Voyage 3, Cohere embed v3) work for many use cases. Domain-specific models (LegalBERT for legal, BioBERT for medical, code-specific embeddings for code) outperform general models on their domains. Fine-tuned models on domain-specific labeled data outperform off-the-shelf models when training data is available. The MTEB leaderboard (huggingface.co/spaces/mteb/leaderboard) provides comparison points; production evaluation on the actual workload (Volume 5 Section B) is essential.

Production search with semantic matching needs beyond what synonym engineering provides. Modern hybrid retrieval (Volume 1 Section C). LTR models where semantic signals complement lexical signals. RAG pipelines for agentic systems.

Alternatives — BM25-family scoring (prior entry) for use cases dominated by lexical matching. Late-interaction models (Section D) for cases where simple similarity isn't sufficient but full cross-encoder is too expensive. Pure vector scoring is rarely optimal alone; combined with lexical scoring in hybrid retrieval is the dominant production pattern.

• Karpukhin et al., "Dense Passage Retrieval for Open-Domain QA" (2020)
• Reimers and Gurevych, sentence-transformers documentation
• BEIR benchmark for retrieval (github.com/beir-cellar/beir)
• MTEB leaderboard for embedding comparison

LambdaMART and gradient-boosted decision tree LTR #

Source: Burges, "From RankNet to LambdaRank to LambdaMART: An Overview" (Microsoft Research, 2010); production implementations in LightGBM ranking, XGBoost ranking, RankLib

Classification — The dominant production LTR algorithm for a decade-plus — gradient-boosted decision trees with metric-aware gradients.

Train a learned ranking model from labeled training data that combines many features (50–500 typical) into per-document scores optimized for ranking metrics (NDCG, MAP) rather than for pointwise regression accuracy.

Hand-tuned scoring (BM25 plus boosts) scales to a handful of signals. As ranking signal sources multiplied through the 2000s — dozens then hundreds of signals per query-document pair — manual tuning became intractable. Early LTR methods (RankNet, RankBoost) treated ranking as classification or regression problem, missing the ranking-specific aspects of the metric. LambdaMART addressed this by combining gradient-boosted decision trees (which scale well to many features) with metric-aware gradients (which let the training process optimize for ranking metrics directly).

The training data. LTR training data consists of (query, document, relevance grade) triples. The relevance grades come from judgment lists (Volume 5 Section A) or from click logs with bias correction (Volume 5 Section E). The training data typically has 100–1000 queries with 20–100 documents per query; the queries are sampled to be representative of production traffic.

The model architecture. A gradient-boosted decision tree ensemble: hundreds of small decision trees combined via boosting. Each tree takes features as input and outputs a contribution to the document's score; the trees' contributions sum to produce the final score. The ensemble has many parameters (typically thousands of tree nodes across the ensemble) but trains efficiently because each tree is small and the boosting framework handles the combination.

The lambda gradient. The breakthrough in LambdaRank/LambdaMART. Rather than computing gradients based on per-document loss, the lambda gradient computes per-pair gradients weighted by the change in ranking metric the pair swap would produce. If swapping two documents would change NDCG by a lot, the gradient on that pair is large; if swapping wouldn't affect NDCG much, the gradient is small. The metric-aware gradient lets training optimize for the ranking metric directly, which produces measurably better top-K rankings than pointwise or naive pairwise approaches.

Pointwise, pairwise, listwise framing. Pointwise treats each (query, document, grade) as an independent regression problem: predict the grade. Pairwise treats each pair as a classification problem: which document of the pair should be ranked higher. Listwise considers the full ranked list at once. LambdaMART is fundamentally pairwise with metric-aware gradients; the lambda terms incorporate listwise information without requiring fully listwise training. The framing matters because different implementations support different framings; choose based on the framing the production training data supports.

Production implementations. LightGBM ranking (Microsoft's gradient boosting library with LambdaRank objective) is widely used in production. XGBoost ranking (similar capability in XGBoost) is a popular alternative. RankLib provides Java implementations including LambdaMART. Each implementation has tuning parameters: tree count, tree depth, learning rate, leaf count, regularization. Production teams typically tune via cross-validation against held-out judgments.

Inference. At query time, the trained model is applied to each candidate document's feature vector. For 1000 candidates with 100 features and a 500-tree model, inference takes roughly 5–20ms on CPU; faster with optimized libraries. Production systems integrate LTR inference into the ranking pipeline (often via plugins like Elasticsearch's Learning to Rank plugin or Solr's LTR contrib module).

Production search systems with sufficient training data (judgment lists or bias-corrected click logs) to train a model. The dominant LTR algorithm in production for mid-stage ranking. Often used between first-stage retrieval and neural reranking (Section D) in cascade architectures.

Alternatives — simpler hand-tuned scoring for cold-start without training data. Neural LTR (next entry) for very large training sets where GBDT may underutilize the data. Neural rerankers (Section D) for top-K reranking where their quality wins justify the cost. LambdaMART/GBDT remains the default for production LTR.

• Burges, "From RankNet to LambdaRank to LambdaMART: An Overview" (Microsoft Research, 2010)
• Liu, Learning to Rank for Information Retrieval (Foundations and Trends, 2009)
• LightGBM ranking documentation (lightgbm.readthedocs.io)
• XGBoost ranking documentation (xgboost.readthedocs.io)
• Elasticsearch Learning to Rank plugin (elasticsearch-learning-to-rank.readthedocs.io)

Code

# LightGBM LTR with LambdaRank objective
import lightgbm as lgb
import numpy as np
import pandas as pd

# Training data format:
# - features: (n_samples, n_features) matrix
# - labels: relevance grades (0-4 typical)
# - group: array of per-query document counts
# e.g. group=[20, 30, 25] means query 1 has 20 docs, query 2 has
30 docs, etc.

X_train = features_df[FEATURE_COLS].values # shape: (N,
n_features)
y_train = features_df[\'relevance_grade\'].values # shape: (N,)
groups_train = features_df.groupby(\'query_id\').size().values #
docs per query

# Create LightGBM dataset with group info (essential for ranking)
train_dataset = lgb.Dataset(
X_train,
label=y_train,
group=groups_train
)

# Train with LambdaRank objective
params = {
\'objective\': \'lambdarank\',
\'metric\': \'ndcg\',
\'ndcg_eval_at\': [3, 5, 10], # evaluate NDCG@3, NDCG@5, NDCG@10
\'learning_rate\': 0.05,
\'num_leaves\': 31,
\'min_data_in_leaf\': 50,
\'lambdarank_truncation_level\': 10, # focus on top-10 positions
\'verbose\': -1,
}

model = lgb.train(
params,
train_dataset,
num_boost_round=500,
valid_sets=[validation_dataset],
callbacks=[lgb.early_stopping(stopping_rounds=20)]
)

# Inference at query time
def rank_candidates(query, candidates):
# Extract features for each (query, candidate) pair
feature_matrix = np.array([
extract_features(query, doc) for doc in candidates
])
# Score each candidate
scores = model.predict(feature_matrix)
# Sort by score descending
ranked = sorted(
zip(candidates, scores),
key=lambda x: x[1],
reverse=True
)
return [doc for doc, score in ranked]

# Feature importance for debugging
importance = model.feature_importance(importance_type=\'gain\')
feature_importance_df = pd.DataFrame({
\'feature\': FEATURE_COLS,
\'importance\': importance
}).sort_values(\'importance\', ascending=False)
print(feature_importance_df.head(20))

Pointwise, pairwise, and listwise loss functions #

Source: Liu, Learning to Rank for Information Retrieval (2009); Burges on RankNet/LambdaRank/LambdaMART; Cao et al. on ListNet (2007); Xia et al. on ListMLE (2008)

Classification — The three framings of the ranking problem as machine learning task, each with different loss functions and characteristic strengths.

Choose the right framing of the ranking problem as a machine learning task: pointwise (regression per document), pairwise (preference classification per pair), or listwise (loss over the full ranked list).

The same training data — (query, document, relevance grade) triples — can be used to train ranking models in three fundamentally different ways. Pointwise treats each grade as a target for regression; pairwise treats each pair of documents as a preference classification problem; listwise treats the full ranked list as the unit of training. The framing affects what the model learns, what training data is most useful, and what model architectures fit. Understanding the trade-offs is the prerequisite for choosing the right framing.

Pointwise framing. The simplest framing: treat each (query, document, grade) as an independent training example. The model learns to predict the grade from the features. At inference time, predict grades for all candidates and sort by predicted grade. Loss functions: mean squared error (for graded relevance), cross-entropy (for graded relevance treated as ordinal classification), log-loss (for binary relevance). Pointwise is simple to implement and easy to train but loses the relational information — the model doesn't know that documents are being compared against each other for the same query.

Pairwise framing. Treat each pair of documents within a query as a training example. The model learns to predict which document should be ranked higher. The loss is computed per pair: if document A has higher grade than B but the model predicts B higher than A, that pair contributes to the loss. RankNet was the first widely-adopted pairwise framing; LambdaRank/LambdaMART are improvements that weight the pair loss by the metric impact of the swap. Pairwise framing uses the relational information explicitly and typically outperforms pointwise on ranking metrics.

Listwise framing. Consider the full ranked list as the training unit. The loss measures the quality of the predicted ranking against the ideal ranking for that query. ListNet uses a probabilistic permutation distribution loss; ListMLE uses maximum likelihood over permutations; LambdaMART's metric-aware gradients incorporate listwise information without being fully listwise. Listwise framings can in principle outperform pairwise because they directly optimize the metric structure of ranking, but they're harder to train (loss surfaces are more complex) and often produce only marginal gains over LambdaMART in practice.

Training data implications. Pointwise needs only individual (query, doc, grade) triples; the relational information isn't used at training time. Pairwise needs pairs of documents for the same query with different grades; if every document for a query has the same grade, no training signal. Listwise needs the full ranked list per query, with reasonable grade distribution. Production teams typically have judgment lists structured to support all three; the choice depends on the algorithm, not the data.

Implementation framework support. LightGBM and XGBoost ranking implement LambdaRank-style pairwise framing with metric-aware gradients (the LambdaMART method). TF-Ranking supports pointwise, pairwise, and listwise framings explicitly. RankLib provides multiple implementations. The choice often comes down to the framework the team is already using; the algorithmic differences within the pairwise/listwise family are typically smaller than the engineering differences between frameworks.

Practical guidance. For most production systems: pairwise framing with metric-aware gradients (LambdaMART) is the default and produces excellent results. Pointwise is appropriate for very simple use cases or when training data is too sparse for pairwise (few documents per query). Listwise is appropriate for very large training sets where the additional complexity is justified by marginal gains. The marginal benefit of listwise over LambdaMART on most workloads is small enough that most teams stick with LambdaMART.

Pairwise (LambdaMART/LambdaRank) for most production LTR. Pointwise for cold-start, very simple cases, or when implementing custom systems without metric-aware gradient infrastructure. Listwise for cases where production-grade implementations (TF-Ranking) are available and additional complexity is justified.

Alternatives — neural rerankers (Section D) for top-K reranking where their architecture wins over feature-based LTR. The framing choice is within LTR; the broader choice is whether LTR or neural reranking fits the use case.

• Liu, Learning to Rank for Information Retrieval (Foundations and Trends, 2009)
• Cao et al., "ListNet: Learning to Rank Using Gradient Descent" (2007)
• Xia et al., "Listwise Approach to Learning to Rank" (ICML 2008)
• TF-Ranking documentation (github.com/tensorflow/ranking)

Feature engineering and ablation methodology #

Source: Production methodology at major search teams; Grainger, AI-Powered Search; Lefortier et al. on online evaluation

Classification — The discipline of designing, validating, and managing features for ranking models.

Build a feature set that contributes meaningfully to ranking quality, validate each feature's value through ablation, and manage the feature pipeline at production scale with consistency between training and serving.

Adding features to an LTR model without ablation is the most common mistake in feature engineering. A feature that looks plausible may add noise rather than signal; many features that domain experts believe should help turn out not to. Without ablation, the model accumulates marginal-or-negative features that bloat the model and confuse interpretation. The discipline of feature engineering involves not just generating candidate features but rigorously validating which ones contribute.

Feature generation. Start from the four categories (Chapter 3): query features, document features, query-document features, context features. For each category, brainstorm candidate features based on domain knowledge: what signals would a domain expert use to determine relevance? E-commerce: brand reputation, inventory level, price competitiveness, sales velocity. Enterprise search: document recency, author authority, document type. The brainstorming produces a candidate list; the validation determines which candidates earn places in the production model.

Feature computation. Each feature is implemented as a function: given a query and a document (and possibly context), return a numeric value. The function runs at training time and at query time; the implementations must produce identical values in both contexts. Training-serving skew (the feature value differs between training and serving) is a major production failure mode that requires careful engineering to prevent. The implementations are typically version-controlled code, deployed as part of the search service, with explicit testing for training-serving consistency.

Single-feature validation. Before including a feature in the model, validate it independently: does it correlate with relevance? Compute the feature for all (query, document) pairs in the judgment list; check whether higher feature values correlate with higher relevance grades. Spearman rank correlation is a common metric; correlation above 0.1 is a weak signal worth considering; above 0.3 is moderate; above 0.5 is strong. Features with near-zero correlation are unlikely to add value to the model.

Ablation studies. The gold-standard validation: train the model with and without each feature; measure the NDCG change. Features that improve NDCG meaningfully when added (or decrease it meaningfully when removed) are contributors; features that don't affect NDCG are non-contributors that should be removed. Ablation is expensive (each feature requires a full model training) but produces definitive answers; production teams typically ablate features in batches rather than individually.

Feature importance from trained models. GBDT models report feature importance (gain or split count). Features with low importance are candidates for removal; features with high importance are core to the model. Importance alone isn't sufficient for removal decisions (high-importance features could still be redundant with each other; low-importance features could still be necessary for specific query types), but it's a starting point for ablation prioritization.

Feature pipeline infrastructure. At small scale, features are computed inline in training and serving code. At larger scale, feature stores (Feast, Tecton, Hopsworks, or custom platforms) manage feature definitions, computation, and serving. The feature store provides: consistent feature definitions across training and serving; pre-computed features for offline access; online feature serving for query-time lookup; feature versioning and rollback. The infrastructure investment is substantial; production teams typically build it once feature count and complexity justify the engineering.

Feature monitoring in production. Production features can drift: a feature that depended on an upstream signal may produce stale or wrong values if the upstream signal changes. Production monitoring: track per-feature statistics over time (mean, variance, distribution); alert when distributions shift significantly; investigate root causes. The monitoring is operations discipline; the planned Volume 6 (Search Operations) covers the broader discipline.

Every production LTR deployment needs feature engineering discipline. The investment compounds: features built once support many model iterations; the validation methodology applies to every new feature added. Teams without this discipline typically build models with many features that don't contribute, producing bloated models that are hard to maintain.

Alternatives — neural rerankers (Section D) that operate directly on raw text rather than engineered features bypass much of the feature engineering work, at the cost of higher computational requirements and less interpretability. Feature engineering remains essential for the LTR portion of cascade architectures even when neural rerankers handle the top-K stage.

• Grainger, AI-Powered Search, chapters on feature engineering for relevance
• Production methodology writings from search teams at Etsy, Spotify, others
• Feast feature store documentation (feast.dev)

Cross-encoder reranking in production #

Source: Nogueira and Cho, "Passage Re-ranking with BERT" (2019); production rerankers: sentence-transformers, BGE Reranker, monoT5, RankZephyr, Cohere Rerank, Voyage Rerank

Classification — Top-K reranking using transformer models that jointly process query and document for fine-grained interaction scoring.

Apply transformer-based cross-encoder scoring to a small candidate set to produce substantially higher top-K quality than feature-based LTR can achieve, accepting the higher computational cost in exchange for the quality.

Volume 1 Section D introduced the cross-encoder pattern at the architectural level: independent retrieval produces candidates; cross-encoder reranks them. This entry covers the methods themselves — which models to use, how to train them, how to deploy them productively in cascade architectures.

The model architecture. A transformer (BERT, T5, Llama-based, or other modern architectures) takes a concatenated input: [CLS] query [SEP] document [SEP]. The transformer's attention layers compute joint query-document representations; a final classification head outputs a single relevance score. The model is fine-tuned on labeled relevance data (typically MS MARCO or domain-specific data) to predict relevance scores that correlate with human judgments.

Model size trade-offs. Smaller models (110M parameter BERT-base, 220M parameter T5-base) are faster and cheaper but less capable. Larger models (335M BERT-large, 770M T5-large, 7B+ Llama-based) are slower and more expensive but produce better rankings. Production deployments choose model size based on latency budget and quality requirements; many teams find that small-to-medium models (220M–500M parameters) provide good cost-quality balance.

Open model options. Sentence-transformers cross-encoders (sbert.net): the canonical open-source option, BERT-based, available in many sizes. BGE Reranker (BAAI): strong quality, multilingual, free and self-hostable. mxbai-rerank: Mixedbread AI's open release. monoT5 and RankT5: T5-based rerankers from the Pradeep/Nogueira/Lin group. RankZephyr: instruction-tuned listwise reranker. Each has trade-offs in quality, latency, and language coverage.

Commercial API options. Cohere Rerank (cohere.com/rerank): easy integration, multiple model sizes, multilingual. Voyage Rerank: high quality, especially on domain-specific data. Mixedbread Rerank: API access to the mxbai-rerank models. Trade-offs vs. self-hosting are the standard ones from the agentic AI series Volume 16: API simplifies operations and provides consistent quality; self-hosting reduces per-query cost at scale and provides more control.

Production deployment. Cross-encoder reranking is typically deployed as a final stage in cascade architectures (Volume 1 Section D). First-stage retrieval (BM25, vector, or hybrid) produces 100–1000 candidates; mid-stage LTR may reduce to 50–200 candidates; cross-encoder reranks the survivors. Latency is bounded by candidate count times per-pair scoring time; production teams tune candidate counts for the latency budget. Hardware (GPU, ideally) and batching (8–64 pairs per inference call) substantially affect achievable throughput.

Fine-tuning on domain data. Off-the-shelf rerankers are trained on general data (MS MARCO, NQ); fine-tuning on domain-specific labeled data typically produces 2–10% quality improvement. The training data: query-document pairs with relevance grades, hard negatives (documents that look relevant but aren't) mined from production, optionally easy negatives (clearly irrelevant documents). Training frameworks: sentence-transformers for self-hosted training; commercial APIs may offer fine-tuning as a service. The investment in fine-tuning pays off when domain quality matters and the team has training data infrastructure.

Calibration concerns. Cross-encoder scores are not directly comparable across queries (the model wasn't trained to produce calibrated scores). This matters for some downstream uses: weighted hybrid scoring (Volume 1 Section C) that combines reranker scores with other signals needs calibration. The calibration methods (Platt scaling, isotonic regression) are standard ML techniques applied to the reranker outputs.

Production search where top-K precision matters and the computational cost of cross-encoder reranking is justified. RAG pipelines (agentic AI Volume 10) where reranking improves the documents passed to the LLM. E-commerce, enterprise search, and customer-service search where ranking quality has business value. Cascade architectures where the cross-encoder is the final stage.

Alternatives — LTR-based reranking (Section B) for cases where the cross-encoder cost isn't justified. Late-interaction (next entry) for cost-quality trade-offs between bi-encoder and cross-encoder. Pure first-stage retrieval for low-stakes applications.

• Nogueira and Cho, "Passage Re-ranking with BERT" (2019)
• Pradeep, Nogueira, Lin on monoT5 and successor models
• Sentence-Transformers documentation (sbert.net)
• Cohere Rerank documentation (cohere.com/rerank)
• BGE Reranker (huggingface.co/BAAI/bge-reranker-v2-m3)

Code

# Production cross-encoder reranking with sentence-transformers
from sentence_transformers import CrossEncoder
import torch

# Load a production-quality reranker
# Trade-offs: ms-marco-MiniLM-L-6-v2 is fast (~80MB);
BGE-reranker-v2-m3 is higher quality (~570MB)
model = CrossEncoder(\'cross-encoder/ms-marco-MiniLM-L-6-v2\',
device=\'cuda\')

def rerank_candidates(query, candidates, top_k=10):
"""
Apply cross-encoder reranking to candidates from first-stage
retrieval.
candidates: list of dicts with \'doc_id\' and \'text\' fields
"""
# Build query-document pairs
pairs = [(query, c[\'text\']) for c in candidates]
# Score in batches for throughput; GPU batching is essential for
production latency
scores = model.predict(
pairs,
batch_size=32,
show_progress_bar=False,
convert_to_numpy=True
)
# Attach scores and sort
for c, s in zip(candidates, scores):
c[\'rerank_score\'] = float(s)
ranked = sorted(candidates, key=lambda c: c[\'rerank_score\'],
reverse=True)
return ranked[:top_k]

# Production pattern: rerank top 100 from retrieval to top 10 for
display
retrieved = first_stage_retrieve(query, top_k=100) # BM25, hybrid,
etc.
reranked = rerank_candidates(query, retrieved, top_k=10)

# Commercial API alternative: Cohere Rerank
import cohere
co = cohere.Client()

def rerank_via_cohere(query, candidates, top_k=10):
response = co.rerank(
model=\'rerank-english-v3.0\',
query=query,
documents=[c[\'text\'] for c in candidates],
top_n=top_k
)
return [
{**candidates[r.index], \'rerank_score\': r.relevance_score}
for r in response.results
]

# Latency monitoring (essential for production)
import time
start = time.perf_counter()
reranked = rerank_candidates(query, retrieved, top_k=10)
latency_ms = (time.perf_counter() - start) * 1000
print(f\'Rerank latency: {latency_ms:.1f}ms for {len(retrieved)}
candidates\')
# Target: < 100ms for 100 candidates on GPU; expect 5-10x longer on
CPU

Late-interaction models (ColBERT family) #

Source: Khattab and Zaharia, "ColBERT" (SIGIR 2020); Santhanam et al., "ColBERTv2" (2021); Lin et al., "PLAID" (2022)

Classification — Reranking architecture that pre-computes per-token document embeddings and combines with per-token query embeddings via late interaction, achieving cross-encoder-like quality at lower cost.

Bridge the cost-quality gap between bi-encoder retrieval (fast but lower quality) and cross-encoder reranking (high quality but expensive) by pre-computing document representations at index time while preserving fine-grained interactions at query time.

Bi-encoders scale beautifully (precomputed doc vectors) but miss fine-grained query-document interactions. Cross-encoders capture interactions but require running the full transformer for every query-document pair at query time. Late-interaction models address this gap: precompute per-token document embeddings (like bi-encoders) but compute per-token query embeddings at query time and combine via a learned interaction function (preserving more interaction information than simple cosine similarity).

The architecture. At index time: pass each document through a transformer, get per-token embeddings, store all of them (not just a pooled document embedding). For a document with 100 tokens, store 100 embeddings rather than 1. At query time: pass the query through the same transformer, get per-token query embeddings. Compute the interaction: for each query token, find its maximum similarity with any document token; sum (or aggregate) these per-query-token maximums to produce the document's score. The interaction is learned during training to optimize for ranking metrics.

Cost analysis. Index size: ColBERT documents take roughly 100x more space than bi-encoder documents (per-token rather than per-document vectors), though ColBERTv2 introduced compression that substantially reduces this. Query-time computation: faster than cross-encoder (no joint transformer pass per pair) but slower than bi-encoder (per-token similarity computation rather than single vector comparison). Production deployment requires more storage but less query-time compute than cross-encoder; the trade-off fits cost-sensitive deployments where cross-encoder quality is needed but cross-encoder cost isn't affordable.

ColBERTv2 improvements. The 2021 paper added residual compression (4 bits per token vector instead of 32) and centroid clustering for retrieval (find approximate matches in the per-token vector space). The improvements reduced storage costs and enabled ColBERT to serve as first-stage retrieval, not just reranking. PLAID (2022) added further optimizations for latency.

Production deployment options. Stanford's ColBERT codebase (github.com/stanford-futuredata/ColBERT) is the reference implementation. Vespa (vespa.ai) supports ColBERT-style late interaction natively. Production deployments are less common than for cross-encoders because the architectural complexity is higher; teams that have invested in ColBERT-style infrastructure report strong cost-quality results.

Quality comparison. ColBERT typically produces quality between bi-encoder retrieval and full cross-encoder reranking, closer to the cross-encoder end. The exact comparison depends on the specific models being compared (ColBERTv2 vs. which cross-encoder, on which benchmark). On BEIR benchmark tasks, ColBERTv2 produces strong results competitive with cross-encoder rerankers at substantially lower query-time cost.

Limitations. The architectural complexity is non-trivial; teams need to build or adopt ColBERT-specific infrastructure that doesn't fit into standard inverted-index or vector-index frameworks. Storage overhead is substantial despite compression. Adoption is slower than cross-encoders because the engineering investment is larger. Production teams that have invested in ColBERT report it as a long-term cost optimization; for shorter-term deployments, cross-encoders are usually simpler.

Production search at scale where cross-encoder cost is prohibitive but cross-encoder quality is needed. Teams with infrastructure engineering capacity to build or adopt ColBERT-specific systems. Cost-sensitive deployments where the query-time cost matters more than the index-time cost.

Alternatives — cross-encoder reranking (prior entry) for cases where the simpler architecture wins. Bi-encoder retrieval (Volume 1 Section B) for cases where its simpler quality is sufficient. The cascade pattern with cross-encoder is more common in production; ColBERT is the option for the specific cases where its trade-offs win.

• Khattab and Zaharia, "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT" (SIGIR 2020)
• Santhanam et al., "ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction" (2021)
• ColBERT codebase (github.com/stanford-futuredata/ColBERT)
• Vespa late-interaction documentation

Personalization features in ranking pipelines #

Source: Production methodology at e-commerce and consumer search companies; Tunkelang on personalized search; Coveo and Algolia personalization documentation

Classification — Pattern for integrating user-specific, session-specific, and contextual signals as features in LTR ranking models.

Adjust ranking based on context features that capture who the user is, what they've done recently, and their current environment, producing per-user-per-query ranking that outperforms uniform ranking on personal-relevance metrics.

Two users entering the same query string often want different results. A returning customer searching "running shoes" probably wants results filtered by their prior preferences (size, brand affinity, price tier). A user in a specific region wants results adjusted for local availability. A user with prior session context (just viewed Nike products) probably wants related items boosted. Without personalization, the ranking treats all users identically, missing per-user relevance signals that improve outcomes meaningfully.

User features. Long-term signals derived from accumulated user behavior: purchase history (what brands, categories, price ranges they've bought), viewing history (what they've looked at without buying), preference signals (saved searches, wish lists, ratings), demographic signals where available and legitimate (age band, region). These features are computed offline and stored in a user profile that the ranking pipeline reads at query time.

Session features. Short-term signals from the current session: queries earlier in the session, items viewed, filters applied, time spent on results, items added to cart. Session features capture the user's current intent more directly than long-term profile data; a user who just searched "red running shoes" and is now searching "women's" is probably looking for women's red running shoes specifically. Session features are typically computed on-the-fly from a session store (Redis, custom in-memory infrastructure).

Contextual features. Time, device, geographic, and environmental signals: hour of day (some queries shift meaning by time — "restaurant" at noon means lunch, at 10pm means dinner), day of week, season, device type (mobile vs desktop affects result presentation needs), geographic location and locale, traffic source. Contextual features are derived from request metadata; they're typically the simplest to compute and add.

Feature integration as LTR features. The personalization signals join other features in the LTR model. A model trained on (query, document, user/session context, relevance) tuples learns to use the context features alongside query, document, and query-document features. The LTR model decides per-query how much weight to give the personalization features; for navigational queries (where the user wants a specific item regardless of personalization) the model learns to weight personalization less; for discovery queries (where personal preference matters) the model weights personalization more.

Cold-start handling. New users without profiles, anonymous sessions, no available context: personalization needs to degrade gracefully. Common patterns: fall back to non-personalized model (when no context available, model uses query and document features only); use weak available signals cautiously (locale-based defaults); let session signals accumulate within the session to enable per-session personalization even without long-term profile.

Privacy considerations. User behavior data is sensitive. Personalization implementations must comply with privacy regulations (GDPR, CCPA, sector-specific rules), retention policies, and user consent frameworks. Best practice: keep personalization signals visible to users ("Recommended based on your recent activity"), allow opt-out, log the signals used for any specific result. The discipline overlaps with the agentic AI series' compliance volume; for search specifically, personalization opacity erodes user trust over time.

Filter bubble considerations. Aggressive personalization can collapse results to a narrow set the user has previously engaged with, missing valuable discovery. Production patterns: diversification rules (Section F) that limit how much personalization narrows results; exploration injection that surfaces non-personalized candidates; periodic re-evaluation that catches when personalization is hurting outcomes. Multi-objective ranking (Section G) provides the framework for balancing personalization against discovery.

E-commerce search where user history and preferences meaningfully affect what users want. Enterprise search where user role, team, and access patterns affect relevance. Consumer search where session context shapes intent. Multi-region deployments where locale-based personalization is mandatory.

Alternatives — non-personalized ranking for narrow use cases or cold-start situations. Lightweight personalization (locale, device) only for privacy-sensitive deployments. The right level of personalization depends on the use case; defaulting to maximum personalization is typically wrong.

• Tunkelang's writing on personalized search
• Grainger, AI-Powered Search, chapters on personalization
• Coveo personalization documentation
• Algolia personalization documentation

Maximal Marginal Relevance (MMR) and diversification #

Source: Carbonell and Goldstein, "The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries" (SIGIR 1998); production implementations across search platforms

Classification — Diversification algorithm that iteratively selects results balancing relevance against similarity to already-selected results.

Produce ranked result lists that balance relevance to the query against diversity of results, addressing the failure mode where pure-relevance ranking surfaces clusters of similar documents and misses alternative intents the query might cover.

A pure-relevance ranking can surface the most relevant document twice (or in slight variants), or surface 10 documents all addressing the same intent when the query has multiple plausible meanings. For navigational queries with one clear intent, the clustering is fine; for ambiguous or discovery queries, the clustering hurts: users who wanted intent B see 10 results for intent A. Diversification addresses this by trading some relevance for diversity, producing result lists that cover multiple intents within the top-K positions.

The MMR algorithm. Iteratively select the next result by maximizing a combined score: MMR_score = lambda × (relevance to query) - (1 - lambda) × (max similarity to already-selected results). The lambda parameter (0 to 1) controls the trade-off: lambda = 1 is pure relevance (ignore diversity); lambda = 0 is pure diversity (ignore relevance); lambda = 0.5 is balanced. Typical production values: 0.7–0.9 (favor relevance with some diversity boost).

The similarity measure. MMR needs a way to measure similarity between documents — to determine which candidates are "similar to already-selected." Common choices: vector similarity in the document embedding space (cosine of document embeddings); category overlap (documents in the same category are similar); explicit attributes (same brand, same product line). The choice affects what "diversity" means; vector similarity captures semantic diversity, while category overlap captures categorical diversity.

The iteration. Start with empty selection. Score all candidates by MMR_score (only the relevance term is active initially, since no selected results to compare against). Pick the highest-scoring candidate. Recompute scores with that candidate now in the selection. Pick the next highest. Repeat until top-K is filled. The greedy selection produces good diversification in practice; it's not optimal in a global sense but is computationally tractable and produces interpretable behavior.

Per-query-class tuning. The right lambda varies by query class. Navigational queries (one clear intent) can use lambda = 0.95 or higher (almost no diversification). Informational/discovery queries benefit from lambda = 0.7–0.8 (meaningful diversification). Specific query types where ambiguity is common ("jaguar" could mean car or animal) need stronger diversification (lambda = 0.6 or lower).

Beyond MMR. More sophisticated diversification methods exist: determinantal point processes (DPP) produce diversification with theoretical guarantees; learned diversification methods train models to predict ideal diverse result sets. DPP is less commonly deployed because the algorithmic complexity is higher and the marginal benefit over well-tuned MMR is small in most workloads. MMR remains the production default; alternative methods are for cases where MMR specifically isn't sufficient.

Production integration. MMR is typically applied as a post-processing step after LTR or reranking: the ranker produces an ordered candidate list with scores; MMR reorders the top-K by introducing diversity. The original ranker's scores become the "relevance" input to MMR; the similarity function operates on document attributes available at query time. Latency overhead is small (the inner loop is O(K^2) similarity comparisons for top-K results).

Discovery and informational queries where users benefit from seeing multiple intents in the result list. Ambiguous queries with multiple plausible interpretations. Browse-style search interfaces where users scan many results. Cases where pure-relevance ranking produces visibly redundant results (multiple variants of the same product, multiple articles on the same sub-topic).

Alternatives — no diversification for navigational queries where one intent dominates. Categorical filtering or facet-based browsing for cases where the user explicitly chooses among intents through UI rather than implicitly through ranking. The pattern is best suited to queries where the system must guess at user intent and benefits from hedging.

• Carbonell and Goldstein, "The Use of MMR, Diversity-Based Reranking" (SIGIR 1998)
• Kulesza and Taskar, "Determinantal Point Processes for Machine Learning" (2012)
• Production methodology writings on diversification

Multi-objective ranking with weighted combination and business rules #

Source: Production methodology at e-commerce and consumer search companies; emerging academic literature on multi-objective LTR

Classification — Pattern for combining multiple ranking objectives (relevance, freshness, diversity, business) into a single final ranking through weighted combination and business rule integration.

Produce final ranking that balances relevance with other objectives (freshness, diversity, business goals) through explicit weighting that can be tuned per query class and validated through A/B testing.

Pure relevance optimization produces problems visible to users and to the business: monotonous results (no diversity), stale-feeling results (no freshness boost), results that ignore business priorities (out-of-stock items shown, low-margin items prioritized over high-margin equivalents, no integration with promotional campaigns). Production ranking must integrate multiple objectives; the discipline is doing so explicitly with tunable weights rather than ad-hoc adjustments scattered throughout the pipeline.

The weighted-combination pattern. Final score = w_relevance × relevance_score + w_freshness × freshness_score + w_diversity × diversity_score + w_business × business_score + w_personalization × personalization_score. The component scores are normalized to comparable ranges (typically [0, 1]); the weights determine how much each objective influences the final order. The weights are explicit configuration that can be tuned, documented, and explained.

Component score computation. Relevance score: typically the LTR or reranker output. Freshness score: typically a decay function over document age (exp(-age/half-life) or similar), with the half-life tuned for the workload (news search: hours; e-commerce: weeks; reference content: months/years). Diversity score: from MMR (Section F) or similar. Business score: company-specific (margin times inventory-availability times promotion-active, for example). Personalization score: from the LTR model's personalization features (Section E).

Per-query-class weighting. Different query classes deserve different weights. Navigational queries: high relevance weight, low diversity, low freshness (the user wants the specific item). Discovery queries: balanced relevance and diversity, moderate freshness. News-style queries: high freshness weight. Promotional queries ("black friday deals"): high business weight. Production systems with query routing (Volume 1 Section E) typically use per-route weights configured per query class.

Business rules integration. Beyond the score-based weighting, business rules apply explicit overrides. Hide out-of-stock items (filter, not score adjustment). Boost promoted items (multiplicative boost on business score). Demote items the company doesn't want to surface (subtract from score). Enforce diversity constraints (no more than 3 items from same brand in top 10). Business rules typically apply after the score-based ranking; they can preserve relative ordering between unaffected items or override it completely depending on the rule type.

Tuning through experimentation. The right weights aren't obvious from theory; production teams tune via A/B testing (Volume 5 Section D). Run experiments with different weight combinations; measure business metrics (conversion, revenue, satisfaction); pick the combination that produces best business outcomes. The tuning is continuous: weights that worked last quarter may not work this quarter as the inventory, user behavior, or competitive landscape shifts.

Explicit documentation. Production teams should document the weights, what they're intended to optimize, and how they've been tuned. Without documentation, the weights become opaque institutional knowledge that's lost when team members change; with documentation, the rationale survives transitions and provides the basis for future tuning decisions.

The trade-off discipline. Multi-objective ranking has no "optimal" solution — different objectives genuinely conflict and there's no single best balance. Production teams that take multi-objective ranking seriously accept this and invest in: explicit weighting decisions; continuous experimentation; business-metric tracking. The teams that don't take it seriously typically have implicit weighting (the LTR model alone, with no explicit non-relevance objectives) that produces invisible drift from business priorities.

Production e-commerce search where business factors (margin, inventory, promotions) affect what should be ranked. News and content search where freshness matters. Enterprise search with diverse content types where one ranking approach doesn't fit all. Cases where pure-relevance ranking produces business-visible failures.

Alternatives — pure-relevance ranking for narrow research or academic contexts where business factors aren't the immediate concern. Implicit multi-objective (the LTR model alone) for simple deployments where explicit multi-objective adds more complexity than value. Most production search needs at least minimal multi-objective discipline.

• Production methodology writings on multi-objective ranking
• Tunkelang on search ranking strategy
• Industry conference talks (Haystack, Berlin Buzzwords) on production multi-objective patterns

Resources for tracking ranking and relevance discipline #

Source: Multiple academic, practitioner, and vendor sources

Classification — Sources for staying current on ranking and relevance practice.

Provide pointers to the active sources of ranking and relevance knowledge: foundational texts, academic and industry venues, practitioner writing, open-source tools, communities.

Ranking discipline spans academic literature (information retrieval, machine learning), industry methodology (production search teams), vendor tooling (commercial rerankers, LTR frameworks), and emerging techniques (LLM-based ranking, novel architectures). Production teams need ongoing engagement with multiple sources to keep current.

Foundational texts. Liu, Learning to Rank for Information Retrieval (Foundations and Trends in IR, 2009) — the canonical LTR survey, still the best comprehensive reference. Manning, Raghavan, Schütze, Introduction to Information Retrieval (free online) — broader IR with strong ranking coverage. Grainger, AI-Powered Search (Manning, 2024) — modern hybrid ranking in production depth. Turnbull and Berryman, Relevant Search (Manning, 2016) — practical relevance engineering with lexical focus.

Academic conferences. SIGIR (ACM Special Interest Group on Information Retrieval) — the premier IR venue, where most ranking research is published. CIKM, WSDM, ECIR — adjacent venues with substantial ranking content. NeurIPS, ICML, ICLR — ML conferences where neural ranking methods appear. The TREC competitions — historical evaluation infrastructure that produced much of the methodology in this volume.

Industry venues. Haystack Conference (haystackconf.com) — the practitioner conference where ranking work is heavily represented. Berlin Buzzwords — European search and data engineering. AI-Powered Search Conference (related to Grainger's book). Vendor developer conferences (Elastic, Algolia, Coveo) periodically cover ranking patterns.

Practitioner writing. Daniel Tunkelang — long-running blog on search strategy and ranking. OpenSource Connections — practical relevance engineering content. Doug Turnbull — writing across multiple venues. Trey Grainger — ongoing AI-Powered Search content. Search team blogs at Etsy, Wayfair, Spotify, GitHub, Pinterest, and other major search-driven companies periodically publish substantial ranking methodology.

Open-source tools and code. LightGBM ranking (lightgbm.readthedocs.io) and XGBoost ranking (xgboost.readthedocs.io) — production-grade GBDT LTR. TF-Ranking (github.com/tensorflow/ranking) — listwise neural LTR. RankLib (sourceforge.net/p/lemur/wiki/RankLib/) — Java implementations of multiple LTR algorithms. Sentence-transformers (sbert.net) — cross-encoders for reranking. ColBERT (github.com/stanford-futuredata/ColBERT) — late-interaction reference implementation. Elasticsearch Learning to Rank plugin and Solr LTR contrib — production integration.

Datasets and benchmarks. MS MARCO — the canonical web search dataset for training and evaluation. BEIR — retrieval benchmark across many domains. MTEB — embedding model benchmark relevant to vector ranking. TREC datasets — historical IR benchmarks. Production teams use these as comparison points for their own systems and as training data for fine-tuning.

Communities. Relevancy Engineering Slack (via OpenSource Connections invitation) — the primary practitioner community. Reddit r/searchengines, r/elasticsearch — casual discussion. Conference attendance (Haystack especially) produces network effects.

Emerging areas to watch. LLM-based ranking continues to evolve: monoT5 and successors, RankGPT-style direct LLM ranking, listwise LLM rerankers. Counterfactual learning to rank using bias-corrected production data. Multi-modal ranking (image + text, video + text). Multi-objective LTR with explicit constraint handling. The frontier is active; tracking SIGIR and Haystack catches most of the developments.

Ranking engineers building or maintaining production ranking systems. Engineers transitioning into ranking from adjacent fields. Continuous education as the discipline evolves. Reference when specific ranking needs go beyond what current knowledge handles.

Alternatives — specialized consulting (RelevantSearch.AI, OpenSource Connections, others) for high-stakes ranking engagements. Internal documentation for teams with mature practice. The combination of external tracking and internal knowledge is the working pattern.

• Liu, Learning to Rank for Information Retrieval (2009)
• Grainger, AI-Powered Search (2024); Turnbull/Berryman, Relevant Search (2016)
• Manning et al., Introduction to Information Retrieval (free online)
• SIGIR, Haystack, Berlin Buzzwords proceedings
• LightGBM, XGBoost, TF-Ranking, sentence-transformers, ColBERT documentation
• MS MARCO, BEIR, MTEB benchmarks
• Relevancy Engineering Slack