What is a Cross Encoder? ( Re-ranker)

Characteristics of Cross Encoder (a.k.a reranker) models:

Calculates a similarity score given pairs of texts.

Generally provides superior performance compared to a Sentence Transformer (a.k.a. bi-encoder) model.

Often slower than a Sentence Transformer model, as it requires computation for each pair rather than each text.

Due to the previous 2 characteristics, Cross Encoders are often used to re-rank the top-k results from a Sentence Transformer model.

In Sentence Transformers, a Cross-Encoder is a model architecture designed to compute the similarity between two sentences by considering them jointly. This is in contrast to Bi-Encoders, which encode each sentence independently into vector embeddings.

Here's a breakdown of what a Cross-Encoder is and how it works:

Key Characteristics:

Joint Encoding:

A Cross-Encoder takes both sentences as input at the same time.

It processes them through the transformer network together, allowing the model to capture intricate relationships and dependencies between the words in both sentences.

Accurate Similarity Scores:

Because of this joint processing, Cross-Encoders tend to produce more accurate similarity scores than Bi-Encoders.

They can capture subtle semantic nuances that Bi-Encoders might miss.

Computational Cost:

Cross-Encoders are significantly more computationally expensive than Bi-Encoders.

They cannot pre-compute embeddings for a large corpus of text.

Similarity scores are calculated on-the-fly for each pair of sentences.

Pairwise Comparisons:

Cross-Encoders are best suited for scenarios where you need to compare a relatively small number of sentence pairs.

They excel in tasks like re-ranking search results or determining the similarity between two specific sentences.

How It Works:

Input:

The two sentences to be compared are concatenated or combined in a specific way (e.g., separated by a special token like [SEP]).

Transformer Processing:

The combined input is fed into a transformer-based model (e.g., BERT, RoBERTa).

The model processes the input jointly, attending to the relationships between words in both sentences.

Similarity Score:

The output of the transformer is typically a single value or a vector that represents the similarity between the two sentences.

This value is often passed through a sigmoid function to produce a similarity score between 0 and 1.

When to Use Cross-Encoders:

Re-ranking:

After retrieving a set of candidate documents using a Bi-Encoder, you can use a Cross-Encoder to re-rank the results for improved accuracy.

Semantic Textual Similarity (STS):

For tasks that require highly accurate similarity scores, such as determining the degree of similarity between two sentences.

Question Answering:

When comparing a question to a set of candidate answers, a Cross-Encoder can provide more accurate relevance scores.

When Not to Use Cross-Encoders:

Large-Scale Similarity Search:

If you need to find the most similar sentences in a large corpus, Bi-Encoders are much more efficient.

Real-Time Applications:

The computational cost of Cross-Encoders can make them unsuitable for real-time applications with high throughput requirements.

In essence:

Cross-Encoders prioritize accuracy over speed, making them ideal for tasks where precision is paramount and the number of comparisons is manageable. Bi-Encoders, on the other hand, prioritize speed and scalability, making them suitable for large-scale information retrieval.

References:

https://www.sbert.net/docs/cross_encoder/usage/usage.html