Node Postprocessor
Node Postprocessors apply transformations or filtering to a set of nodes before returning them. In LlamaIndex, node postprocessors are integrated into the query engine, functioning after the node retrieval step and before the response synthesis step. LlamaIndex provides an API for adding custom postprocessors and offers several ready-to-use node postprocessors. Some of the most commonly used node postprocessors are:
CohereRerank: This module is a component of the Cohere natural language processing system that selects the best output from a set of candidates. It uses a neural network to score each candidate based on relevance, semantic similarity, theme, and style. The candidates are then ranked according to their scores, and the top N are returned as the final output.
LLMRerank: Similar to the CohereRerank approach, but it uses an LLM to re-order nodes, returning the top N ranked nodes.
SimilarityPostprocessor: This postprocessor removes nodes that fall below a specified similarity score threshold.
No comments:
Post a Comment