LLM Completion Routers
LLM Function Calling Routers
Semantic Routers
Zero Shot Classification Routers
Language Classification Routers
Below is an example of LLM Completion Router
===========================================
from langchain_anthropic import ChatAnthropic
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import PromptTemplate
# Set up the LLM Chain to return a single word based on the query,
# and based on a list of words we provide to it in the prompt template
llm_completion_select_route_chain = (
PromptTemplate.from_template("""
Given the user question below, classify it as either
being about `LangChain`, `Anthropic`, or `Other`.
Do not respond with more than one word.
<question>
{question}
</question>
Classification:"""
)
| ChatAnthropic(model_name="claude-3-haiku")
| StrOutputParser()
)
# We setup an IF/Else condition to route the query to the correct chain
# based on the LLM completion call above
def route_to_chain(route_name):
if "anthropic" == route_name.lower():
return anthropic_chain
elif "langchain" == route_name.lower():
return langchain_chain
else:
return general_chain
...
# Later on in the application, we can use the response from the LLM
# completion chain to control (i.e route) the flow of the application
# to the correct chain via the route_to_chain method we created
route_name = llm_completion_select_route_chain.invoke(user_query)
chain = route_to_chain(route_name)
chain.invoke(user_query)
LLM Function Calling Router / Pedantic Router
=============================================
This leverages the function-calling ability of LLMs to pick a route to traverse. The different routes are set up as functions with appropriate descriptions in the LLM Function Call. Then, based on the query passed to the LLM, it is able to return the correct function (i.e route), for us to take.
Semantic Router
This router type leverages embeddings and similarity searches to select the best route to traverse.
Each route has a set of example queries associated with it, that become embedded and stored as vectors. The incoming query gets embedded also, and a similarity search is done against the other sample queries from the router. The route which belongs to the query with the closest match gets selected.
from semantic_router import Route
# we could use this as a guide for our chatbot to avoid political
# conversations
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
# this could be used as an indicator to our chatbot to switch to a more
# conversational prompt
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
# we place both of our decisions together into single list
routes = [politics, chitchat]
encoder = OpenAIEncoder()
from semantic_router.layer import RouteLayer
route_layer = RouteLayer(encoder=encoder, routes=routes)
route_layer("don't you love politics?").name
# -> 'politics'
Zero Shot Classification Router
==============================
“Zero-shot text classification is a task in natural language processing where a model is trained on a set of labeled examples but is then able to classify new examples from previously unseen classes”. These routers leverage a Zero-Shot Classification model to assign a label to a piece of text, from a predefined set of labels you pass in to the router.
references:
https://towardsdatascience.com/routing-in-rag-driven-applications-a685460a7220
No comments:
Post a Comment