Mistral 7B based Langchain Chat

Mistral-7B is a powerful language model (opensourced currently) with 7.3 Billion parameters that outperforms many state of the art models with higher number of parameters. It can be downloaded for offline use, used in cloud or from HuggingFace. Using HuggingFaceHub from langchain you can load and use Mistral-7B using following code:

repo_id = "mistralai/Mistral-7B-v0.1"

llm = HuggingFaceHub(huggingfacehub_api_token='your huggingface access token here', 

                     repo_id=repo_id, model_kwargs={"temperature":0.2, "max_new_tokens":50})

An embedding is a numerical representation of a piece of data in the form of multidimentional floating point type vectors. You can have embeddings for text, images, audio, video, documents etc. An embedding is not just a numerical representation, it is a numerical representation that captures the contextual and semantic meaning of the data it represents.

A pretrained model can be used for creating embeddings. Sentence Transformers library from HuggingFace offers many such models. You can install it like this:

# Install dependencies

!pip install huggingface_hub

!pip install chromadb

!pip install langchain

!pip install pypdf

!pip install sentence-transformers

# import required libraries

from langchain.document_loaders import PyPDFLoader

from langchain.text_splitter import CharacterTextSplitter

from langchain.embeddings import HuggingFaceEmbeddings

from langchain.llms import HuggingFaceHub

from langchain.vectorstores import Chroma

from langchain.chains import ConversationalRetrievalChain

# Load the pdf file and split it into smaller chunks

loader = PyPDFLoader('report.pdf')

documents = loader.load()

# Split the documents into smaller chunks 

text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)

texts = text_splitter.split_documents(documents)

# We will use HuggingFace embeddings 

embeddings = HuggingFaceEmbeddings()

#Using Chroma vector database to store and retrieve embeddings of our text

db = Chroma.from_documents(texts, embeddings)

retriever = db.as_retriever(search_kwargs={'k': 2})

# We are using Mistral-7B for this question answering 

repo_id = "mistralai/Mistral-7B-v0.1"

llm = HuggingFaceHub(huggingfacehub_api_token='your huggingface access token here', 

                     repo_id=repo_id, model_kwargs={"temperature":0.2, "max_new_tokens":50})

# Create the Conversational Retrieval Chain

qa_chain = ConversationalRetrievalChain.from_llm(llm, retriever,return_source_documents=True)

#We will run an infinite loop to ask questions to LLM and retrieve answers untill the user wants to quit

import sys

chat_history = []

while True:

    query = input('Prompt: ')

    #To exit: use 'exit', 'quit', 'q', or Ctrl-D.",

    if query.lower() in ["exit", "quit", "q"]:

        print('Exiting')

        sys.exit()

    result = qa_chain({'question': query, 'chat_history': chat_history})

    print('Answer: ' + result['answer'] + '\n')

    chat_history.append((query, result['answer']))

References

https://medium.com/@nimritakoul01/chat-with-your-pdf-files-using-mistral-7b-and-langchain-f3be9363301c