What is Python Poetry

Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you. Poetry offers a lockfile to ensure repeatable installs, and can build your project for distribution.

The installer script is available directly at install.python-poetry.org, and is developed in its own repository. The script can be executed directly (i.e. ‘curl python’) or downloaded and then executed from disk (e.g. in a CI environment).

Linux, macOS, Windows (WSL)

curl -sSL https://install.python-poetry.org | python3 -

This was mostly a straight forward installation 

export PATH="/Users/retheesh/.local/bin:$PATH"

poetry --version

Poetry (version 1.5.1)

references:

https://python-poetry.org/docs/#installing-with-the-official-installer

What is Llama-2 and Llama 2-Chat

Meta has this week released an Open Source version of LLM mode, Llama 2, for public use. The large language model (LLM), which can be used to create a chat GPT like chatbot.

Many believe that Llama 2 is the industry’s most important release since ChatGPT in November 2022.

Llama-2, an updated version of Llama 1, trained on a new mix of publicly available data. Meta increased the size of the pretraining corpus by 40%, doubled the context length of the model, and adopted grouped-query attention. Llama 2 was released with 7B, 13B, and 70B parameters.

Llama 2-Chat, a fine-tuned version of Llama 2 that is optimized for dialogue use cases. The variants of this model have 7B, 13B, and 70B parameters as well.

Pretraining data: The Llama-2 training corpus includes a new mix of data from publicly available sources that does not include data from Meta’s products or services. Removed data from certain sites known to contain a high volume of personal information about private individuals. The model was trained on 2 trillion tokens of data as this provides a good performance–cost trade-off, up-sampling the most factual sources in an effort to increase knowledge and dampen hallucinations.

FineTuning: Llama 2-Chat is the result of several months of research and iterative applications of alignment techniques, including both instruction tuning and RLHF, requiring significant computational and annotation resources. RLHF is a model training procedure that is applied to a fine-tuned language model to further align model behavior with human preferences and instruction following.

references:

https://generativeai.pub/unlocking-data-privacy-how-to-build-your-private-enterprise-data-app-with-private-gpt-and-llama-2-eb50d032d145

PoC with LaMini-Flan LLM model

python -m venv venv

source venv/bin/activate

pip install torch torchvision torchaudio

pip install transformers langchain streamlit==1.24.0

pip install accelerate

All the files can be downloaded from here https://huggingface.co/MBZUAI/LaMini-Flan-T5-248M and click on “Files and Versions”

Need to download all the 11 files 

Below is sample code for demoing this. 

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

from langchain.llms import HuggingFacePipeline

import torch

checkpoint = "./model/"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)

base_model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint,

                                                    device_map='auto',

                                                    torch_dtype=torch.float32)

llm = HuggingFacePipeline.from_model_id(model_id=checkpoint,

                                        task = 'text2text-generation',

                                        model_kwargs={"temperature":0.60,"min_length":30, "max_length":600, "repetition_penalty": 5.0})

                                        

from langchain import PromptTemplate, LLMChain

template = """{text}"""

prompt = PromptTemplate(template=template, input_variables=["text"])

chat = LLMChain(prompt=prompt, llm=llm)

yourprompt = input("Enter your prompt: ")

reply = chat.run(yourprompt)

print(reply) 

references:

https://levelup.gitconnected.com/building-a-local-chatbot-on-your-local-pc-100-offline-100-privacy-b617cc29558b

What is LLMChain, PromptTemplate

A LLMChain is the most common type of chain. It consists of a PromptTemplate, a model (either an LLM or a ChatModel), and an optional output parser. This chain takes multiple input variables, uses the PromptTemplate to format them into a prompt. It then passes that to the model.

Language models take text as input - that text is commonly referred to as a prompt. Typically this is not simply a hardcoded string but rather a combination of a template, some examples, and user input. LangChain provides several classes and functions to make constructing and working with prompts easy.

A prompt template refers to a reproducible way to generate a prompt. It contains a text string ("the template"), that can take in a set of parameters from the end user and generates a prompt.

A prompt template can contain:

instructions to the language model,

a set of few shot examples to help the language model generate a better response,

a question to the language model.

Here's the simplest example:

from langchain import PromptTemplate

template = """\

You are a naming consultant for new companies.

What is a good name for a company that makes {product}?

"""

prompt = PromptTemplate.from_template(template)

prompt.format(product="colorful socks")

References:

https://docs.langchain.com/docs/components/chains/llm-chain#:~:text=A%20LLMChain%20is%20the%20most,passes%20that%20to%20the%20model.

What is TorchVision and TorchAudio

 torchvision and torchaudio are Python packages that are part of the PyTorch ecosystem. PyTorch is an open-source deep learning library developed by Facebook's AI Research lab (FAIR) that provides a flexible and efficient framework for building and training various types of deep neural networks.

torchvision:

torchvision is a package that provides image and video datasets, model architectures, and image transformation utilities for use with PyTorch. It is commonly used in computer vision tasks and helps researchers and practitioners to easily access and work with standard datasets and pre-trained models. Some key components of torchvision include:

Datasets: torchvision.datasets module provides popular image and video datasets such as CIFAR-10, CIFAR-100, MNIST, ImageNet, and more, allowing you to quickly load and use these datasets in your projects.

Transforms: torchvision.transforms module provides a set of common image transformations like resizing, cropping, flipping, normalization, and data augmentation, making it easy to preprocess and augment images before feeding them into a neural network.

Pre-trained Models: torchvision.models module provides pre-trained deep learning models such as ResNet, VGG, AlexNet, etc., which you can use directly or fine-tune on your own tasks.

torchaudio:

torchaudio is a package that provides audio processing functionalities for PyTorch. It is designed to work seamlessly with PyTorch tensors and allows you to work with audio data in the same way as image data in torchvision. Some key functionalities of torchaudio include:

Data I/O: torchaudio provides functions to load and save audio data in various formats, making it easy to work with audio datasets.

Audio Transformations: torchaudio.transforms module offers a range of audio transformations like resampling, time stretching, frequency masking, and spectrogram computation, enabling you to preprocess and augment audio data for deep learning models.

Audio Dataset: torchaudio.datasets module provides access to common audio datasets for tasks like speech recognition and audio classification.

Both torchvision and torchaudio are valuable extensions of PyTorch that streamline the process of working with image and audio data, respectively, and enable users to build and experiment with a wide range of deep learning models in computer vision and audio processing domains.

references:

ChatGPT 

What is all-MiniLM-L6-v2

This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.

Usage (Sentence-Transformers)

pip install -U sentence-transformers

from sentence_transformers import SentenceTransformer

sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')

embeddings = model.encode(sentences)

print(embeddings)

Without sentence-transformers, you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

from transformers import AutoTokenizer, AutoModel

import torch

import torch.nn.functional as F

#Mean Pooling - Take attention mask into account for correct averaging

def mean_pooling(model_output, attention_mask):

    token_embeddings = model_output[0] #First element of model_output contains all token embeddings

    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()

    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for

sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub

tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')

model = AutoModel.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')

# Tokenize sentences

encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings

with torch.no_grad():

    model_output = model(**encoded_input)

# Perform pooling

sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

# Normalize embeddings

sentence_embeddings = F.normalize(sentence_embeddings, p=2, dim=1)

print("Sentence embeddings:")

print(sentence_embeddings)

References:

https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2

What is DistilGPT2

DistilGPT2 (short for Distilled-GPT2) is an English-language model pre-trained with the supervision of the smallest version of Generative Pre-trained Transformer 2 (GPT-2). Like GPT-2, DistilGPT2 can be used to generate text. Users of this model card should also consider information about the design, training, and limitations of GPT-2. 

Model Description: DistilGPT2 is an English-language model pre-trained with the supervision of the 124 million parameter version of GPT-2. DistilGPT2, which has 82 million parameters, was developed using knowledge distillation and was designed to be a faster, lighter version of GPT-2.

references:

https://huggingface.co/distilgpt2