What are some of the basic RAG techniques With LlamaIndex - Part 1 - Prompt Engineering

Prompt Engineering

If you're encountering failures related to the LLM, like hallucinations or poorly formatted outputs, then this should be one of the first things you try.

Customizing Prompts => Most of the prebuilt modules having prompts inside, these can be queried and viewed and also can be updated as required 

documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine(response_mode="tree_summarize")

# define prompt viewing function

def display_prompt_dict(prompts_dict):

    for k, p in prompts_dict.items():

        text_md = f"**Prompt Key**: {k}<br>" f"**Text:** <br>"

        display(Markdown(text_md))

        print(p.get_template())

        display(Markdown("<br><br>"))

prompts_dict = query_engine.get_prompts()

# from response synthesiser 

prompts_dict = query_engine.response_synthesizer.get_prompts()

display_prompt_dict(prompts_dict)

query_engine = index.as_query_engine(response_mode="compact")

prompts_dict = query_engine.get_prompts()

display_prompt_dict(prompts_dict)

response = query_engine.query("What did the author do growing up?")

print(str(response))

For customizing the prompt, it can be done like below 

from llama_index.core import PromptTemplate

# reset

query_engine = index.as_query_engine(response_mode="tree_summarize")

# shakespeare!

new_summary_tmpl_str = (

    "Context information is below.\n"

    "---------------------\n"

    "{context_str}\n"

    "---------------------\n"

    "Given the context information and not prior knowledge, "

    "answer the query in the style of a Shakespeare play.\n"

    "Query: {query_str}\n"

    "Answer: "

)

new_summary_tmpl = PromptTemplate(new_summary_tmpl_str)

query_engine.update_prompts(

    {"response_synthesizer:summary_template": new_summary_tmpl}

)

Advanced Prompts => 

Partial formatting

Prompt template variable mappings

Prompt function mappings

Partial formatting (partial_format) allows you to partially format a prompt, filling in some variables while leaving others to be filled in later.

This is a nice convenience function so you don't have to maintain all the required prompt variables all the way down to format, you can partially format as they come in.

This will create a copy of the prompt template.

qa_prompt_tmpl_str = """\

Context information is below.

---------------------

{context_str}

---------------------

Given the context information and not prior knowledge, answer the query.

Please write the answer in the style of {tone_name}

Query: {query_str}

Answer: \

"""

prompt_tmpl = PromptTemplate(qa_prompt_tmpl_str)

2. Prompt Template Variable Mappings

Template var mappings allow you to specify a mapping from the "expected" prompt keys (e.g. context_str and query_str for response synthesis), with the keys actually in your template.

This allows you re-use your existing string templates without having to annoyingly change out the template variables.

qa_prompt_tmpl_str = """\

Context information is below.

---------------------

{my_context}

---------------------

Given the context information and not prior knowledge, answer the query.

Query: {my_query}

Answer: \

"""

template_var_mappings = {"context_str": "my_context", "query_str": "my_query"}

prompt_tmpl = PromptTemplate(

    qa_prompt_tmpl_str, template_var_mappings=template_var_mappings

)

Prompt Function Mappings

You can also pass in functions as template variables instead of fixed values.

This allows you to dynamically inject certain values, dependent on other values, during query-time.

qa_prompt_tmpl_str = """\

Context information is below.

---------------------

{context_str}

---------------------

Given the context information and not prior knowledge, answer the query.

Query: {query_str}

Answer: \

"""

def format_context_fn(**kwargs):

    # format context with bullet points

    context_list = kwargs["context_str"].split("\n\n")

    fmtted_context = "\n\n".join([f"- {c}" for c in context_list])

    return fmtted_context

prompt_tmpl = PromptTemplate(

    qa_prompt_tmpl_str, function_mappings={"context_str": format_context_fn}

)

references:

https://docs.llamaindex.ai/en/stable/optimizing/basic_strategies/basic_strategies/