Building Powerful RAG Applications with Haystack 2.x

RAG LLMs are gaining significant traction in today’s era, where access to information is ever-growing. They offer a powerful way to bridge the gap between the vast potential of LLMs and the need for reliable, factual information. As LLM and information retrieval techniques advance, we can expect RAG applications to become even more sophisticated and ubiquitous.

What is Haystack 2.x?

Haystack is an open-source framework for building production-ready LLM applications, retrieval-augmented generative pipelines, and state-of-the-art search systems that work intelligently over large document collections. Learn more about Haystack and how it works.

Haystack is an end-to-end framework that you can use to build powerful and production-ready Pipelines with Large Language Models (LLMs) for different search use cases. Whether you want to perform retrieval-augmented generation (RAG), question answering, or semantic document search, you can use the state-of-the-art LLMs and NLP models in Haystack to provide custom search experiences and make it possible for your users to query in natural language.

Haystack is built in a modular fashion so that you can combine the best technology from OpenAI, Chroma, Marqo, and other open-source projects, like Hugging Face’s Transformers or Elasticsearch.

Haystack 2.x is a major update to the design of Haystack components, Document Stores, and Pipelines. We believe that the Pipeline concept is a fundamental requirement and an optimal fit for building applications with LLMs. Therefore, Pipelines and Components are still the foundation of Haystack 2.x.

Get Started

Have a look at this blog to learn how to get up and running with Haystack quickly. It contains instructions for installing, and running your first RAG pipeline, and adding data and further resources.

Build your first RAG application

Let’s build your first Retrieval Augmented Generation (RAG) Pipeline and see how Haystack answers questions.

First, install the minimal form of Haystack:

pip install haystack-ai

You only need an OpenAI key as an environment variable,OPENAI_API_KEY

OPENAI_API_KEY="Write a key here"

The following code will load your data to the Document Store, build an RAG pipeline, and ask a question based on the data.

• Import the components and the InMemoryDocumentStore.

import os
from haystack import Pipeline, Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders.answer_builder import AnswerBuilder
from haystack.components.builders.prompt_builder import PromptBuilder

# set the required environment variables
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

• Here, initialize the simple InMemoryDocumentStore, then write the text of each Document into the Store.

document_store = InMemoryDocumentStore()
document_store.write_documents([
    Document(content="My name is Bhavik and I live in India."),
    Document(content="My name is Harsh and I live in the USA."),
    Document(content="My name is Harshil and I live in Germany.")
])

• A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer.

prompt_template = """
Given these documents, answer the question.
Documents:
{% for doc in documents %}
    {{ doc.content }}
{% endfor %}
Question: {{question}}
Answer:
"""

• Additionally, insert your OpenAI API key for the OpenAIGenerator. You can pass any parameters to the components at this stage.

retriever = InMemoryBM25Retriever(document_store=document_store)
prompt_builder = PromptBuilder(template=prompt_template)
llm = OpenAIGenerator()

• Create a pipeline instance and add each component one by one to your pipeline. The order doesn’t matter.

rag_pipeline = Pipeline()

# Add each component one by one to your pipeline. The order doesn't matter.
# At this stage, the Pipeline validates the components without running them yet.
rag_pipeline.add_component("retriever", retriever)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)

# rrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")

• Ask a question and define the results

question = "Who lives in India?"
results = rag_pipeline.run(
    {
        "retriever": {"query": question},
        "prompt_builder": {"question": question},
    }
)

• Print out the answer

print(results["llm"]["replies"])

• Output:

Reference

Haystack Introduction

Conclusion

This document provides a solid foundation for understanding RAG LLMs and how Haystack 2.x empowers you to build them. Explore the provided resources to delve deeper and create your own powerful RAG applications.