Here is a complete, production-ready code guide to building a Retrieval-Augmented Generation (RAG) pipeline using LangChain and Chroma (an open-source vector store). [1, 2, 3]

This script takes a raw text document, splits it into chunks, stores it in a vector database, and queries an LLM using that stored data as context. [4, 5, 6, 7]

📋 Prerequisites

First, install the required LangChain and provider packages: [8, 9, 10]

pip install langchain langchain-community langchain-chroma langchain-openai

Note: Ensure you have your OPENAI_API_KEY set up in your environment variables. [11, 12]

💻 Complete LangChain RAG Implementation

import os
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser
 
# ==========================================
# 1. LOAD AND PREPARE DOCUMENTS
# ==========================================
# Create a dummy text file to act as our private data source
with open("company_policy.txt", "w") as f:
    f.write(
        "Project Quantum is our top-secret AI project. "
        "The project launch date is set for October 15, 2026. "
        "All team members must complete security training by September 1, 2026."
    )
 
# Load the document into LangChain memory
loader = TextLoader("company_policy.txt")
raw_documents = loader.load()
 
# ==========================================
# 2. CHUNK THE TEXT
# ==========================================
# LLMs have context limits, and vector search works best with small, focused snippets.
# We split text into chunks of 500 characters, with a 50-character overlap to keep context intact.
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
documents = text_splitter.split_documents(raw_documents)
 
print(f"Loaded {len(raw_documents)} document(s) and split into {len(documents)} chunk(s).")
 
# ==========================================
# 3. INITIALIZE EMBEDDINGS AND VECTOR STORE
# ==========================================
# Initialize the model that converts text chunks into mathematical vector numbers
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
 
# Create an in-memory Chroma vector store populated with our document chunks
# In production, add `persist_directory="./chroma_db"` to save it to disk.
vector_store = Chroma.from_documents(documents=documents, embedding=embeddings)
 
# Convert the vector store into a 'Retriever' interface to fetch similar vectors easily
retriever = vector_store.as_retriever(search_kwargs={"k": 2}) # Fetches top 2 most relevant chunks
 
# ==========================================
# 4. DEFINE THE LLM & PROMPT
# ==========================================
# Initialize our primary brain (the LLM)
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
 
# Construct a strict prompt template instructing the LLM to only use provided context
prompt_template = """
You are a helpful assistant. Answer the question using ONLY the provided context. 
If you do not know the answer based on the context, say "I don't know".
 
Context:
{context}
 
Question: {question}
Answer:
"""
prompt = ChatPromptTemplate.from_template(prompt_template)
 
# ==========================================
# 5. ASSEMBLE THE RAG CHAIN (LCEL)
# ==========================================
# LangChain Expression Language (LCEL) chains tasks sequentially using the pipe `|` operator.
def format_docs(docs):
    # Helper function to merge retrieved text chunks into a single string block
    return "\n\n".join(doc.page_content for docs in docs)
 
rag_chain = (
    {
        "context": retriever | format_docs, # Pass question to retriever, format outputs as string
        "question": RunnablePassthrough()    # Pass user's raw question directly through untouched
    }
    | prompt                                # Inject context and question into the prompt template
    | llm                                   # Send filled prompt to the OpenAI LLM
    | StrOutputParser()                     # Clean the LLM output, extracting just the text response
)
 
# ==========================================
# 6. RUN THE QUERY
# ==========================================
query = "When is the launch date for Project Quantum and what training is needed?"
print(f"\nUser Query: {query}")
 
# Execute the chain
response = rag_chain.invoke(query)
print(f"AI Response:\n{response}")

🧠 Deep-Dive Explanation of Key Components

🧱 1. Document Loaders & Text Splitters

TextLoader: Converts raw files (.txt, .pdf, .html) into structured LangChain Document objects containing both text content and structural metadata.
RecursiveCharacterTextSplitter: This is the gold standard for text splitting. It looks for natural paragraph breaks (\n\n), sentence breaks (\n), and word spaces ( ) to split your text down to your target chunk_size without cutting words in half. [13, 14, 15, 16, 17]

🧮 2. Embeddings vs. Vector Stores

OpenAIEmbeddings: It converts a human sentence like “Project Quantum launch” into a 1536-dimensional array of numbers representing its literal meaning.
Chroma: When you call from_documents, Chroma takes the text, sends it to OpenAI to get the numeric embeddings, and indexes them. When you query it, it computes the cosine similarity between your question’s numbers and your document’s numbers to instantly find the closest context matches. [18, 19, 20]

🔗 3. LCEL (LangChain Expression Language) [21]

The pipe (|) syntax joins Lego blocks together:

Inputs arrive via .invoke("Your Question").
"question": RunnablePassthrough() copies your text straight forward.
"context": retriever | format_docs takes your text, sends it to Chroma to pull matching chunks, and joins them into one unified string block.
The dictionary feeds into the prompt, which slots the values into {context} and {question}.
The filled template goes to the llm, and StrOutputParser() strips away system metadata so you only get clean text back. [22, 23, 24, 25, 26]