LangChain · RAG · Sahaj's blog

Retrieval-augmented generation end to end with LCEL: retrieve chunks from my own notes, stuff them into a grounded prompt, answer with a local model, and return the sources with the answer.

Retrieval-augmented generation (Lewis et al., 2020) gives a model knowledge it wasn’t trained on, at answer time:

Index (offline): load, split, embed, store. That’s the last three posts.
Retrieve: embed the question and fetch the top-k chunks.
Generate: put the chunks in a prompt that says “answer only from this context”, and call the model.

Two habits from building it:

tell the model to say “I don’t know” when the context doesn’t contain the answer
return the sources with the answer, so it can be checked (this whole blog runs on the same rule)

The knowledge base here is a handful of facts from my own learning-path page. The model is the local Qwen2.5-0.5B-Instruct with greedy decoding.

import torch
from langchain_core.documents import Document
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableParallel, RunnablePassthrough, RunnableLambda
from langchain_huggingface import HuggingFaceEmbeddings, HuggingFacePipeline, ChatHuggingFace
torch.set_num_threads(4)

notes = [
    ("Phase 1 of my learning path was Python, from August to October 2023.", "learning-path#phase-1"),
    ("Phase 2 was C++, from November 2023 to January 2024, ending with a linked list.", "learning-path#phase-2"),
    ("Phase 7, machine learning, ran from August to December 2024 and ended with hyperparameter search.", "learning-path#phase-7"),
    ("Phase 13 is LangChain, in April and May 2026.", "learning-path#phase-13"),
]
store = InMemoryVectorStore.from_documents(
    [Document(t, metadata={"source": s}) for t, s in notes],
    HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2"))
retriever = store.as_retriever(search_kwargs={"k": 2})

chat = ChatHuggingFace(llm=HuggingFacePipeline.from_model_id(
    model_id="Qwen/Qwen2.5-0.5B-Instruct", task="text-generation",
    pipeline_kwargs={"max_new_tokens": 40, "do_sample": False, "return_full_text": False}))
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer using only the context. If the answer is not in the context, say \"I don't know\".\n\nContext:\n{context}"),
    ("human", "{question}")])
fmt = RunnableLambda(lambda docs: "\n".join(d.page_content for d in docs))

rag = RunnableParallel(docs=retriever, question=RunnablePassthrough()).assign(
    answer=RunnableLambda(lambda x: {"context": fmt.invoke(x["docs"]), "question": x["question"]}) | prompt | chat | StrOutputParser())

for q in ["Which language did I study in phase 2?", "What did phase 7 end with?", "What is my favourite food?"]:
    out = rag.invoke(q)
    print(f"Q: {q}\nA: {out['answer'].strip()}\n   sources: {[d.metadata['source'] for d in out['docs']]}")

Output:

Q: Which language did I study in phase 2?
A: C++
   sources: ['learning-path#phase-1', 'learning-path#phase-2']
Q: What did phase 7 end with?
A: phase 7 ended with hyperparameter search.
   sources: ['learning-path#phase-7', 'learning-path#phase-2']
Q: What is my favourite food?
A: I don't know.
   sources: ['learning-path#phase-1', 'learning-path#phase-13']

LangChain · RAG

Sources