Chunking
Splitting documents into smaller passages before indexing them for RAG retrieval.
What is Chunking?
A RAG system cannot embed an entire 500-page document as one vector — too coarse, too imprecise. Chunking splits documents into smaller passages (typically 200–1000 tokens) that get embedded and indexed individually.
The strategy matters. **Fixed-size chunking** (every 500 tokens) is simple but cuts mid-sentence. **Recursive chunking** respects paragraphs and sections. **Semantic chunking** uses an embedding model to find topic breaks. **Sentence-window chunking** indexes single sentences but retrieves with surrounding context.
Chunk size is a tradeoff: smaller chunks → more precise retrieval but less context per chunk; larger chunks → more context but vaguer matches. The current best practice is 300–800 tokens with 50–100 token overlap.
Chunking quality is the #1 lever on RAG performance after retrieval model choice. Poor chunking explains 60% of mediocre RAG systems.
A Chennai HR-tech indexed company policies with 2000-token chunks and got vague answers. Switching to recursive 400-token chunks with 80-token overlap raised answer precision from 64% to 88% — same documents, better splits.
Want to master this?
Learn Chunking in a structured cohort
3-month live program with mentors, real projects, and 50+ partner placement support.