Module 3 · Lesson 1 of 7
Cutting documents into chunks
Cut it up, then search
A long document is cut into short, slightly overlapping chunks, and each chunk gets its own embedding.
Scene 1 of 4 · One long policy
The refund policy has 15,000 words: far too long and too mixed to match one question.
- Refund policy — healthy — 15,000 words
- Chunker — healthy
- Vector index — healthy
- Chunks for this policy
- not shown yet
- One long policy. The refund policy has 15,000 words: far too long and too mixed to match one question.
- Cut into chunks. The chunker cuts it into pieces of 200 new words each: 75 chunks in all.
- Overlap the edges. Each chunk also repeats the last sentence of the one before, so no idea is cut in half.
- One embedding each. Each chunk gets its own embedding, so the index holds 75 of them for this one policy.
- Now you try. Change the chunk size, and see how many chunks the 15,000-word policy becomes. For example: 25 chunks of 600 new words, over this example's 500-word guide. Chunks this big mix several topics, so each embedding is a blur of all of them.
Scene 1 of 4: One long policy. The refund policy has 15,000 words: far too long and too mixed to match one question.
Key points
- A whole document is too long and too mixed to match one question.
- Each chunk holds a few paragraphs about one idea.
- Repeating a sentence across the cut keeps ideas whole.
- Every chunk gets its own embedding in the index.
Before a document can be searched by meaning, it is cut into short pieces called (A short passage cut from a longer document, so it can be turned into an embedding and found on its own.), often a few paragraphs each. Each chunk gets its own (A list of numbers that stands for the meaning of a piece of text. Texts with similar meanings get similar lists.).
A chunk that is too long mixes several topics, so its embedding is vague and matches poorly. A chunk that is too short loses the sentences around it that explain it.
Many systems let neighbouring chunks overlap by a sentence or two. Then an idea split across a boundary still appears whole in one chunk.
In system design: Chunk size and overlap are choices you tune by testing real questions, not fixed rules.