# Chunking Strategy Matters More Than Your Embedding Model

> Three chunking changes that moved retrieval quality more than swapping models did.

- **Author:** [Saanvi Kulkarni (@saanvi_kulkarni)](https://www.promptabide.com/saanvi_kulkarni)
- **Published:** 2026-05-15
- **Updated:** 2026-08-22
- **Tags:** `ml`, `rag`, `embeddings`, `retrieval`
- **Views:** 6901
- **Likes:** 8

## Prompt

```
I spent two weeks A/B testing embedding models and got a 2% recall improvement. Then I changed how I chunk and got 19%.

**What moved the needle:**

1. **Chunk on structure, not character count.** Splitting a markdown doc at headings beats a 512-character window, because a window cuts mid-sentence and the fragment embeds as noise.
2. **Prepend the breadcrumb to every chunk.** "Billing > Refunds > Partial refunds\n\n<chunk text>". Retrieval on a chunk that says "this must be approved by a manager" is hopeless without knowing what "this" is.
3. **Overlap only across the sentence boundary**, not a fixed token count. Fixed overlap duplicates whole paragraphs and skews your top-k towards long documents.

**What did not matter:** the embedding model, once past a reasonable baseline. Also chunk size, within a wide range, once the splits were structural.

Measure recall@10 before and after each change on a fixed question set. If you change two things at once you will learn nothing.
```

## Output

*Produced by ChatGPT · gpt-4-turbo.*

```
recall@10 baseline (512-char window):        0.61
+ structural splits:                         0.70
+ breadcrumb prefix:                         0.78
+ sentence-boundary overlap:                 0.80
swapping embedding model (on top of above):  0.82
```

---

Canonical HTML: https://www.promptabide.com/bides/chunking-beats-embedding-model-saanvi-kulkarni-3
Agent guide: https://www.promptabide.com/llms.txt · https://www.promptabide.com/agent-instructions.md
Sitemap: https://www.promptabide.com/sitemap.xml