Document Extraction

I thought I had solved document extraction.

My Node.js project could pull text out of a PDF. The library was free. No API bill, no config, no drama. I ran it on a few sample docs, the text came back clean, and I moved on to the "real" parts of the pipeline — chunking, embeddings, retrieval.

Everything looked solved.

Until I uploaded a real corporate document.

The PDF opened fine. No errors. Text came back — a whole string of it, sitting right there in my terminal. Extraction "worked."

And then my RAG system answered a simple question completely wrong.

I did what anyone does — I blamed the usual suspects. Checked the LLM prompt. Checked the embeddings. Checked retrieval, checked the vector DB, checked if the right chunks were even being pulled.

All fine.

The bug wasn't in my LLM. It wasn't in my embeddings. It wasn't in retrieval.

It started much earlier than any of that.

The parser had quietly destroyed the document before the rest of my pipeline ever got a chance to see it.

That was the moment it clicked for me: document extraction was never about "reading a PDF." It's about preserving knowledge before anything downstream can use it. And preserving knowledge, it turns out, costs money.

The Idea This Whole Article Is About

Everything below comes back to one sentence:

Document extraction is the first place in the RAG pipeline where you make a quality-vs-cost decision — whether you realize you're making it or not.

Not "which library should I use." Not "OCR is hard." Just this: the moment a document enters your system, you're already choosing between cheap-and-lossy or expensive-and-faithful, and that choice quietly sets the ceiling for every stage that comes after it.

How a "Free" Parser Bill Actually Gets Paid

Here's the sequence that fooled me, laid out plainly:

Free parser
     │
     ▼
Table becomes word soup / column order scrambled
     │
     ▼
Chunk looks fine (no error, just wrong)
     │
     ▼
Embedding of a wrong chunk
     │
     ▼
Retrieved anyway — vector search doesn't know it's damaged
     │
     ▼
LLM answers confidently... and wrong
     │
     ▼
User asks again, rephrases, retries
     │
     ▼
More LLM calls, more tokens, more cost

Enter fullscreen mode Exit fullscreen mode

Notice what's missing from that chain: an error message. A broken extraction almost never crashes anything. It just silently produces a confident, well-formatted, wrong answer — and the bill for that mistake doesn't show up on the extraction line. It shows up later, on the LLM line, disguised as "users retrying" or "hallucination rate."

That's the trap. The cost didn't disappear when you picked the free parser. It just moved to the most expensive stage in the whole pipeline and hid there.

Where Does The Money Actually Go?

Let's make this concrete instead of abstract.

Say you're processing 1 million pages a month.

Option A — Free, open-source parser 

API bill:  $0/month

Enter fullscreen mode Exit fullscreen mode

But:

  • Tables lose their structure
  • OCR on scanned pages fails silently on a noticeable chunk of them
  • Diagrams and screenshots vanish into <Image> with zero text
  • Multi-column pages get scrambled

Result:

Bad extraction
     ↓
Users don't get their answer
     ↓
They ask again (2nd LLM call)
     ↓
They rephrase and ask a third time (3rd LLM call)
     ↓
Support ticket, because the bot "doesn't know" a document it clearly ingested

Enter fullscreen mode Exit fullscreen mode

Your extraction bill is $0. Your effective cost per successfully-answered question just went up 2-3x, because you're paying for the LLM call three times to get one right answer.

Option B — Enterprise document intelligence 

Extraction bill: suppose it costs a few hundred dollars a month more

Enter fullscreen mode Exit fullscreen mode

But:

  • The large majority of tables, forms, and layout survive intact
  • Diagrams get described in usable text
  • Multi-column reading order is preserved

Result:

Good extraction
     ↓
Right chunk retrieved the first time
     ↓
One LLM call, correct answer
     ↓
No retries, no ticket

Enter fullscreen mode Exit fullscreen mode

Now run that logic across volume. If even a modest slice of queries under Option A need a retry — and each retry is a full LLM round-trip — the "free" parser can plausibly end up costing more in extra LLM calls than the paid extraction service would have cost outright. And that's before counting the users who just give up and stop trusting the bot.

In that scenario, the "expensive" parser doesn't stay the expensive choice. It becomes the cheaper system.

That's the whole game of this series: the line item you're worried about is rarely the one actually draining your budget.

There's another bill nobody invoices you for: engineering time. Every document type that breaks — a weird table layout, a scanned page with faint ink, a two-column report — becomes another edge case someone on your team has to notice, debug, test, and maintain. A free parser isn't really free if it quietly costs your team weeks of patching extraction bugs instead of building the product.

The Cost Nobody Writes About: Repeated Boilerplate

This is the part that changed how I think about extraction the most, and I almost skipped writing about it because it seemed too small to matter. It isn't.

Imagine every page of a 400-page bank document has this footer:

Confidential
Acme Bank
2025

Enter fullscreen mode Exit fullscreen mode

Trivial, right? Except your parser doesn't know it's a footer. It just sees text, on every single page.

Page 1  →  "Confidential Acme Bank 2025" + real content
Page 2  →  "Confidential Acme Bank 2025" + real content
Page 3  →  "Confidential Acme Bank 2025" + real content
   ...
Page 400 → "Confidential Acme Bank 2025" + real content

Enter fullscreen mode Exit fullscreen mode

That one sentence just got embedded 400 times. It now exists as 400 near-identical vectors sitting in your database, all clustered close together in embedding space.

Here's where it actually bites you: when someone asks a real question, "Confidential Acme Bank 2025" chunks are dense, repetitive, and semantically generic enough that they can crowd out the top-k slots that should've gone to your actual content. Retrieval keeps returning noise instead of knowledge — not because retrieval is broken, but because you fed it 400 copies of nothing.

Uncleaned boilerplate
       ↓
400 near-duplicate chunks
       ↓
Bloated vector index
       ↓
Noisy top-k retrieval
       ↓
Wasted context tokens sent to the LLM
       ↓
Higher bill, worse answers

Enter fullscreen mode Exit fullscreen mode

Stripping headers, footers, page numbers, and watermarks before chunking isn't a cosmetic cleanup step. It's a cost optimization — and it's one almost nobody talks about, because it doesn't show up on any pricing page.

Other Repeat Offenders

Once you start looking for this pattern, it shows up everywhere in corporate documents:

  • Repeated logos
  • Repeated page numbers ("Page 47 of 212")
  • Repeated watermarks
  • Repeated disclaimers ("This document is confidential and intended only for...")
  • Repeated signature blocks
  • Repeated legal boilerplate at the top or bottom of every page

Each one follows the exact same chain:

More boilerplate  →  More chunks  →  More embeddings
        →  Bigger vector DB  →  More retrieval noise
        →  More LLM tokens  →  More money

Enter fullscreen mode Exit fullscreen mode

None of these individually feel expensive. A logo isn't expensive. A page number isn't expensive. But multiply a few of these by hundreds of pages and thousands of documents, and you've quietly inflated your vector database and your retrieval noise for zero added knowledge. This is the kind of decision that's invisible in a demo with 3 sample PDFs and very visible once you're running 50,000 real documents in production.

The Other Quality-vs-Cost Fork: Images and Diagrams

Corporate documents today aren't just paragraphs of text. They're full of:

  • Architecture diagrams
  • Screenshots
  • Invoices
  • Graphs and charts

And here you hit the exact same fork in the road as everywhere else in this pipeline:

                Document contains an image
                         │
            ┌────────────┴────────────┐
            ▼                         ▼
      Ignore it                Run OCR + Vision
            │                         │
            ▼                         ▼
        Free                  Higher extraction cost
            │                         │
            ▼                         ▼
   Knowledge is gone          Knowledge is preserved
   ("Show me the payment      ("Show me the payment
    flow" → no answer)         flow" → correct answer)

Enter fullscreen mode Exit fullscreen mode

Ignoring images is free right up until someone asks a question whose answer only exists inside a diagram. Then "free" extraction just means "the bot can't answer that," and you're back to the same retry loop as the table example above — except this time, no amount of retrying will fix it, because the knowledge was never captured in the first place.

Same fork, same trade-off, same underlying question every time: are you willing to pay upfront for extraction quality, or are you going to pay downstream for the gaps it leaves behind?

Bringing It Back to One Sentence

Every section in this article — tables, OCR, headers and footers, logos, diagrams — is really the same decision wearing a different costume:

You either pay a little more at extraction to preserve knowledge, or you pay a lot more later — in retries, in noisy retrieval, and in LLM tokens — to make up for what extraction quietly threw away.

Cheap extraction isn't actually cheap. It just defers the bill to the most expensive part of your pipeline and hides it inside a metric that looks unrelated, like "hallucination rate" or "average tokens per query."

That's the real reason document extraction deserves its own episode in a series about where RAG money goes. It's not the biggest line item. But it's the stage that decides how big every other line item gets to be.

I started this series believing embeddings were where RAG became expensive. Two episodes in, I'm realizing something very different — every stage before the LLM quietly decides how much the LLM will eventually cost you.

The cheapest parser was never the one with the lowest API price. It's the one that produces the fewest unnecessary LLM calls.

Coming up in Episode 3

I trust my extraction pipeline a lot more now. But a new question showed up immediately:

Once I have clean text — how should I even split it?

Because bad chunking can destroy perfectly extracted knowledge just as easily as a bad parser can.

Less noise, more action. Let's dig.