TextBases

Copying text from a PDF often produces something like this:

The copied paragraph looks normal
inside the PDF, but every visual line
becomes a hard line break after pasting.

The next paragraph should remain
separate from the first one.

Enter fullscreen mode Exit fullscreen mode

What you usually want is this:

The copied paragraph looks normal inside the PDF, but every visual line becomes a hard line break after pasting.

The next paragraph should remain separate from the first one.

Enter fullscreen mode Exit fullscreen mode

The important part is not simply “remove all line breaks.”

The real goal is:

Join wrapped lines inside a paragraph while preserving line breaks that carry meaning.

That distinction matters because PDF text can contain paragraphs, headings, lists, citations, addresses, tables, code, and multi-column layouts. A cleanup rule that works perfectly for prose can damage structured content.

Why copied PDF text contains hard line breaks

A PDF is primarily a page-layout format.

Text may be positioned using coordinates, columns, text boxes, captions, footnotes, and visual line endings. When you copy it, the clipboard may preserve those visual boundaries as newline characters.

Depending on the source and operating system, the pasted text may contain:

  • \n — line feed
  • \r\n — carriage return plus line feed
  • blank lines represented by two or more newline sequences

A normal paragraph can therefore become a list of short physical lines even though it should be one continuous block of text.

Why the obvious replacement is dangerous

A common first attempt is:

text.replace(/\s+/g, " ")

Enter fullscreen mode Exit fullscreen mode

This replaces every whitespace sequence with one space.

It may look convenient, but it also destroys:

  • paragraph boundaries
  • list formatting
  • code indentation
  • addresses
  • table-like text
  • intentionally separated sections

Another common attempt is:

text.replace(/\r?\n/g, " ")

Enter fullscreen mode Exit fullscreen mode

This preserves tabs and some other whitespace, but it still merges every line, including real paragraph breaks.

For plain prose, a better strategy is to protect paragraph boundaries first.

A paragraph-preserving JavaScript approach

function cleanPdfText(input) {
  return input
    // Normalize Windows and old Mac line endings
    .replace(/\r\n?/g, "\n")

    // Temporarily protect blank-line paragraph boundaries
    .replace(/\n[\t ]*\n+/g, "__PARAGRAPH_BREAK__")

    // Join remaining single line breaks
    .replace(/[\t ]*\n[\t ]*/g, " ")

    // Normalize repeated spaces
    .replace(/[ \t]{2,}/g, " ")

    // Restore paragraphs
    .replace(/__PARAGRAPH_BREAK__/g, "\n\n")

    .trim();
}

Enter fullscreen mode Exit fullscreen mode

Example:

const copied = `The copied paragraph looks normal
inside the PDF, but every visual line
becomes a hard line break after pasting.

The next paragraph should remain
separate from the first one.`;

console.log(cleanPdfText(copied));

Enter fullscreen mode Exit fullscreen mode

Output:

The copied paragraph looks normal inside the PDF, but every visual line becomes a hard line break after pasting.

The next paragraph should remain separate from the first one.

Enter fullscreen mode Exit fullscreen mode

This is still a heuristic, not a full PDF-layout parser. It assumes that blank lines represent real paragraph boundaries and single newlines represent wrapped prose.

The same idea in Python

import re


def clean_pdf_text(text: str) -> str:
    normalized = text.replace("\r\n", "\n").replace("\r", "\n")

    protected = re.sub(
        r"\n[\t ]*\n+",
        "__PARAGRAPH_BREAK__",
        normalized,
    )

    joined = re.sub(
        r"[\t ]*\n[\t ]*",
        " ",
        protected,
    )

    compact = re.sub(r"[ \t]{2,}", " ", joined)

    return compact.replace(
        "__PARAGRAPH_BREAK__",
        "\n\n",
    ).strip()

Enter fullscreen mode Exit fullscreen mode

The logic is intentionally simple:

  1. Normalize newline styles.
  2. Protect blank-line paragraph boundaries.
  3. Replace remaining single newlines with spaces.
  4. Normalize repeated spaces.
  5. Restore the paragraphs.

Watch for hyphenated words

PDF text can also split a word across lines:

The formatter should preserve mean-
ing while cleaning the copied text.

Enter fullscreen mode Exit fullscreen mode

Blindly joining the lines produces:

The formatter should preserve mean- ing while cleaning the copied text.

Enter fullscreen mode Exit fullscreen mode

You might be tempted to remove every hyphen followed by a line break:

text.replace(/-\n/g, "")

Enter fullscreen mode Exit fullscreen mode

That can work for some words, but it can also break valid hyphenated terms.

For example:

client-
side

Enter fullscreen mode Exit fullscreen mode

may be intended as client-side, not clientside.

A safer workflow is to flag these cases for review rather than assuming every end-of-line hyphen should disappear.

You can detect them with:

const possibleSplitWords = text.match(/[A-Za-z]-\r?\n[A-Za-z]/g) ?? [];

Enter fullscreen mode Exit fullscreen mode

Then review the matches manually.

Cases that should not be cleaned automatically

Do not apply a prose-oriented newline rule blindly to:

Lists

- Install the dependency
- Run the build
- Check the output

Enter fullscreen mode Exit fullscreen mode

Addresses

TextBases
123 Example Street
Singapore

Enter fullscreen mode Exit fullscreen mode

Code

if (ready) {
  start();
}

Enter fullscreen mode Exit fullscreen mode

Citations and references

Line placement may separate authors, publication titles, page numbers, or identifiers.

Tables and financial data

Columns can collapse into one unreadable sentence.

Poetry, lyrics, and legal clauses

Line boundaries may be part of the intended meaning or structure.

Multi-column PDFs

Copied text can jump between columns in the wrong order. Removing line breaks cannot reconstruct the original reading sequence.

A safer cleanup workflow

A reliable workflow is intentionally conservative:

  1. Copy one small section from the PDF.
  2. Check whether the text is ordinary prose or structured content.
  3. Preserve blank lines that represent real paragraphs.
  4. Join only the wrapped lines inside those paragraphs.
  5. Review end-of-line hyphens.
  6. Check punctuation and spacing.
  7. Compare the result with the source PDF.
  8. Repeat section by section.

For quick prose cleanup, I use the free TextBases Remove Line Breaks tool. It is useful when I do not need to write a script, while the code examples above are better when the cleanup needs to be part of a repeatable workflow.

A useful decision rule

Use automatic cleanup when:

  • the input is ordinary paragraph text
  • single newlines are clearly caused by visual wrapping
  • blank lines reliably separate paragraphs
  • you can compare the output with the source

Use manual review when:

  • formatting carries meaning
  • the PDF contains columns, tables, citations, or footnotes
  • words are split across line endings
  • the text will be used in legal, financial, medical, academic, or customer-facing work

Final thought

Removing line breaks is easy.

Knowing which line breaks should remain is the actual problem.

The safest solution is not the most aggressive regex. It is a small, understandable transformation that preserves paragraphs and makes uncertain cases visible for human review.