Strip Special Characters From PDF-Extracted CSV Data

Justified PDF columns insert soft hyphens that pdfplumber preserves. Invisible in a viewer, they break exact matching. The stripper deletes them, rejoining the word.

Input (soft hyphen shown as ¬):
id,term
1,inter¬national
2,manage¬ment

Output (all boxes on):
id,term
1,international
2,management

Extractors often emit a non-breaking space as a thousands separator. The stripper removes it (only regular space is kept), which collapses the number — usually fine for numeric parsing, but verify it is what you want.

Input (NBSP shown as ~):
id,population
1,12~345
2,1~200~000

Output (all boxes on):
id,population
1,12345
2,1200000

If you wanted '12 345' kept, fold NBSP → space first
with /tool/csv-cleaner.

A bulleted list extracted into a cell carries • markers. They are deleted because they are symbols, leaving the text (and a leading space where the bullet was).

Input:
id,features
1,• Waterproof • Wireless

Output (all boxes on):
id,features
1, Waterproof  Wireless

Tidy the spacing with /tool/csv-whitespace-trimmer.

A frequent misconception: that the tool turns fi into fi. It does not — fi (U+FB01) is a Unicode letter, so the keep-list preserves it unchanged. If you need real fi, normalise upstream.

Input:
id,word
1,finally
2,floor

Output (all boxes on) — UNCHANGED:
id,word
1,finally
2,floor

Use Unicode NFKC normalisation before export to get
'finally' and 'floor'.

Typeset ranges use em/en dashes (— –), which are not in the kept punctuation set, so they are deleted and the surrounding tokens fuse. Decide whether to replace them with a hyphen instead.

Input:
id,pages
1,pp. 10—24
2,Vol. 3–5

Output (all boxes on):
id,pages
1,pp. 1024
2,Vol. 35

To turn '—' into '-' instead of deleting it, use
/tool/csv-find-replace.

fi (U+FB01), fl (U+FB02), and other typographic ligatures are classified as letters by \p{L}, so they survive unchanged. The tool does NOT turn them into fi/fl. For that, apply Unicode NFKC normalisation before exporting the CSV.

Extractors substitute NBSP (U+00A0) for spacing; the stripper deletes it because only the regular space is kept. 12 345 and New York fuse. If you need the space preserved, fold NBSP to a regular space first with csv-cleaner.

— and – are not in the kept punctuation set (only the ASCII hyphen - is), so 10—20 becomes 1020. To convert a dash to a hyphen instead of deleting it, use csv-find-replace.

If the extractor split a cell across the wrong columns, the stripper cannot repair it — it only edits cell contents, not structure. Re-extract with better table detection (e.g. Camelot lattice mode) or correct the columns manually.

Deleting a • that was followed by a space leaves a leading or double space. The stripper does not collapse whitespace; chain csv-whitespace-trimmer to clean it up.

The first row is protected. If the extractor put artefacts (soft hyphens, NBSPs) in the header cells, they survive. Clean the header separately with csv-find-replace or remove and re-add it.

Periods, commas, and ASCII hyphens are kept punctuation, so 19.99, 1,200, and 2026-01-15 pass through intact as long as Digits and Punctuation stay on. Don't untick those for tabular figures.

€, £, $, %, ±, ×, ÷ are not in the keep-set and are removed, which can strip meaning from financial or scientific tables. Use csv-find-replace if a symbol must be preserved or substituted.

Free is 2 MB / 500 rows; Pro is 100 MB / 100,000 rows. A large multi-page extraction may exceed free — split with csv-row-splitter or upgrade before stripping.

If a cell holds a multi-line value (extracted wrapped text, properly quoted), the newline is not in the keep-set and is deleted, concatenating the lines. If you need to preserve line breaks, do not strip that column.