Strip Emails, Phones & Financial PII Out of LLM Training Data Locally

JSONL prompt/completion pair stays valid JSON

A line from a fine-tuning file where the user message contains an email and a phone. Because tags drop inside the existing quotes, each line stays parseable for your loader.

Input (one .jsonl line):
{"messages":[{"role":"user","content":"resend the invoice to mia@acme.io or call 0207 946 0958"},{"role":"assistant","content":"Done."}]}

Output (-scrubbed):
{"messages":[{"role":"user","content":"resend the invoice to [REDACTED_EMAIL] or call [REDACTED_PHONE]"},{"role":"assistant","content":"Done."}]}

Instruction CSV — email and card removed, name kept

A typical instruction-tuning CSV. Two identifiers are tagged, but the customer name is NOT a detected category and passes straight through — the thing teams most often forget.

Input (instructions.csv):
instruction,response
"refund Jane Okafor at jane@x.io on 4111 1111 1111 1111","refunded"

Output (instructions-scrubbed.csv):
instruction,response
"refund Jane Okafor at [REDACTED_EMAIL] on [REDACTED_CARD]","refunded"

# Name 'Jane Okafor' is NOT redacted — no name detector

Synthetic example IDs survive the card pass

Prompts often embed fake 16-digit placeholders. The Luhn gate means only checksum-valid runs become [REDACTED_CARD], so your deliberately-synthetic samples aren't corrupted.

Input (prompts.txt):
Use order 1234567890123456 as the example. Real card 4242 4242 4242 4242 must be hidden.

Output (-scrubbed):
Use order 1234567890123456 as the example. Real card [REDACTED_CARD] must be hidden.

# 1234567890123456 fails Luhn → kept
# 4242424242424242 passes Luhn → redacted

Spreadsheet of leads becomes scrubbed JSON

A leads workbook used as raw training material. The first sheet is flattened to a JSON array of row objects, scrubbed, and returned as JSON — the download keeps the .xlsx name but carries JSON text.

Input: leads.xlsx  (first sheet)
  | Email          | Mobile        |
  | kit@demo.org   | 07700 900123  |

Download: leads-scrubbed.xlsx  (CONTENT is JSON):
[
  {
    "Email": "[REDACTED_EMAIL]",
    "Mobile": "[REDACTED_PHONE]"
  }
]

Coverage check before a training run

After scrubbing, grep the cleaned corpus for the tags and for stray identifiers. A non-zero tag count plus zero stray emails is your go signal; a zero count usually means the file isn't the one you thought.

# confirm tags landed
grep -c "\[REDACTED_" corpus-scrubbed.jsonl.txt
# -> 1428

# confirm no raw emails slipped (should be 0)
grep -Ec "[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}" corpus-scrubbed.jsonl.txt
# -> 0

# names/addresses are NOT covered — handle separately

There is no name, address, or date-of-birth detector — only six shape-matchable categories. Prose identifiers, mailing addresses, and quasi-identifiers pass through to your training run untouched. Treat this tool as a *direct-identifier* scrub, then run the result through the CSV/JSON Data Scrambler for field-level pseudonymisation of names and addresses.

A zero count is valid — but for a corpus you *expect* to contain PII it usually means you scrubbed the wrong file, an already-clean export, or a spreadsheet whose data lives on a sheet other than the first. Re-check the source before assuming the corpus is safe.

The card pass requires a Luhn-valid 13–19 digit run, so synthetic example IDs that fail Luhn survive. This is what keeps your deliberately-synthetic samples usable — but it also means a real card written with a checksum-breaking typo won't be caught. Don't rely on the scrub to fix malformed real data.

The drop area accepts a multi-file selection, but a run processes only files[0]. For a directory of shards, loop them one at a time, or call the server-safe API in your pipeline. Don't assume a multi-file pick produced multi-file output — check the -scrubbed filename.

For .xlsx/.xls/.ods inputs the first sheet is converted to a JSON array of row objects, scrubbed, and returned as that JSON — the filename keeps .xlsx but the content is JSON text, not a rebuilt workbook. If your loader expects binary Excel, save the JSON and re-import, or export to CSV before scrubbing.

Text inputs pass through the tier size gate (Free 10 MB, Pro 100 MB, Pro + Media 500 MB, Developer 2 GB). An oversize file throws File "…" is N MB — exceeds the … limit for your plan. and nothing is scrubbed. Shard the corpus or upgrade. The spreadsheet path reads the workbook directly into the tab, so huge workbooks are memory-bound instead.

Detectors run in a fixed order and the Luhn-gated card pass runs before phone. A 13–19 digit internal ID that coincidentally satisfies Luhn becomes [REDACTED_CARD] — over-redaction rather than a leak, but it can blank out a field you wanted to keep as a training signal. Check ID-bearing columns after a run.

The US SSN detector only matches the strict NNN-NN-NNNN dashed form (with SSA invalid-block rules). A nine-digit run like 123456789 is too ambiguous and is left alone. If your corpus stores SSNs without dashes, normalise them to the dashed form before scrubbing or they'll reach the model.

Anything matching local@domain.tld is replaced, including a sample address you put in docstrings or few-shot examples on purpose. The scrubber can't tell a real contact from an intentional placeholder. In documentation-heavy corpora, review redactions so you don't lose meaningful example addresses.

Redaction destroys the value; if instead you need the raw corpus protected in transit or at rest for a trusted partner, encrypt it with the AES-256 Encryptor (Web Crypto AES-GCM-256, PBKDF2 100k). To verify a downloaded dataset matches its published hash, use the Multi-Hash Fingerprinter.