Better PII Redaction

PII redaction looks simple until real text arrives. Names appear without labels, account numbers resemble harmless identifiers, phone numbers come in many formats, and business documents mix structured facts with free-form language. A reliable redaction pipeline needs more than one detector.

A practical approach is to combine Microsoft Presidio with GLiNER. Presidio gives you a mature framework for detecting, scoring, explaining, and anonymising sensitive entities. GLiNER adds flexible named-entity recognition that can identify custom categories from natural-language labels without training a new model for every project.

Why Presidio?

Presidio is useful because it treats redaction as an engineering system, not just a model call. It includes recognisers for common data types such as email addresses, phone numbers, credit-card numbers, IP addresses, URLs, and national identifiers. It also provides an analyser engine, confidence scores, context-aware detection, and an anonymiser engine for replacing, masking, hashing, or encrypting detected values.

This makes Presidio a strong foundation for compliance-heavy workflows. You can inspect why something was detected, tune thresholds, add deny lists, define custom recognisers, and apply different anonymisation strategies depending on the entity type.

Where GLiNER helps

Rule-based and pattern-based detection works well for structured identifiers, but it can miss soft PII: people, organisations, locations, job titles, patient names, customer references, or internal project names. This is where GLiNER is valuable.

GLiNER can detect entities based on labels you provide, such as person name, company name, medical condition, bank name, or internal employee identifier. That flexibility helps teams adapt redaction to their own documents without building a fully supervised NER dataset first.

A hybrid architecture

A robust pipeline usually looks like this:

1. Normalise input: extract text from PDFs, emails, tickets, logs, chat transcripts, or documents.
2. Run Presidio recognisers: detect structured PII such as emails, phone numbers, card numbers, IPs, URLs, and known identifiers.
3. Run GLiNER: detect contextual entities such as names, organisations, addresses, projects, departments, or domain-specific sensitive terms.
4. Merge spans: deduplicate overlapping detections and keep the strongest confidence score or most sensitive entity type.
5. Apply redaction policy: replace entities with tags such as <PERSON>, mask partial values, hash identifiers, or remove them entirely.
6. Log safely: store only metadata about the redaction process, not the original sensitive text.

Example detection strategy

Presidio should handle high-precision recognisers first. For example, emails, payment card numbers, phone numbers, and IP addresses are usually better detected with specialised logic than with a general NER model. GLiNER can then add broader semantic coverage for labels such as:

• person name
• home address
• company name
• customer identifier
• medical record reference
• financial institution
• internal project name

The key is not to let every model output become an automatic deletion. Use confidence thresholds, entity-specific rules, and document context. A customer support ticket, a medical note, and a firewall log each need different redaction policies.

Handling overlapping entities

Hybrid detection creates overlaps. A phrase such as Jane Smith, Acme Bank might be detected by GLiNER as a person and organisation, while Presidio may also recognise part of the text through a custom recogniser. The redaction layer should merge overlaps before modifying the text. Otherwise, offsets can shift and produce broken redactions.

A simple policy is to sort spans by start position, prefer longer spans when confidence is similar, and preserve the most sensitive label when two detections overlap. For regulated data, it is usually safer to over-redact than to leak sensitive information.

Choosing anonymisation methods

Not every use case needs the same output. For analytics, replacing names with stable hashes may preserve useful grouping without exposing identities. For logs sent to an LLM, simple tags like <EMAIL> or <PERSON> may be enough. For documents shared externally, irreversible removal may be the safest option.

• Replace: [email protected] becomes <EMAIL>.
• Mask: +44 7700 900123 becomes +44 77** ***123.
• Hash: identifiers become consistent pseudonyms for analytics.
• Encrypt: sensitive values can be recovered by authorised systems only.

Testing redaction quality

Redaction should be tested like any other security-sensitive feature. Build a test set with realistic examples, edge cases, false positives, and false negatives. Include messy formats, OCR errors, abbreviations, international phone numbers, names with punctuation, and domain-specific identifiers.

Track precision and recall separately. Low precision means useful text is removed unnecessarily. Low recall means sensitive data leaks. Most organisations should optimise recall first, then reduce false positives through better recognisers, labels, thresholds, and allow lists.

Common pitfalls

• Relying only on regex: patterns miss contextual PII such as names and organisations.
• Relying only on NER: models can miss structured identifiers that deterministic recognisers catch well.
• Ignoring offsets: redacting text in the wrong order can corrupt spans.
• Logging raw input: debug logs can accidentally become a second PII store.
• Using one policy everywhere: different document types need different thresholds and labels.

Final thought

Presidio and GLiNER are stronger together than either is alone. Presidio provides the redaction framework, explainability, and anonymisation controls. GLiNER adds flexible semantic detection for the sensitive entities that rules often miss. Together, they form a practical foundation for protecting data before it enters analytics systems, search indexes, support tools, or LLM workflows.