Why BERT-NER Fails on Healthcare Data: Comparing PII Detection Approaches

This benchmark reveals critical performance gaps in PII detection for medical and business environments. BERT-NER, a general-purpose NER model, achieves only F1=0.167 on healthcare documents—near total failure—while Ensemble methods reach F1=0.662 for standard formats. For production systems, the choice depends on infrastructure: GPU-enabled Piiranha offers superior accuracy (F1=0.9866), while Regex remains the fastest solution for well-formatted data structures.