ended3월 21일· 1 sources
How to Deduplicate 100,000 Records in 13 Seconds with Python
Python으로 10만 건의 레코드를 13초 만에 중복 제거하는 방법
Why it matters
GoldenMatch is a Python library that deduplicates large CSV datasets by auto-detecting column types, applying blocking strategies to avoid O(n²) comparisons, and scoring pairs using vectorized fuzzy matching with optional LLM-based scoring for borderline cases. It clusters matches via Union-Find and produces golden records through configurable merge strategies, processing 100,000 records in about 13 seconds.
1
Sources
+0
24h
—
Growth
182d
Active
GoldenMatchdeduplicationfuzzy matchingrecord linkagePython