My co-author and I are working on a Python script for automatic family reconstitution using historical civil registration. Currently, we use the Jaro-Winkler algorithm and fuzzy logic to match data (birth, marriage, and death records). While both approaches have their strengths, neither provides perfect results. Has anyone worked on similar challenges and can recommend alternative methods to improve linkage accuracy?
Hello,
You may try this Python package: https://recordlinkage.readthedocs.io/en/latest/
It uses principles of probabilistic record linkage
Hi dear Szymon Antosik,
I hope this answer find you well. Szymon Antosik
To improve the accuracy of linking historical civil registration data, consider exploring alternative methods like Levenshtein distance for more flexible string matching, or machine learning techniques such as Random Forests or Gradient Boosting Machines, which can incorporate multiple features like dates and locations. Probabilistic record linkage, as implemented in tools like PyRecordLinkage, assigns match probabilities based on various attributes. Additionally, using more contextual information (e.g., birth dates or parental names) and preprocessing steps like blocking or clustering can reduce false matches. Libraries like Dedupe also offer machine learning-powered record linkage, which could help refine your results. Combining these approaches may lead to better overall accuracy.
- Badges
- Science topic
- Similar topics
- Mathematics