Skip to content

Always use most accurate way of fuzzy matching, greatly improve performance of fuzzy matching - #173

Merged
RandomNinjaAtk merged 1 commit into
RandomNinjaAtk:mainfrom
aaronjwood:optimize-fuzzy-search
Feb 1, 2024
Merged

RandomNinjaAtk merged 1 commit into
RandomNinjaAtk:mainfrom
aaronjwood:optimize-fuzzy-search

Conversation

@aaronjwood

Copy link
Copy Markdown
Contributor

I found the Bash implementation of calculating the Levenshtein distance to take between ~2-5 seconds depending on the input, using the Python module (important to note that it uses Cython) is MUCH faster:

time python -c 'from pyxdameraulevenshtein import damerau_levenshtein_distance; print(damerau_levenshtein_distance("Very Long Album Title: The Hits Part 2 Vol. 2", "Some Other Album Part One: Vol 1."))'
32

real    0m0.009s
user    0m0.009s
sys     0m0.000s
time python -c 'from pyxdameraulevenshtein import damerau_levenshtein_distance; print(damerau_levenshtein_distance("Similar Album Title: Part One Vol 1.", "Similar Album Title: Part Two Vol 2."))'
4

real    0m0.015s
user    0m0.012s
sys     0m0.004s

Because of this I felt we could remove the rough-but-fast-and-early way of detecting if there's a match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants