Why are the column dropdowns empty after uploading my file?

The dropdowns only offer columns with expected names. The keyword column must be named KW, kw, Keyword, keyword, keywords or Keywords, and the volume column must be vol, Vol, Volume, volume, search volume or Search Volume. Rename your headers to one of these before uploading.

Which of two near-duplicate keywords gets kept?

The one that appears first in your file, not the one with the highest volume. The volume column is selected in the UI but never used in the keep/drop decision, so sort your file by search volume descending before uploading if you want the top-volume variant of each duplicate group to survive.

What actually counts as a near duplicate?

Matching uses rapidfuzz token_sort_ratio with a fixed 99 threshold that cannot be changed in the UI. In practice that catches word reorderings ('mens running shoes' vs 'running shoes mens') and near-identical strings, while plurals and small word substitutions usually score below 99 and are kept as separate keywords.

Keyword Deduplication Tool: free Keyword Research tool

#about

Uses rapidfuzz with token_sort_ratio scorer and process.extractOne() to identify near-duplicate keywords at 99-point similarity threshold.

Automatically detects file encoding via chardet (first 100,000 bytes).

Keeps the first occurrence in file order (sort by volume descending before upload to keep the highest volume variant) and exports both processed and dropped keywords for review.

Uses stqdm for progress tracking.

Streamlit app

Platform: Browser-based (no installation required)
Input: CSV or Excel file with keywords and search volumesEncoding auto-detected via chardet
Output: Excel with deduplicated and dropped keywords
Category: Keyword Research

Use cases

Cleaning keyword lists before clustering
Removing near-duplicate keywords from exports
Consolidating keyword research from multiple sources

#features

Rapidfuzz token_sort_ratio with a fixed 99-point threshold
Chardet encoding detection (100KB sample)
Keeps the first occurrence of each duplicate group (sort by volume first to keep top-volume variants)
Column dropdowns require preset header names (Keyword/KW and Volume/vol variants)
stqdm progress bar integration
Two-sheet Excel output via xlsxwriter
Supports CSV and Excel (.xlsx) input

#how-to-use

Name your columns Keyword and Volume (or another preset variant) before upload
Sort the file by search volume descending so the highest volume variants are kept
Upload the CSV or Excel file
Select the keyword and volume columns from the dropdowns
Click Dedupe to run rapidfuzz matching
Download Excel with Processed Keywords and Dropped Keywords sheets

#faq

Why are the column dropdowns empty after uploading my file?: The dropdowns only offer columns with expected names. The keyword column must be named KW, kw, Keyword, keyword, keywords or Keywords, and the volume column must be vol, Vol, Volume, volume, search volume or Search Volume. Rename your headers to one of these before uploading.
Which of two near-duplicate keywords gets kept?: The one that appears first in your file, not the one with the highest volume. The volume column is selected in the UI but never used in the keep/drop decision, so sort your file by search volume descending before uploading if you want the top-volume variant of each duplicate group to survive.
What actually counts as a near duplicate?: Matching uses rapidfuzz token_sort_ratio with a fixed 99 threshold that cannot be changed in the UI. In practice that catches word reorderings ('mens running shoes' vs 'running shoes mens') and near-identical strings, while plurals and small word substitutions usually score below 99 and are kept as separate keywords.

/contact

Want me to run this for you? I run this tool as a managed service, or build something custom around your data. You get the insights without touching the code.

Send me a message →