How to use the List Deduplicator
Remove duplicates from your list in three steps:
1
Paste your list
Paste any list with duplicate items into the input area. Select the delimiter that matches your list format.
2
Configure matching options
Toggle case-sensitive matching, trim whitespace before comparison, keep last occurrence instead of first, and skip empty items.
3
Copy the unique list
Click 'Remove Duplicates' and copy or download the deduplicated output. Stats pills show exactly how many duplicates were removed.
When to use this tool
Use to extract unique items from any list with repeated entries:
- →Deduplicating a list of email addresses or usernames before sending a campaign or importing into a system
- →Removing repeated entries from a scraped list of URLs, product SKUs, or domain names
- →Finding unique values in a comma-separated data export from a spreadsheet or database query
- →Cleaning up a list of tags, keywords, or category names that has accumulated duplicates over time
- →Deduplicating a list of CSS class names or JavaScript import paths before review or commit
- →Removing repeated entries from merged or concatenated lists where duplicates were introduced by the merge
Frequently asked questions
Q:Is the original order of items preserved?
Yes — by default, the first occurrence of each unique item is kept in its original position and all subsequent duplicates are discarded. The remaining items appear in the same relative order they appeared in the original list. Enable 'Keep last occurrence' to instead retain the most recent copy of each duplicate while still preserving relative order.
Q:What does case-insensitive matching do?
With case-insensitive matching (the default), items are compared after converting to lowercase, so 'Apple', 'apple', and 'APPLE' are all treated as the same item and only the first occurrence is kept (in its original casing). With case-sensitive matching enabled, 'Apple' and 'apple' are treated as distinct items and both are kept.
Q:What does 'Trim whitespace before comparison' do?
When enabled (the default), leading and trailing spaces are stripped from each item before comparing it for uniqueness. This means ' apple ' and 'apple' are treated as duplicates. The actual item content in the output is preserved in its original untrimmed form — only the comparison key is trimmed, not the output itself.
Q:What is the difference between 'keep first' and 'keep last' occurrence?
Keep first (the default) retains the earliest appearance of each unique item and discards all later duplicates. Keep last retains the most recent appearance and discards all earlier duplicates. Keep last is useful when your list is ordered by recency and you want the most recent version of each entry — for example, keeping the most recently added URL rather than the first one encountered.
Q:Can I deduplicate a comma-separated list?
Yes — select the Comma delimiter, paste your comma-separated values, and click Remove Duplicates. The output is a comma-separated string with duplicates removed. The same works for semicolons, pipes, tabs, or any custom delimiter. The output uses the same delimiter as the input.
Q:Is there a limit on list size?
No — the deduplication uses a JavaScript Set for O(n) time complexity, so it processes one million items nearly as fast as one thousand. All processing runs in your browser with no server involvement. There is no size limit and no privacy concern with sensitive list data.