Remove Duplicate Lines
Delete duplicate lines from any list, keeping first occurrences in order, or sort and dedupe in one pass. Runs in your browser.
Private by construction. The conversion happens in this page's JavaScript. Nothing you paste here leaves your machine, and the tool keeps working with the network unplugged.
Two honest modes. Remove duplicates keeps the first occurrence of each line and preserves your original order — the right behavior for config files, playlists, and anything where order carries meaning. Sort + dedupe is the classic sort -u: alphabetical order, one of each. Comparison is exact and case-sensitive, so Apple and apple are different lines — deduplication that silently case-folds or trims whitespace destroys data you didn't know you had. Email lists, log extracts, and export files are the usual customers, and none of them belong on someone else's server.
Questions this tool gets asked
Does it treat “Apple” and “apple” as duplicates?
No - comparison is exact, including case and whitespace. That is deliberate: silent case-folding corrupts data like IDs and passwords. Run the text through the case converter first if you want case-insensitive deduplication.
Which copy of a duplicate is kept?
The first one, in both modes. In order-preserving mode everything else about your ordering is untouched.
Do trailing spaces make lines “different”?
Yes, because the bytes differ - “cherry” and “cherry ” are distinct lines. If your data may carry stray trailing whitespace, clean it first; invisible whitespace is the most common reason two “identical” lines both survive.
How big a list can it handle?
Hundreds of thousands of lines comfortably - the dedupe is a single pass over the text inside your browser.