Today I had a list consisting of 16,000 lines where most where duplicates. The list was in Google Spreadsheets (I’m allergic to Excel on a Mac) and I wanted to remove all the duplicates from the list and only get a much shorter list of the unique values. Now this is amazingly hard to do in Google Docs, and none of my local text editors had a feature for it either.
After complaing to my tech colleagues in Berlin, there turned out to be an incredibly fast way to do it on a Mac.
This is how to do it:
- Just save the list in a .txt file on your local drive. I saved it as UnsortedLongList.txt
- Open the Terminal.
- Go to the directory where you saved the file. In my case in Documents, so cd Documents took me there.
- Type the following command into Terminal: sort UnsortedLongList.txt| uniq > NoDupes.txt
- That’s it!
Now you have a txt document with a much shorter list consisting of the unique values only.
Thanks Jon! You saved me a lot of time in the future. I have to remember to ask stuff more often.