How to delete duplicate words within line or selection


#1

I’m working on a thesaurus .txt file with thousands of lines of synonyms. Each line has the main word with all its synonyms separated by a pipe. I’m basically merging different databases I found online and while I was able to delete duplicate lines, I also need to delete duplicate words within a line or even better, delete duplicate words within a selection to make it faster to clean up. All while keeping the pipes.

Here is a sample:

abandonar|abandonarse|desistir
abandonar|abandonarse|entregarse
abandonar|desistir|renunciar|marcharse

Should clean up like:

abandonar|abandonarse|desistir|entregarse|renunciar|marcharse

Is it possible to this with Atom or is there any other editor that can accomplish this? Any help will be greatly appreciated.

Thanks


#2

You can do this in init.coffee without too much difficulty. You would add a command that uses editor.getText() to grab a string of the entire contents of the buffer or editor.getSelectedText() to grab any selections you’ve made. Then you can simply use string functions to split, sort, dedupe, etc., and then insert the results in place of the text you extracted.