Getting stdout from BufferedProcess is very slow for ~15K lines!


#1

I have a package that runs a command line tool and I want to display the console output in the new tab of Atom.

The relevant snippet of code is here: https://github.com/QIICR/atom-dicom-dump/blob/master/lib/dicom-dump.coffee#L75-L89

With the output on the order of 15K lines, it takes very very long time to load the result into the window.

Am I doing something wrong?

I am not an expert in either Atom or coffee script. I would appreciate any suggestions.


#2

The most obvious thing I can see is that every time some more output is received from the process, you’re completely resetting the text of the editor using editor.setText. For simplicity’s sake, let’s say for example that the output of your process comes in 10 lines at a time. You do:

  1. Delete the contents of the empty editor and add 10 lines
  2. Delete the 10 lines from the editor and add 20 lines
  3. Delete the 20 lines from the editor and add 30 lines
  4. Delete the 30 lines from the editor and add 40 lines

and so on … up to delete the 14,990 lines from the editor and add 15,000 lines. This is pretty wasteful. You could add the new output to the end of the current contents of the editor.

(Additionally, Atom’s text editor component isn’t really optimized for being used as a log viewer or terminal window, so it may still not perform as well as one would hope.)


#3

@leedohm this makes perfect sense! would you be so kind to point me to the function that adds text at the end of the current contents? Do I need to use https://atom.io/docs/api/v1.13.1/TextEditor#instance-insertText?

(Additionally, Atom’s text editor component isn’t really optimized for being used as a log viewer or terminal window, so it may still not perform as well as one would hope.)

I don’t really use it as a log viewer. You can think about this package as a converter from some format into text. All I need to do is to run the converter, and show the text in a new editor tab. If there is a better way to accomplish that, I would greatly appreciate your suggestions!


#4

If you’re not using it as a log viewer, you don’t really need to do all the intermediate steps. You can collect the output and then when the program creating the output is complete, you can use editor.setText once to add the entire output in the editor.

Yes, if you keep using your current approach you’ll need to put the cursor at the end of the file, of course, but that’s pretty simple to do with editor.setCursorBufferPosition([Infinity, Infinity]).


#5

I don’t see how I can wait for completion with BufferedProcess. Is there another mechanism to run a command line process synchronously?


#6

Should I use the commands from the fs module, as described in Difference between fs functions and fs functions ending with 'sync'?

Will play with those options and update the thread.


#7

@leedohm your suggestion about appending worked very nicely! Thank you!!!