Not able to spawn npm modules from Electron Package in OS X [solved]


#1

Once upon a time you decide to develop an electron application to help developers and designers spin up projects with less technical know-how. While developing your electron application with electron-prebuilt you decide you want to let people do the following.

spawn('npm', ['install'],{cwd: project.directory});
spawn('bower', ['install'],{cwd: project.directory});
spawn('gulp', {cwd: project.directory});

So you proceed to add buttons to spawn off the bower install, npm install, and gulp.

Everything is gravy (working) with electron-prebuilt and you are very happy. You then package up your application to be distributed for Darwin x64 with electron 0.29.2. Suddenly there are ENOENT errors being thrown on all three spawn tasks (npm, gulp, and bower). You research and discover that you need to set the PATH properly for OS X applications. So you use sindresorhan’s fix-path module to solve this which can be distilled down into this line of code.

process.env.PATH=childProcess.execFileSync(process.env.SHELL, ['-i', '-c', 'echo $PATH']).toString().trim()

But still, you are getting ENOENT errors in the Darwin x64 package for all three spawn tasks. You discover that nvm only exposes the modules in the bin directory via a subshell. Some issues on github point to this problem:

https://github.com/sindresorhus/fix-path/issues/3
https://github.com/maxogden/monu/issues/21#issuecomment-100564734 

So you put an issue in with nvm to expose the modules programmatically. While you wait for that to manifest you change your attack. Instead of using spawn, you will use bower and npm programmatic api’s. It sucks because they buffer output, so you can’t hook into a stream and get constant feedback to the user, but at this point you can’t be picky. Your spawn tasks for npm and bower become:

npm.load({ prefix: project.directory, loglevel: 'error', loaded: false }, function(err) {
  npm.commands.install([], function(err, data) {
    if (err) project.status='alert';
    else {
      $scope.addTerminalLine(project, "Completed npm install");
      $scope.successHandler(project, 'npm')
    };
  });
 });
bower.commands.install(undefined, undefined, {
  cwd: project.directory, silent: true, quiet: true, production: true })
    .on('err', function(err) {
  });
  .on('end', function(data) {
    $scope.addTerminalLine(project, "Completed bower install");
    $scope.successHandler(project, 'npm')
  });
});

Ahhh life is good again. Both of these commands work in the Darwin x64 package. But wait, you can’t run gulp tasks with this approach since gulp has no programmatic API. Normally you would just require the gulpfile and run the tasks. But in this app there is the potential to have multiple projects at once (which means multiple gulpfiles). So you can’t just require(gulpfile) and solve the problem.

So you decide to put your gulp command as the start script in your projects’ package.json file. Then you can use this code to launch gulp.

npm.load({ prefix: project.directory, loglevel: 'error', loaded: false }, function(err) {
        npm.commands.start([], function(err, data) {
          if (err) project.status='alert';
          else {

          };
        });
      });

But alas, even though this works with electron-prebuilt, it again doesn’t work in when packaged for Darwin x64. You can see in the console that it “started” the gulp task because npm.commands.start emits data into the console, but again it doesn’t actually work. You’re not getting ENOENT errors like when you tried to use spawn('gulp', {cwd: project.directory}); (after running fixPath of course), but it’s still not working. So you keep researching ways to launch gulp.

Sindre Sorhus was able to do it here: https://github.com/sindresorhus/gulp-app
Which I located from: https://github.com/sindresorhus/awesome-electron

My implementation of his solution looks like this:

fixPath();
var gulpPath = path.join(__dirname, '../', 'node_modules', 'gulp', 'bin', 'gulp.js');
gulp=spawn(gulpPath, {cwd: project.directory});

Which works fantastically in prebuilt, but again not at all in a packaged app. The process is spawned with all of it’s properties, but nothing ever comes out of the streams and the gulp task doesn’t actually execute. Using debugger I grabbed the pid (process id) from the spawned process and used ps -p 1234 where 1234 was the process id. The system reported no process. Things working in prebuilt but not as a package for Darwin x64 seems to be the common theme here.

What would you do?

verified not to be an issue in linux

Tested with and without NVM. Does not work in either case

I’m currently in research mode. I’m going to go through the awesome-electron repo and find how others do it

Electron version: 0.29.2
Platform: Darwin
Arch: x64

#2

My next idea is to try firing off these tasks from the mainProcess and doing IPC messages to the renderer process. Maybe that will work better!


#3

I had a fantastic suggestion from @diegotorres

gulp=spawn(process.env.SHELL, ['-c', 'cd ' + project.directory + ' && gulp'])

The return of this acts like a regular spawn call. You can read output from the streams and detect when the process closes still.

SOLVED


#4

Can also be fixed by using this command:

launchctl setenv PATH "$PATH"

So you could do something like this when your app starts.

childProcess.execFileSync(process.env.SHELL, ['-i', '-c', 'launchctl setenv PATH "$PATH"'])

And that allows this to work:

var gulpPath = path.join(__dirname, '../', 'node_modules', 'gulp', 'bin', 'gulp.js');
gulp=spawn(gulpPath, {cwd: project.directory});

This will not work if the application is launched via spotlight! I recommend the above solution for that case


#5

Could you kindly take a look at this question that I posted on SO? I think the issue I am facing is similar to what you had.

Thank you very much.


#6

I’m not sure because I don’t package as an asar. I would package as an asar if I could, but asar pack has a bug on OS X. There is an issue in for it right now https://github.com/atom/asar/issues/33. Until then I can’t package my app as an asar!

That said, I took a look at your question. I’m not sure why that is happening, but it would be interesting to see if you still have that problem when you don’t use an asar to distribute.


#7

Thank you, @erikmellum

This is the repo where I have the issue discussed in the SO question - and I have adapted the gulp-based asar packaging that comes with the electron-boilerplate (which is also mentioned in Sindre Sorhus’ awesome-electron list of generators). And fortunately the packaging itself is working fine for me for all the three platforms, but for the issue of not being able to use the packed file as an argument to a spawned node process.

I have posted another separate thread with the question - hopefully someone can suggest a solution.


#8

It looks like this fix isn’t always valid. My environment used zsh. I tested my Darwin x64 package on a mac with regular Bash and suddenly ran into the same issue where the spawn task is created, but never does anything. You can get the pid of the newly created process, and notice that it never runs by using ps -p 1234. So I will be testing out probably using https://atom.io/docs/api/v0.187.0/BufferedProcess (Buffered Process) which I believe is how atom solves this bug.

Another thing I tested that does not work is spawning in the mainProcess and using message passing to the renderer process. The mainProcess spawn object has the same issue (unfortunately). I located this thread that discusses this bug as well: BufferedProcess and $PATH


#9

Solved! All the above fixes I believe are symptoms of the true cause. If you do this:

var spawn = require('child_process').spawn
spawn(...)

Then you will get errors when trying to spawn in an OS X Package. Easily fixed by doing this instead:

var childProcess = require('child_process')
childProcess.spawn(...)

#10

Interesting… Thank you.

I will explore the option of require-ing the entire child_process first and then using spawn or fork from there.


#11

@floydpink let me know if that changes anything for your situation.


#12

Unfortunately, that did not help.

I got some responses from @zcbenz on the GitHub issue that I raised and the current status is I can make fork(module) work in place of spawn(node, module) where the module is within an asar package - but for some reason after the forked child process (with { silent : true }) gets up and running the electron processes (both Browser and Renderer) stops working.


#13

Thank you! This thread has saved us a ton of time.