Why is apm so ridiculously slow (for me)?

How long does it take you, those of you who do, to build Atom from source?
On my Win 8 machine it’s a good 20 minutes job, although I haven’t actually timed it yet. I should try that next time.

I just timed it and it took just shy of ten minutes to do a clean build. (Nine minutes and fifty-six seconds, to be exact.)

Ok just tried it as well.
Exactly 20mins, give or take 2 seconds.
I should moonlight as a time stopper in some kind of sport. Synchronized swimming probably.

Clearly OSX has an edge here. Would be interesting to get an impression on Linux as well.

All of the metrics people have been discussing above essentially assume that installing a package is roughly equivalent to transferring a single file. As anyone who has worked on mobile web development lately knows … it isn’t the throughput that is the bottleneck anymore … but the latency of creating new connections.

Given that, I decided to take a look through the source for the apm install command. Here’s what I’ve been able to determine is the process for installing a single Atom package at a high level:

  1. Perform at atom.io API call to get the information on the requested package
  2. Download the latest release tarball from the GitHub repository (this is almost always a lot smaller than cloning the entire repository)
  3. Fork a process to use npm to actually install the Atom package into the user’s packages directory
    • This will install any dependencies the package has … incurring more API calls and more downloads. The worst case scenario of this (for the latency hypothesis) would be for the npm utility to have an API call and a download for each installation.

So, I suspect that it is the latency of establishing many small downloads that is killing performance for the people experiencing this issue.

Of course, without profiling … all of this is supposition. And I don’t really have the tools to measure this kind of thing.

2 Likes

But then… there wouldn’t be any difference between the OSs, since the protocols and API calls used are all the same etc.
Not that this difference between OSX and Windows is confirmed yet.

As per my benchmarks in the OP I also included the time to npm install packages. While it was a lot longer than just the git clone, ~23 seconds is still quite reasonable compared to almost eight minutes.

Having an indicator that shows the progress for the download/installation would improve the ux (especially for slow downloads).

Found the problem.
apm was clearly designed by people with fast internet connections. This is because it downloads a complete copy of it’s node.lib (for node-gyp) every single fuking time you install or update a package*. All ~25MB. That’s absolutely ridiculous and pointless. Normal node-gyp caches them for a reason, they’re big and it’s a waste of space.

This sounds like a problem with node-gyp to me. apm doesn’t download node.lib itself, and it uses “normal” node-gyp to build packages. It doesn’t redownload when I install packages.

Are you sure it does that every time? Because it will have to redownload if the version of node shipped with Atom changes, which happens reasonably often.

Well, this is both Atom’s fault and node-gyp’s fault.

node-gyp doesn’t have very through documentation of the CLI flags, and the Atom devs never properly checked how node-gyp was behaving.

node-gyp install, which Atom runs for every single package install/update, downloads regardless of status. However there is the --ensure flag which will make it check before downloading.

However, it has been solved! node-gyp shouldn’t redownload the linking libraries on every install #211

I believe Atom apm package installer is very slow as well. Every time I install a package it’s a pain. sometimes it resolves when I clear the npm cache (for some reason) npm cache clean.

I think the speed issues come from the too many requests apm sends to https://registry.npmjs.org. To get a sense of it, try running: apm install <any-package> --verbose, and you’ll find too many requests being made. Another downside is that most of the requests are repeated.

My problem was solved over a year ago. If you have an issue make a new topic for it.

In general you shouldn’t ask for new topics for old subjects. If the subject is the same then new comments should be posted on old topics. This makes it easier for people searching. Less topics to visit. In any case @leedohm would just send them back here with his boilerplate duplicate post. (grin)

2 Likes

It’s been around 15 min since I installed Atom and not even a single package got installed…
Any suggestions?