Installation of packages without network access to amp package repository


#1

I’ll be developing in an isolated network that simply is NOT going to allow access to just download and install packages over the internet. EVERYTHING needs to be committee approved before it’s put into that environment. I’ve managed to get Atom installed with the default packages, and decide to try and add atom-html-preview manually. I got the zip file from https://github.com/webBoxio/atom-html-preview approved and have it in my network now. I’ve tried extracting the contents to my .atom/packages directory in an atom-html-preview directory. When I restart Atom, the plugin shows up, but doesn’t function. I see it in the setting/Packages tab, but clicking it’s settings button doesn’t give me the extra information I see from my personal install. Ctrl-P does nothing. The Preview HTML > Enable Preview shows up but doesn’t give me an html preview.

Is there more that I need to do to get everything hooked up? I’m trying to find some way to make this manageable in a very controlled environment.

BTW… Windows 7


Atom packages in a highly controlled environment
Can atom package manager work with access solely to a NPM repository?
#2

You probably have to install the package’s dependencies. These can be found in package.json in the package directory. Normally, you would be able to do this by running apm install from the package’s directory, but I understand that this is not possible for you. In general packages are hosted on npmjs.org. Note that each of those packages can have other dependencies, which may have other dependencies, etc. so this can be a lengthy process… :frowning:


#3

Looking at the package.json the package needs underscore-plus:

"dependencies": {
  "underscore-plus": "^1.2.1"
}

When you manually download and extract the zip file you need to run apm install in the package’s directory afterwards to pull in any dependencies. This will download all dependencies and place them in the node_modules folder and recursively pulls in their dependencies as well. Since this is not possible in your environment, you’ll have to do that manually.

Recursively go over each package.json file. If it lists a package as a dependency search on npm for the package and follow the link to package’s github page to read its package.json and repeat the whole process.

So in your example:

  • get underscore-plus approved.
  • extract its contents into atom-html-preview/node_modules
  • look at underscore-pluspackage.json to see if it has any dependencies. It seems it needs underscore.
    • get underscore approved
    • extract its contents into atom-html-preview/node_modules/underscore-plus/node_modules
    • look at underscore's package.json to see if it has any dependencies. Whew, looks like it doesn’t.

Depending on the amount of dependencies each package has this can be a very lengthy process but I don’t think it can be simplified. The only way I see is to do an apm install at a location with no restrictions and zip the result and get that approved.


#4

Thanks, I’ll explore that. You’re right that this could get very tedious, but I see why there may be no simple answer. I think this provides the info I need to troubleshoot this installation.

thanks


#5

You should scan the tree on github to see all nested dependencies and get them all approved at once.


#6

agreed… don’t have GitHub access from work, so even this is “painful”


#7

Ok, I’ve done all of this (it seems odd that the node_modules would nest like this (seems there would be redundancy if multiple packages had similar dependencies)). However, in my AppData\Local\packages, I’ve extracted the atom-html-preview contents, then I added a node_modules directory to that directory and extracted underscore-plus to that, then added a node_modules under it and extracted underscore. The directory path to underscore.js is AppData\Local\atom\packages\atom-html-preview\node_modules\underscore-plus\node_modules\underscore\underscore.js.
It’s still not working. The developer tools console has the message: Failed to activate package named ‘atom-html-preview’ Error: Cannot find module ’underscore-plus’.

Am I misssing a step, or placing things in the wrong place.


#8

Why are modules repeated in nested node_modules folders? That’s to avoid version dependency. You can write an app that requires Foo 1.2 and includes a module Bar that depends on Foo 1.3 as well as a module Quux that depends on Foo 0.9. So you can have all these versions of Foo in the same app and different parts of the app use different versions of Foo.

It kind of scares me, but it seems to work…


#9

That is a key (great) feature of npm. It removes most of the dependency nightmares of older package managers.

Usually you should not mess around with the npm stuff. You will end up with a setup that can’t be automatically repeated and you can end up with a mess in general.


#10

This is the wrong directory. Atom stores all of its configuration files and packages inside the .atom folder of your user’s home directory. Try moving atom-html-preview and all its nested stuff to: ~/.atom/packages/atom-html-preview.


#11

Hmmm… placed them in the windows equivalent (found the .atom\packages\atom-html-preview directory there), and copied the node_modules directory structure and I still get the same error. It did not already have a node_modules directory.


#12

OK, I guess I can see how that keeps things straight. I had originally placed all of them under the \packages directory, so I moved to follow this structure

Following Alchiadus’ instructions, I had to create a node_modules directory to place the dependencies in recursively. Not sure how I could follow those instructions without messing with this directory… maybe I misunderstand your response.


#13

No. I forgot the topic of the thread. Ignore that.


#14

I created a zip with all the files of the atom-html-preview package, including npm dependencies, in case you still need it. You can find it here:


Just extract to the .atom/packages directory


#15

Thanks, but for all the reasons this is a pain, I can’t get to dropbox from the network at work, and this would have to go through all the same approvals.

It did give me an idea though. I have this working on my mac at home (where I have the luxury of having it “just work” by installing from within atom. I can compare the file system structures here to what I have at work (it is windows at work, so a bit of apples and… uh… windows), but it’s another lead to chase down.

I may also be able to work out installing all the packages I’d like to use, archiving them here and just getting that approved. Does anyone know of a reason that I wouldn’t be able to install what I want, compress up the ~/.atom/packages directory from my mac, get it approved and extract to {my user name}.atom\packages on my work machine and expect that to work?


#16

I double-checked my previous post with the instructions and saw I’ve been too hasty.
atom-html-preview requires underscore-plus version ^1.2.1, the latest version is 1.6.6 so this is okay. However, the latest version of underscore-plus requires underscore version ~1.6.0, the latest version is 1.7.0, so this is not okay.

You can try to get the correct version of underscore approved: https://github.com/jashkenas/underscore/releases/tag/1.6.0

I’m afraid this is just a normal day in the life of apm / npm, sorry I overlooked this before.

You should be able to install all the packages you’d like to use at home and get the whole package directory approved, that would personally be what I’d do in your situation. I see no reason that shouldn’t work.


#17

Ah, with the side discussion around nested dependencies, it should have occurred to me to check version numbers.

I looked, and I do, indeed, have underscore 1.7. Since it’s the weekend, I took wild short and changed the dependency specific in the atom-html-preview to say underscore 1.7 was ok (I know, not a good idea, but it’ll be Monday before I can get 1.6 approved and into that environment.) I still get the same results (will revert to 1.6 when I can get it in my environment. Though this is certainly additional paid in trying to get all of this set up correctly there.

I did start doing a more thorough comparison of what I have on my personal machine, that works, and notice that, once installed, there’s a lot more properties in the package.son file under atom-html-preview than what is in the simple extracted content. It would appear that it gets updated somewhere along to way to indicate where everything is cached, etc, so, is having all of that extra content in the package.son going to be a problem if I zip up the whole thing and drop it into the .atom/packages directory?? (I"m not sure they’ll approve this approach of me gathering it all together at home and then them approving my zip file, but, if they do, I guess I’ll find out if it works by trying it.

side note: it would be really nice if there was a way to just load all of the downloaded files to somewhere in the local file system/network and tell amp/npm to look there first for folks in the same sort of locked down environment that I am (similar to running a local gem server). As companies get more and more security conscious, this is going to be an issue for a lot of folks in very controlled environments. I love how simple’s been made when, like at home, I have unfettered access to the repositories, but big security minded companies just aren’t going to allow such.


#18

The apm server API is documented:

https://atom.io/docs/latest/apm-rest-api

You could write a server that conforms to that standard and then submit a PR to make the server apm connects to configurable. Then it is just a matter of finding the dependency closure of all the packages you want to serve and making them available from your server once they are approved.


#19

hmmmm… don’t suppose the APM server is open source?


#20

No, it isn’t. I believe that the apm server is tightly integrated with the rest of the atom.io website right now: