I’ve worked pretty hard to stay out the whole JS package manager debate. NPM’s always done what I needed. Prior to yesterday evening, I had no baseline for comparison. Sure, like everyone else, I’ve read articles that point out how slow NPM is in comparison to Yarn. I tend to switch tabs/windows/whatever the second I forget what I was doing (i.e. every five minutes), so I don’t often wait on NPM. Yarn’s API also apparently comes from a totally different methodology. I’ve grown used to the madness surrounding NPM’s method, so I’ve honestly had no reason to change.
Note
I’m still trying to figure out how to properly build content for AMP. I recorded a few asciicasts for this post and haven’t yet figured out how to load and scale them well with a bunch of external stuff. For now, you’ll have to load them externally. Sorry about that. While you’re there, you might consider donating because asciinema is hella rad.
Background
Digital Ocean
The one thing that I actually require from my package managers is that they have to work in my environments. In production mode at work, that’s almost never an issue. Companies understand the need for good hardware to match traffic. In production mode at home, I’m pretty cheap. I don’t maintain apps with massive hardware needs because a) that’s expensive and b) I don’t ever have good ideas. I’ve gotten away with the penultimate Digital Ocean Standard Droplet for years and I have the traffic to prove it.
Ghost
I use Ghost as a blogging platform. It runs rather well on basic droplets. It’s a well-constructed JS app, so it’s not massively bloated. At the same time, I’m not running an enterprise server and my traffic is so small Ghost doesn’t ever get a chance to run out of memory. MySQL is typically using at least twice as many resources as Ghost, which is probably a good thing because it’s only running for Ghost.
Today, after applying a really important Node security patch, I decided to finally update Ghost. Ghost-CLI is a great little tool that basic does everything for me. Sort of. Like normal, I ran
ghost update
but the script failed. I didn’t initially notice what had happened, so, after trying all sorts of things that didn’t work, I reverted the updates and cracked open the directory in question.
Origin
If you’d like to test this as well, snag Ghost v1.18.4, nuke node_modules, and remove any .lock files.
NPM
CLI
$ cd path/to/1.18.4
$ rm -rf node_modules
$ rm -rf *.lock
$ ls -alh
total 76K
drwxrwxr-x 4 cjharries cjharries 4.0K Dec 12 03:56 .
drwxrwxrwt. 39 root root 4.0K Dec 12 03:47 ..
drwxr-xr-x 8 cjharries cjharries 4.0K Dec 12 02:03 content
drwxr-xr-x 4 cjharries cjharries 4.0K Dec 12 02:03 core
-rw-r--r-- 1 cjharries cjharries 32K Dec 12 02:03 Gruntfile.js
-rw-r--r-- 1 cjharries cjharries 1.4K Dec 12 02:03 index.js
-rw-r--r-- 1 cjharries cjharries 1.1K Dec 12 02:03 LICENSE
-rw-r--r-- 1 cjharries cjharries 453 Dec 12 02:03 MigratorConfig.js
-rw-r--r-- 1 cjharries cjharries 4.2K Dec 12 02:03 package.json
-rw-r--r-- 1 cjharries cjharries 2.9K Dec 12 02:03 PRIVACY.md
-rw-r--r-- 1 cjharries cjharries 4.0K Dec 12 02:03 README.md
$ npm install
npm WARN deprecated [email protected]: All versions below 4.0.1 of Nodemailer are deprecated. See https://nodemailer.com/status/
npm WARN deprecated [email protected]: Use uuid module instead
npm WARN deprecated [email protected]: This module is deprecated
npm WARN deprecated [email protected]: This package is deprecated. Use Array.isArray.
npm WARN deprecated [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
[1] 16161 killed npm installecode: sill doParallel extract 1263
Analysis
I can honestly say I’ve never seen npm kill an installation script. After running it again a few times to make sure, I hit Google. It turns out that this is a fairly common Digital Ocean issue. That thread suggests creating a swap file to increase memory. On an HDD, I’d do that in a heartbeat. On an SSD, that’s an invitation to never see my data again. Of course Digital Ocean suggests beefing up the droplet, because they need to make money. I get it.
Neither solution makes any sense, though. All I wanted to do was install a list of remote dependencies properly built for my system. I rarely run that command. What am I supposed to do, temporarily bulk up for a single command? That’s ridiculous.
Memory Usage
To make sure I wasn’t making this up, I ran the installation beside htop to track everything.
Yarn
After a bit more Googling, I quickly learned that’s there no way to, say, install dependencies one-by-one via NPM unless I wanted to call them one-by-one. My hopes for Yarn actually working were pretty low following NPM’s spectacular performance. At least installing Yarn was quick and easy.
CLI
$ cd path/to/1.18.4
$ rm -rf node_modules
$ rm -rf *.lock
$ ls -alh
total 76
drwxrwxr-x 4 cjharries cjharries 4096 Dec 12 04:34 .
drwxrwxrwt. 41 root root 4096 Dec 12 04:33 ..
drwxr-xr-x 8 cjharries cjharries 4096 Dec 12 02:04 content
drwxr-xr-x 4 cjharries cjharries 4096 Dec 12 02:04 core
-rw-r--r-- 1 cjharries cjharries 32178 Dec 12 02:04 Gruntfile.js
-rw-r--r-- 1 cjharries cjharries 1403 Dec 12 02:04 index.js
-rw-r--r-- 1 cjharries cjharries 1065 Dec 12 02:04 LICENSE
-rw-r--r-- 1 cjharries cjharries 453 Dec 12 02:04 MigratorConfig.js
-rw-r--r-- 1 cjharries cjharries 4199 Dec 12 02:04 package.json
-rw-r--r-- 1 cjharries cjharries 2968 Dec 12 02:04 PRIVACY.md
-rw-r--r-- 1 cjharries cjharries 3994 Dec 12 02:04 README.md
$ yarn install
yarn install v1.3.2
info No lockfile found.
[1/5] Validating package.json...
warning [email protected]: The engine "cli" appears to be invalid.
[2/5] Resolving packages...
warning brute-knex > knex > [email protected]: Use uuid module instead
warning [email protected]: All versions below 4.0.1 of Nodemailer are deprecated. See https://nodemailer.com/status/
warning nodemailer > mailcomposer > [email protected]: This module is deprecated
warning sanitize-html > [email protected]: This package is deprecated. Use Array.isArray.
warning grunt > [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
warning grunt-docker > grunt > [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
warning grunt-docker > grunt > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning grunt-docker > grunt > glob > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning grunt-docker > grunt > glob > [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
warning grunt-docker > grunt > findup-sync > glob > [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
warning grunt-docker > docker > pygmentize-bundled > through2 > xtend > [email protected]:
[3/5] Fetching packages...
[4/5] Linking dependencies...
[5/5] Building fresh packages...
success Saved lockfile.
Done in 38.62s.
Analysis
It worked. I’m not sure it needs more analysis than that.
Memory Usage
Watch Yarn not get killed. Also, it doesn’t seem to consume everything in reach. Dunno why; don’t care; it works.
Side-By-Side
I actually recorded this first because I couldn’t believe they were that dissimilar. If you watched the memory consumption above, there’s nothing here you haven’t seen.
Verdict
I’m moving to Yarn this week. Not only did it actually work in my environment, it worked much faster than NPM was killed. If you’re always running on high-end hardware with lots of resources (e.g. your servers are running Google Chrome), NPM will probably do the job. If you’re not made of money, Yarn’s a better bet.