Package management basics

• Previous
• Overview: Understanding client-side tools
• Next

In this article we'll look at package managers in some detail to understand how we can use them in our own projects — to install project tool dependencies, keep them up-to-date, and more.

A dependency is a third-party bit of software that was probably written by someone else and ideally solves a single problem for you. A web project can have any number of dependencies, ranging from none to many, and your dependencies might include sub-dependencies that you didn't explicitly install — your dependencies may have their own dependencies.

A simple example of a useful dependency that your project might need is some code to calculate relative dates as human-readable text. You could certainly code this yourself, but there's a strong chance that someone else has already solved this problem — why waste time reinventing the wheel? Moreover, a reliable third-party dependency will likely have been tested in a lot of different situations, making it more robust and cross-browser compatible than your own solution.

A project dependency can be an entire JavaScript library or framework — such as React or Vue — or a very small utility like our human-readable date library, or it can be a command line tool such as Prettier or eslint, which we talked about in previous articles.

Without modern build tools, dependencies like this might be included in your project using a simple <script> element, but this might not work right out of the box and you will likely need some modern tooling to bundle your code and dependencies together when they are released on the web. A bundle is the term that's generally used to refer to a single file on your web server that contains all the JavaScript for your software — typically compressed as much as possible to help reduce the time it takes to get your software downloaded and displayed in your visitors' browser.

In addition, what happens if you find a better tool that you want to use instead of the current one, or a new version of your dependency is released that you want to update to? This is not too painful for a couple of dependencies, but in larger projects with many dependencies this kind of thing can become really challenging to keep track of. It makes more sense to use a package manager such as npm, as this will guarantee that the code is added and removed cleanly, as well as a host of other advantages.

We've met npm already, but stepping back from npm itself, a package manager is a system that will manage your project dependencies.

The package manager will provide a method to install new dependencies (also referred to as "packages"), manage where packages are stored on your file system, and offer capabilities for you to publish your own packages.

In theory you may not need a package manager and you could manually download and store your project dependencies, but a package manager will seamlessly handle installing and uninstalling packages. If you didn't use one, you'd have to manually handle:

• Finding all the correct package JavaScript files.
• Checking them to make sure they don't have any known vulnerabilities.
• Downloading them and putting them in the correct locations in your project.
• Writing the code to include the package(s) in your application (this tends to be done using JavaScript modules, another subject that is worth reading up on and understanding).
• Doing the same thing for all of the packages' sub-dependencies, of which there could be tens, or hundreds.
• Removing all the files again if you want to remove the packages.

In addition, package managers handle duplicate dependencies (something that becomes important and common in front-end development).

In the case of npm (and JavaScript- and Node-based package managers) you have two options for where you install your dependencies. As we touched on in the previous article, dependencies can be installed globally or locally to your project. Although there tend to be more pros for installing globally, the pros for installing locally are more important — such as code portability and version locking.

For example, if your project relied on Webpack with a certain configuration, you'd want to ensure that if you installed that project on another machine or returned to it much later on, the configuration would still work. If a different version of Webpack was installed, it may not be compatible. To mitigate this dependencies are installed locally to a project.

To see local dependencies really shine, all you need to do is try to download and run an existing project — if it works and all the dependencies work right out of the box, then you have local dependencies to thank for the fact that the code is portable.

For a package manager to work, it needs to know where to install packages from, and this comes in the form of a package registry. The registry is a central place that a package is published to and thus can be installed from. npm, as well as being a package manager, is also the name of the most commonly-used package registry for JavaScript packages. The npm registry exists at npmjs.com.

npm is not the only option. You could manage your own package registry — products like Microsoft Azure allow you to create proxies to the npm registry (so you can override or lock certain packages), GitHub also offers a package registry service, and there will be likely more options appearing as time goes on.

What is important is that you ensure you've chosen the best registry for you. Many projects will use npm, and we'll stick to this in our examples throughout the rest of the module.

Let's run through an example to get you started with using a package manager and registry to install a command line utility.

Parcel is a(nother) tool that developers commonly use in their development process. Parcel is clever in that it can watch the contents of our code for calls to dependencies and automatically installs any dependencies it sees that our code needs. It can also automatically build our code.

In our previous chapter we installed Prettier as a global tool. Here however, let's use npm to install Parcel as a local tool, as best practices dictate. We'll install it as part of an experimental app.

First of all, create a new directory to store our experimental app in, somewhere sensible that you'll find again. We'll call it parcel-experiment, but you can call it whatever you like:

mkdir parcel-experiment
cd parcel-experiment

Next, let's initialise our app as an npm package, which creates a config file — package.json — that allows us to save our configuration details in case we want to recreate this environment later on, or even publish the package to the npm registry (although this is somewhat beyond the scope of this article).

Type the following command, making sure you are inside the parcel-experiment directory:

npm init

You will now be asked some questions; npm will then create a default package.json file based on the answers:

• name: A name to identify the app. Just press Return to accept the default parcel-experiment.
• version: The starting version number for the app: Again, Just press Return to accept the default 1.0.0.
• description: A quick description of the app's purpose. Type in something really simple, like "A simple npm package to learn about using npm", then press Return .
• entry point: This will be the top-level JavaScript file of the app. The default index.js is fine for now — press Return .
• test command, git repository, and keywords: press Return to leave each of these blank for now.
• author: The author of the project. Type your own name, and press Return .
• license: The license to publish the package under: Press Return to accept the default for now.

Press Return one more time to accept these settings.

Go into your parcel-experiment directory and you should now find you've got a package.json file. Open it up and it should look something like this:

{
  "name": "parcel-experiment",
  "version": "1.0.0",
  "description": "A simple npm package to learn about using npm",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Chris Mills",
  "license": "ISC"
}

So this is the config file that defines your package. This is good for now, so let's move on.

Run the following command to install Parcel locally:

npm install parcel-bundler

Once that's done All The Things, we're now ready for some "modern client-side development" (which really means using build tools to make the developer experience a little easier). First of all however, take another look at your package.json file. You'll see that npm has added a new field, dependencies:

"dependencies": {
  "parcel-bundler": "^1.12.4"
}

This is part of the npm magic — if in the future you move your codebase to another location, on another machine, you can recreate the same set up by running the command npm install, and npm will look at the dependencies and install them for you.

One disadvantage is that Parcel is only available inside our parcel-experiment app; you won't be able to run it in a different directory. But the advantages outweigh the disadvantages.

Anyway, on with the setup.

Parcel expects an index.html and an index.js file to work with, but otherwise it is very unopinionated about how you structure your project. Other tools can be very different, but at least Parcel makes it easy for our initial experiment.

So now we need to add an index.html file to our working directory. Create index.html in your test directory, and give it the following contents:

<!DOCTYPE html>
<html lang="en-US">
  <head>
    <meta charset="utf-8">
    <title>My test page</title>
  </head>
  <body>
    <script src="./index.js"></script>
  </body>
</html>

Next, we need to add an index.js file in the same directory as index.html. For now index.js can be empty; it just needs to exist. Create this now.

Now we'll run our newly installed Parcel tool. In your terminal, run the following command:

 parcel index.html

You should see something like this printed to your terminal:

Server running at http://localhost:1234
✨  Built in 193ms.

Now we're ready to benefit from the full JavaScript package ecosystem. For a start, there is now a local web server running at http://localhost:1234. Go there now and you'll not see anything for now, but what is cool is that when you do make changes to your app, Parcel will rebuild it and refresh the server automatically so you can instantly see the effect your update had.

Now for some page content. Let's say we want to show human-readable relative dates, such as "2 hours ago", "4 days ago" and so on. The date-fns package's formatDistanceToNow() method is useful for this (there's other packages that do the same thing too).

In the index.js file, add the following code and save it:

import { formatDistanceToNow } from 'date-fns'

const date = '1996-09-13 10:00:00';
document.body.textContent = formatDistanceToNow(new Date(date)) + ' ago';

Go back to http://localhost:1234 and you'll see how long ago it is since the author turned 18.

What's particularly special about the code above is that it is using the formatDistanceToNow() function from the date-fns package, which we didn't install! Parcel has spotted that you need the module, searched for it in the npmjs.com package registry, and installed it locally for us, automatically. You can prove this by looking in our package.json file again — you'll see that the dependencies field have been updated for us:

"dependencies": {
  "date-fns": "^2.12.0",
  "parcel-bundler": "^1.12.4"
}

Parcel has also added the files required for someone else to pick up this project and install any dependencies that we've used. If you take a look in the directory you ran the parcel command in, you'll find a number of new files; the most interesting of which are:

• node_modules: The dependency files of Parcel and date-fns.
• dist: The distribution directory — these are the automatically packaged, minified files Parcel has built for us, and the files it is serving at localhost:1234. These are the files you would upload to your web server when releasing the site online for public consumption.

So long as we know the package name, we can use it in our code and Parcel will go off, fetch, and install (actually "copy") the package into our local directory (under node_modules).

However, this code is not ready for production. Most build tooling systems will have a "development mode" and a "production mode". The important difference is that a lot of the helpful features you will use in development are not needed in the final site, so will be stripped out for production, e.g. "hot module replacement", "live reloading", and "uncompressed and commented source code". Though far from exhaustive, these are some of the common web development features that are very helpful at the development stage, but are not very useful in production. In production, they will just bloat your site.

Now stop the previous Parcel command (using Ctrl + C).

We can now prepare our bare bones example site for an imaginary deployment. Parcel provides an additional command to generate files that are suited to publication, making bundles (mentioned earlier) with the build option.

Run the following command:

parcel build index.html

You should see an output like so:

✨  Built in 9.35s.

dist/my-project.fb76efcf.js.map    648.58 KB     64ms
dist/my-project.fb76efcf.js        195.74 KB    8.43s
dist/index.html                        288 B    806ms

Again, the destination for our production files is the dist directory.

However, as with all tools that "help" developers there's often a trade off. In this particular case it's the file size. The JavaScript bundle my-project.fb76efcf.js is a whopping 195K — very large, given that all it does is print a line of text. Sure there's some calculation, but we definitely don't need 195K worth of JavaScript to do this!.

When you use development tooling it's worth questioning whether they're doing the right thing for you. In this case, the bundle is nearly 200K because it has in fact included the entire date-fns library, not just the function we're using.

If we had avoided any development tools and pointed a <script src=""> element to a hosted version of date-fns, roughly the same thing would have happened — all of the library would be downloaded when our example page is loaded in a browser.

However, this is where development tooling has a chance to shine. Whilst the tooling is on our machine, we can ask the software to inspect our use of the code and only include the functions that we're actually using in production — a process known as "Tree Shaking".

This makes a lot of sense as we want to reduce file size and thus make our app load as quickly as possible. Different tooling will let you tree shake in different ways.

Although the list grows by the month, there are three main offerings for tools that generate bundles from our source code: Webpack, Rollup, and Parcel. There will be more available than this, but these are popular ones:

• The RollUp tool offers tree shaking and code splitting as its core features.
• Webpack requires some configuration (though "some" might be understating the complexity of some developers' Webpack configurations).
• In the case of Parcel (prior to Parcel version 2), there's a special flag required — --experimental-scope-hoisting — which will tree shake while building.

Let's stick with Parcel for now, given that we've already got it installed. Try running the following command:

parcel build index.html --experimental-scope-hoisting

You'll see that this makes a huge difference:

✨  Built in 7.87s.

dist/my-project.86f8a5fc.js    10.34 KB    7.17s
dist/index.html                   288 B    753ms

Now the bundle is approximately 10K. Much better.

If we were to release this project to a server, we would only release the files in the dist folder. Parcel has automatically handled all the filename changes for us. We would recommend having a look at the source code in dist/index.html just so you can see what changes Parcel has performed automatically.

There's a lot of tools available and the JavaScript package ecosystem is growing at an unprecedented rate, which has pros and cons. There's improvements being made all the time and the choice, for better or worse, is constantly increasing. Faced with the overwhelming choice of tooling, probably the most important lesson is to learn what the tool you select is capable of.

This tutorial installed the Parcel package using npm, but as mentioned earlier on there are some alternatives and it's worth at least knowing they exist and having some vague idea of the common commands across the tools. You've already seen some in action, but lets look at the others.

The list will grow over time, but at time of writing, the following main package managers are available:

• npm at npmjs.org
• pnpm at pnpm.js.org
• yarn at yarnpkg.com

npm and pnpm are similar from a command line point of view — in fact pnpm aims to have full parity over the argument options that npm offers. It differs in that it uses a different method for downloading and storing the packages on your computer, aiming to reduce the overall disk space required.

Where npm is shown in the examples below, pnpm can be swapped in and the command will work.

yarn is often thought to be quicker than npm in terms of installation process (though your mileage may vary). This is important to developers because there can be a significant amount of time wasted on waiting for dependencies to install (and copy to the computer).

Let's review the common actions you'll want to perform with package managers.

npm init
yarn init

As shown above, this will prompt and walk you through a series of questions to describe your project (name, license, description and so on) then generate a package.json for you that contains meta information about your project and its dependencies.

npm install date-fns
yarn add date-fns

We also saw install in action above. This would directly add the date-fns package to the working directory in a sub-directory called node_modules, along with date-fns's own dependencies.

By default this command will install the latest version of date-fns, but you can control this too. You can ask for date-fns@1, which gives you the latest 1.x version (which is 1.30.1). Or you could try date-fns@^2.3.0, which means the latest version after or including 2.3.0 (2.8.1 at the time of writing).

npm update
yarn upgrade

This will look at the currently installed dependencies and update them, if there is an update available, within the range that's specified in the package.

The range is specified in the version of the dependency in your package.json, such as date-fns@^2.0.1 — in this case the caret character ^ means all minor and patch releases after and including 2.0.1, up to but excluding 3.0.0.

This is determined using a system called semver, which might look a bit complicated from the documentation but can be simplified by considering only the summary information and that a version is represented by MAJOR.MINOR.PATCH, such as 2.0.1 being major version 2 with patch version 1. An excellent way to try out semver values is to use the semver calculator.

It's important to remember that npm update will not upgrade the dependencies to beyond the range defined in the package.json — to do this you will need to install that version specifically.

npm audit
yarn audit

This will check all of the dependency tree for your project and run the specific versions you're using against a vulnerability database and notify you if there are potential vulnerable packages in your project.

A good starting point for learning about vulnerabilities is the Snyk project, which covers both JavaScript packages and other programming languages.

npm ls date-fns
yarn why date-fns

This command will show what version of a dependency is installed and how it came to be included in your project. It's possible that another, top level, package could have pulled in date-fns. It's equally possible (and not ideal) that you have multiple versions of a package in your project (this has been seen many times over with the lodash package, as it's so useful).

Although the package manager will do it's best to deduplicate packages you may want to investigate exactly which version is installed.

You can find out more about the individual commands for npm and yarn online. Again, pnpm commands will have parity with npm, with a handful of additions.

The package managers also support creating your own commands and executing them from the command line. For instance, we could create the following command:

npm run dev
# or yarn run dev

This would run a custom script for starting our project in "development mode". In fact, we regularly include this in all projects as the local development setup tends to run slightly differently to how it would run in production.

If you tried running this in your Parcel test project from earlier it would (likely) claim the "dev script is missing". This is because npm, yarn (and the like) are looking for a property called dev in the scripts property of your package.json file.

Parcel can run a development server using the command parcel serve filename.html, and we'd like to use that often during our development.

So let's create a custom shorthand command — "dev" — in our package.json.

If you followed the tutorial from earlier, you should have a package.json file inside your parcel-experiment directory. Open it up, and its scripts member should look like this:

"scripts": {
  "test": "echo \"Error: no test specified\" && exit 1",
},

Update it so that it looks like this, and save the file:

"scripts": {
  "test": "echo \"Error: no test specified\" && exit 1",
  "dev": "parcel serve index.html"
},

We've added a custom dev command as an npm script.

Now try running the following in your terminal, making sure you are inside the parcel-experiment directory:

 npm run dev

This should start Parcel and serve up your index.html at the local development server, as we saw before:

Server running at http://localhost:1234
✨  Built in 5.48s.

In addition, the npm (and yarn) commands are clever in that they will search for command line tools that are locally installed to the project before trying to find them through conventional methods (where your computer will normally store and allow software to be found). You can learn more about the technical intricacies of the run command, although in most cases your own scripts will run just fine.

You can add all kinds of things to the scripts property that help you do your job. We certainly have, and others have too.

This brings us to the end of our tour of package managers. Our next move is to build up a sample toolchain, putting all that we've learnt so far into practice.

• Previous
• Overview: Understanding client-side tools
• Next

• Introduction to npm scripts
• package.json reference

• Client-side tooling overview
• Command line crash course
• Package management basics
• Introducing a complete toolchain
• Deploying our app