A Docker module for Node.js

Docker is a good way to run servers, but driving it from the command line gets long: a Dockerfile for each service, a build, then a run with its ports, volumes, network, restart policy and environment variables, and the same again in reverse to stop it. The Docker module wraps all of that up, so that starting a service is one function call in a script:

const Docker = require("Docker");
const serviceName = "webpockets-server"
const modulesPath = __dirname+"/../node_modules";
const contextPath = __dirname+"/src";
const port = 17999;

const containerId = Docker.Service.Server.Node(
    serviceName,
    contextPath,
    modulesPath,
    port
);

console.log("Container ID: ", containerId);

And stopping it is another:

const Docker = require("Docker");

const serviceName = "webpockets-server"
Docker.Service.Stop(serviceName)

There's no Dockerfile in the project. The function that starts a Node server knows what a Node server needs, writes the Dockerfile itself, builds the image and runs it.

The shape of the module

The module is built the way all my Node modules are: one tree, a file per branch, every function checking its arguments with Security, which is its only dependency. The low branches wrap single Docker commands, and Service builds whole servers out of them:

Docker
├── Daemon      Execute, Check, SplitOutput
├── Image       Add, Remove, Exists
├── Container   Start, Stop, Remove, Exists, ID, Command, Logs
├── Network     Create
├── Config      Environment, Network, Restarting, LogPath
├── Data        Script
└── Service     Start, Stop, StopAll
    ├── Server      Apache, ApacheFpm, MySQL, Node, NodeCluster, Gitea
    ├── Nginx       ReverseProxy, LoadBalancer, FPM
    ├── Program     NodeJS, MultiNodeJS, PHP, MultiPHP, Deno
    ├── OS          Ubuntu, UbuntuGui, XWindows
    ├── Media       FFMPEG
    ├── Automation  HTTP, Chrome
    └── AI          model servers

Running docker

At the bottom, everything goes through one function, Docker.Daemon.Execute. It checks that Docker is installed, runs the docker command with the arguments it's given and waits for it to finish. A failed command throws with Docker's own error output; otherwise the output is returned, or handed to a callback that turns it into something more useful:

Execute: (args, parsingCb) =>
{
    args = Security.Array(Security.String)(args)
    Security.OptionalCallback(parsingCb)

    Docker.Daemon.Check()

    const { stdout, stderr, status } = spawnSync("docker", args, {shell: true})

    const stderrStr = stderr.toString()
    const stdoutStr = stdout.toString()

    if (status !== 0) {
        throw new Error(`An error has occurred whilst running docker command.\nSTDERR: "${stderrStr}"\n with arguments:\n${JSON.stringify(args)}`)
    }
    else if( parsingCb ) {
        return parsingCb(stdoutStr)
    }
    else {
        return stdoutStr
    }
},

Docker.Image and Docker.Container are thin wrappers over single commands. Docker.Image.Add is docker build -t, Docker.Container.Start is docker run -itd with a name, and Docker.Container.Exists lists containers filtered by name and checks that what came back is a list of hashes. Because the commands run synchronously, a script that starts several services runs from top to bottom, with each one up before the next begins.

Starting a service

Docker.Service.Start is where a service comes together. It takes a name, the text of a Dockerfile, the folder to build in and any extra arguments for docker run, and then:

1. adds the network, the restart policy and the environment variables from the current settings (see below);
2. stops any service already running under that name, removing its container and image;
3. writes the Dockerfile into the folder and builds an image named after the service;
4. runs a container with the same name;
5. deletes the Dockerfile again, whether or not that worked.

Start: ( imageName, dockerfile, contextPath, extraArguments ) =>
{
    imageName = Security.String(imageName)
    dockerfile = Security.String(dockerfile)
    contextPath = Security.String(contextPath)
    extraArguments = Security.Array(Security.String)(extraArguments)

    const networkName = Docker.Config._networkName;
    if (networkName !== undefined) {
        extraArguments.push(`--network`, networkName)
    }
...
    const containerName = imageName;
    const dockerfilePath = `${contextPath}/Dockerfile`;
    Docker.Service.Stop(imageName)
    fs.writeFileSync( dockerfilePath, dockerfile )
    let containerId = -1;
    try {
        Docker.Image.Add(
            imageName,
            contextPath
        )
        containerId = Docker.Container.Start(
            imageName,
            containerName,
            extraArguments
        )
    }
    catch (e) {
        console.error("Error", e)
        throw e
    }
    finally {
        fs.unlinkSync( dockerfilePath )
    }

    return containerId;
}

The image, the container and the service all share one name, so the name is all you need to stop it again: Docker.Service.Stop stops the container, removes it and removes the image. And since starting always stops whatever was there first, running a start script a second time restarts everything in it.

A service writes its own Dockerfile

Each service is a function that knows how its kind of server is set up. It writes the Dockerfile as an array of lines, works out the ports and volumes from its arguments, and hands both to Docker.Service.Start. Here's the Node server from the start of the article:

const dockerfileContent = [
    `FROM node:latest`,
    `ENV LANG=C.UTF-8`,
    `ENV DEBIAN_FRONTEND=noninteractive`,
    `ENV TZ=Europe`,
    `RUN apt-get update && apt-get upgrade -y`,
    `RUN apt-get install -y nano curl wget`,
    `RUN mkdir -p /usr/src/node/server /usr/src/node/node_modules /usr/src/node/ssl`,
    ...developerModeLines,
    `WORKDIR /usr/src/node/server`,
    `CMD ["node", "index.js"]`,
].join(`\n`)

const args = [
    `-v"${modulesPath}":"/usr/src/node/node_modules"`,
];
if (developerMode) {
    args.push(`-v"${contextPath}":"/usr/src/node/server"`,)
}
if(optionalHostPort !== undefined ) {
    args.push( `-p${optionalHostPort}:${optionalHostPort}` )
}

if (optionalSslPath !== undefined) {
    args.push(`-v"${optionalSslPath}":"/usr/src/node/ssl"`);
}

return Docker.Service.Start( serviceName, dockerfileContent, contextPath, args )

The other servers follow the same pattern. Docker.Service.Server.MySQL takes a name, a data folder on the host (which it creates), a database name, a user, a password and a port, and writes a Dockerfile that installs MySQL, creates the database and the user when the server starts, and keeps its data in that folder. Docker.Service.Server.Apache takes the project folder, the path of the public folder inside it and a port, and can add Xdebug on port 9001 and a command for cron to run every minute.

Settings for a block of code

Some settings apply to every service in a script rather than to one: which Docker network they join, whether Docker restarts them when they stop, and what environment variables they get. These are declared around a block of code. Each one takes a value and a callback, sets the value for everything started inside the callback, and puts the old value back afterwards:

Docker.Config.Restarting(10, () => {
    Docker.Config.Environment('SITE_MODE', 'live', () => {
        Docker.Config.Environment('SITE_LOG', 'errors', () => {
            Docker.Service.Server.Node('site.http', contextPath, modulesPath, 18001)
        })
    })
})

Docker.Service.Start reads the settings as it builds the docker run command, so that server gets --network cubelogix-internal --restart on-failure:10 -e SITE_MODE=live -e SITE_LOG=errors. Environment variables add up as the blocks nest, and the network starts as cubelogix-internal, so every service can reach the others. A restart limit of 0 means --restart always.

The services never see these settings as arguments, which keeps their signatures short, and nothing has to be passed down through a script that starts a dozen services. Each setting is kept in a variable in Docker.Config that starts with an underscore, and is restored in a finally, so the blocks nest and an error inside one can't leave a setting behind:

Environment: (name, value, cb) => {
    Security.String(name)
    Security.String(value)
    Security.Callback(cb)

    const prevEnvVariables = Docker.Config._envVariables;
    try {
        Docker.Config._envVariables = [...prevEnvVariables, {name, value}];

        cb();
    }
    finally {
        Docker.Config._envVariables = prevEnvVariables;
    }
},

Docker.Config.LogPath works the same way: inside it, Docker.Service.Program.PHP mounts that folder and sends the script's output to a log file in it.

Start and stop scripts

Every project has a start.js and a stop.js next to its code, run with node start and node stop. A PHP site with a database looks like this:

const Docker = require('Docker');
const DB_PORT = 13001;
const HTTP_PORT = 18001;

Docker.Service.Server.MySQL(
    'synthesizing.db',
    '/data/synthesizing.db',
    'synthesizing_01',
    'synthesizing',
    'password',
    DB_PORT,
);
console.log(`synthesizing.db started (port ${DB_PORT})`)

Docker.Service.Server.Apache(
    'synthesizing.http',
    `${__dirname}/..`,
    `synthesizing/public_html`,
    HTTP_PORT,
    true,
    "php /var/www/html/synthesizing/public_html/index.php cron minute"
);
console.log(`synthesizing.http started (port ${HTTP_PORT})`)

const Docker = require('Docker');

Docker.Service.Stop('synthesizing.http');
console.log('synthesizing.http stopped');

Docker.Service.Stop('synthesizing.db');
console.log('synthesizing.db stopped');

Services are named after the site and their job, synthesizing.db and synthesizing.http, and each site has its own pair of port numbers: 13001 and 18001 here, 13003 and 18003 for the next site, so all of them can run on one machine at once. The script is the whole description of what the site runs, kept in the project.

Sites behind one proxy

The same block style declares the sites behind a reverse proxy. Inside the callback each Site call adds a domain and the port it's served on, and when the callback returns, Start writes an Nginx configuration with all of them, publishes ports 80 and 443, gets a certificate for every domain with certbot and adds a monthly job to renew them:

Docker.Service.Nginx.ReverseProxy.Start('proxy', () => {
    Docker.Service.Nginx.ReverseProxy.Site('example.com', 18001)
    Docker.Service.Nginx.ReverseProxy.Site('example.org', 18003)
})

Nginx passes each request on to the site's port on 172.18.0.1, the address of the host machine as seen from inside the containers, and Docker.Service.Nginx.ReverseProxy.ChangePort can move a site to a new port and reload Nginx without restarting the proxy.

The other services

Things to know

• Starting a service always removes the old image and builds it again.
• Commands go through a shell, so the arguments are joined into one command line. The services quote every path they mount, so paths with spaces work.
• Docker.Config.Network calls Docker.Network.Create, but that branch isn't in Docker.Config's own copy of the tree, so as it stands it throws a TypeError. Services join cubelogix-internal by default, which needs creating once with Docker.Network.Create, and creating a network that already exists is an error from Docker.
• Docker.Service.StopAll stops every running container on the machine, not only the ones this module started.
• Docker.Service.Server.Gitea is an empty placeholder.