Gamehoster

I wanted to find a way to make multiplayer games while focusing on the gameplay and the graphics, without having to worry about hosting or networking. Gamehoster is the solution to this.

It lets you write snippets of code for the things that matter, the game updates and the data schemas for the players and the levels, without having to worry about the underlying details of network communication, synchronisation, and deploying the server.

It builds on the principles of Sitehoster: a folder on your own computer, mirrored to your own server. In Gamehoster, the folder is a game. I designed the engine so that the custom parts of a game slot into it automatically, and it uses a generic, packed binary protocol to send each player updates for just what they can see.

The game runs on the server, which is the only place it's simulated. The browser draws what it's sent and sends back the player's input. Asteroids and Key Hunt both run on it, and the examples here are from Asteroids.

Defining a game

A game is a folder inside a folder named for its domain, and the two names make its address. One server hosts any number of domains, each with any number of games:

game.asteroids.net/        →  a domain
  asteroids/               →  wss://game.asteroids.net/asteroids
game.keyhunt.com/
  keyhunt/                 →  wss://game.keyhunt.com/keyhunt
game.gamehoster.org/
  pong/                    →  wss://game.gamehoster.org/pong
  chess/                   →  wss://game.gamehoster.org/chess

Inside a game's folder, every file has a fixed name that says what it is: a settings file, schemas that declare the state, and bodies of JavaScript that run at particular moments:

An instance is one running copy of the game, a room or an arena, with its own state, players and entities. Gamehoster's games guide and files reference go through every file a game can have.

Settings

The settings file is a handful of numbers. The tick rate is how often the game is simulated, and the send rate is how often each player is sent an update:

{
  "gamehoster-game-title": "Asteroids",
  "gamehoster-game-browse": "none",
  "gamehoster-game-create": "none",
  "gamehoster-game-tickRate": 30,
  "gamehoster-game-sendRate": 15,
  "gamehoster-game-capacity": 50,
  "gamehoster-game-botRate": 15,
  "gamehoster-game-warmInstances": 1,
  "gamehoster-game-botsMax": 20,
  "gamehoster-game-botsZeroAt": 20
}

Schemas

Every piece of state that a player might be sent is declared in a schema, as a list of fields. Each field has a name, a type and a default, and it's either public, sent to everyone who can see the object, or private, sent only to the player it belongs to. A number can also say its range and how many decimal places matter, which is what the protocol packs it into, and whether it wraps round, so that the browser eases it the short way across the edge of a wrapping world:

{
  "gamehoster-player-schema-name": "x",
  "gamehoster-player-schema-type": "number",
  "gamehoster-player-schema-default": 0,
  "gamehoster-player-schema-visibility": "public",
  "gamehoster-player-schema-wrap": "arenaW",
  "gamehoster-player-schema-min": 0,
  "gamehoster-player-schema-max": 3600,
  "gamehoster-player-schema-decimals": 1
}

There are five types: number, int, bool, string, and tick, for a moment in game time like when a bullet was fired. State that no player should ever see, like a ship's velocity or a bot's plans, isn't a field at all: it goes in a persistent bag kept on the server, one for the instance and one for each player and entity, which is never sent. Gamehoster's types reference has the details.

Bodies

A body is a file of plain JavaScript, written as the inside of a function, that reads and writes the game through one object, Gamehoster. Here's the body that moves an Asteroids ship each tick:

const me = Gamehoster.Context.playerId
if (!Gamehoster.Player.State.Get(me, "alive")) return

const here = Gamehoster.Context.instanceId
const thrust   = Gamehoster.Instance.State.Get(here, "thrust")
const maxSpeed = Gamehoster.Instance.State.Get(here, "maxSpeed")
const friction = Gamehoster.Instance.State.Get(here, "friction")
const W = Gamehoster.Instance.State.Get(here, "arenaW")
const H = Gamehoster.Instance.State.Get(here, "arenaH")

const mem = Gamehoster.Player.Persistent.Get(me)
const angle = Gamehoster.Player.State.Get(me, "angle")
let vx = mem.vx
let vy = mem.vy

if (mem.thrusting) {
  vx += Math.cos(angle) * thrust
  vy += Math.sin(angle) * thrust
  const sp = Math.hypot(vx, vy)
  if (sp > maxSpeed) { vx = vx / sp * maxSpeed; vy = vy / sp * maxSpeed }
} else {
  vx *= friction
  vy *= friction
}
Gamehoster.Player.State.Set(me, "thrusting", !!mem.thrusting)

let x = (((Gamehoster.Player.State.Get(me, "x") + vx) % W) + W) % W
let y = (((Gamehoster.Player.State.Get(me, "y") + vy) % H) + H) % H

mem.vx = vx
mem.vy = vy
Gamehoster.Player.State.Set(me, "x", x)
Gamehoster.Player.State.Set(me, "y", y)

The body doesn't know anything about sockets, other players' browsers or what has been sent to whom. It reads the state, works out the new state, and writes it, and the engine does the rest.

A command is what a player sends to the server, and it's a folder with three files. Its schema says what values it carries, its handler is a body that runs when it arrives, and a small manifest lists the fields the handler is allowed to write:

// gamehoster-command-schema.json
[
  {
    "gamehoster-command-schema-name": "angle",
    "gamehoster-command-schema-type": "number"
  }
]

// gamehoster-command.json
{ "writes": ["player.angle"] }

// gamehoster-command-handler.js
const me = Gamehoster.Context.playerId
if (!Gamehoster.Player.State.Get(me, "alive")) return
Gamehoster.Player.State.Set(me, "angle", Gamehoster.Command.Get("angle"))

The manifest is what makes commands safe. A command comes from a player's browser, which can send anything, so whatever its handler tries to write, only the fields in the manifest, on the sender's own player, are kept.

The view body decides what each player can see. It runs for each player before every update and marks the objects they should be sent. Leave it out and everyone is sent everything, which is right for a small game like Pong. Gamehoster's functions reference lists everything on the Gamehoster object, and the examples are complete games to read through.

Slotting into the engine

Sync

A local tool mirrors your folder of games up to the server, in the same way as Sitehoster, pushing only the files that have changed. On the way, it turns each game into two generated files. Every body becomes a real function that takes the Gamehoster object:

function fnLiteral(src, indent) {
  if (src == null) return 'null'
  const body = src.replace(/\n+$/, '')
  const pad = indent || '  '
  const inner = body.split('\n').map(l => l ? pad + '  ' + l : '').join('\n')
  return 'function (Gamehoster) {\n' + inner + '\n' + pad + '}'
}

gamehoster.server.js holds the settings, the schemas, and every body that runs on the server, and the server simply loads it. gamehoster.client.js is the library the browser loads. It's the same generic client for every game, with this game's schema tables, binary codec settings, rates and browser bodies baked in. Because both come from the same folder, the server and the client can't disagree about the game. Gamehoster's local tool guide and sync reference cover this in full.

The server

The server is one box. Caddy deals with HTTPS, serves each game's client library straight from disk, and passes the WebSockets through to one Node.js process. When a browser connects to wss://game.asteroids.net/asteroids, the process finds the game from the host name and the path, checks the page it came from is one the game allows, and runs the game's join body, which puts the player into an instance. Gamehoster's infrastructure reference has the whole layout.

Each game then runs on its own timers: one that ticks every instance at the tick rate, one that sends every player an update at the send rate, and one for the bots:

g.tickTimer = setInterval(() => { const t0 = performance.now(); world.step(Date.now()); recorder.timing(key, 'tick', performance.now() - t0) }, tickMs)
g.botTimer = setInterval(() => { const t0 = performance.now(); world.stepBots(Date.now()); recorder.timing(key, 'bots', performance.now() - t0) }, botMs)
g.sendTimer = setInterval(() => {
  const t0 = performance.now()
  for (const inst of world.instances.values()) {
    for (const pid of inst.players.keys()) {
      const p = world.players.get(pid); if (!(p && p.socket && p.socket.readyState === 1)) continue
      const r = world.encodeUpdateFor(inst, p)
      if (r) {
        p.socket.send(r.bytes)
        recorder.update(key, inst.id, r.bytes.length, r.appear, r.changed, r.gone)
      }
    }
    inst.dirty.clear()
  }
  recorder.timing(key, 'update', performance.now() - t0)
}, sendMs)

The tick

A command doesn't run when it arrives. It's put in a queue for its instance, and every tick runs the same three steps: the commands that have arrived since the last tick, in order, then every player's and entity's update, and then the instance's update, which does whatever involves more than one object, like collisions:

advanceInstance(inst) {
  const g = this.game
  const q = inst.queue; inst.queue = []
  for (const c of q) {
    if (c.type) this.run(g.commands[c.type].handler, { self: c.sender, instance: inst, command: c.params, commandType: c.type }, 'command')
    if (c.seq) { const sp = this.players.get(c.sender); if (sp) sp.ackSeq = Math.max(sp.ackSeq || 0, c.seq) }
  }
  const pu = g.scripts.playerUpdate
  for (const pid of [...inst.players.keys()]) {
    const ctx = { self: pid, instance: inst }
    this.run(pu, ctx, 'playerUpdate')
  }
  for (const m of inst.entities.values()) {
    for (const eid of [...m.keys()]) {
      const e = m.get(eid); if (!e) continue
      const et = g.entities[e.type], ctx = { entity: eid, instance: inst }
      this.run(et.update, ctx, 'entityUpdate')
    }
  }
  this.run(g.scripts.instanceUpdate, { instance: inst }, 'instanceUpdate')
}

Applying a command records it as applied for the player who sent it, and that count goes back to them in every update.

Reads and writes

Each body is run with its own Gamehoster object, which knows who the body is running for. Its writes don't go straight into the state. They're held until the body has finished, and reads in the same body see them, so a body sees its own changes. Then the writes are applied together, checked against what the body is allowed to write: a command's manifest, or anything at all for the game's own updates.

Applying a write is also how the engine knows what has changed. Every write to a schema field marks the field as changed, on the instance, so the next update can include it without anything having to be compared:

commit(e) {
  if (!e.container) return
  e.container[e.name] = e.value
  if (e.inst && e.inst.dirty && this.isStreamedField(e.kind, e.type, e.name)) {
    let s = e.inst.dirty.get(e.id); if (!s) { s = new Set(); e.inst.dirty.set(e.id, s) }
    s.add(e.name)
  }
}

There's no history of past ticks and no copy of old state to compare against: an instance is only ever its present state, plus the list of what has changed since the last send. Gamehoster has two kinds of random numbers as well. Random is seeded from the object and the tick, so it gives the same answer to anyone who asks on the same tick. Entropy is real randomness that stays on the server, for things a player mustn't be able to predict, like where a rock appears.

Instances and bots

The join body decides where a new player goes. Asteroids and Key Hunt put each player in the first instance with room, and open a new one when they're all full. Pong waits for two players, and Chess matches players by rating. The settings can also keep a number of instances running at all times, even with nobody in them, so there's always a game to drop into.

A bot is a player with no socket. Its brain is a body that runs on the bot timer, reads the game, and acts only by sending the same commands that a person would, so nothing in the game has to treat it differently. The engine adds and removes bots one at a time, towards a target set by how many people are playing, and never lets a bot take a place a person could have. Asteroids and Key Hunt both go into their bots in detail, and Gamehoster's instance reference covers the tick, the writes and the bots in full.

The front end

The game's own website is a page with a canvas, the game's client library, and one call to start it:

<canvas id="game" width="960" height="600"></canvas>
<script src="https://game.gamehoster.org/asteroids/gamehoster.client.js"></script>
<script>
  Gamehoster.start('game', { name: 'Ada' })
</script>

The client opens the socket, sends the join, and runs the game's browser bodies. A setup body runs once, to get the canvas ready. A render body draws every frame. An update body runs after each update from the server, and there's an input body for each mouse or keyboard event the game wants, named after the event, like gamehoster-frontend-input-mouse-move.js. A settings file can limit how often each command is sent.

The store

The client keeps a store of every object the player can see. Each update from the server says what has come into view, with all its state, what has changed, and what has gone, and the store keeps each object's latest state along with a short history of the last few updates, each stamped with its tick.

The render body doesn't draw the newest update. It draws a little behind it, one and a half updates behind, so that there's almost always a later update to move towards. That's 3 ticks, or a tenth of a second, in Asteroids. The client has its own clock, which runs at the game's tick rate and is nudged towards where the server should be, by at most 10% faster or slower, so a late or early packet doesn't make things jump:

advance(nowMs) {
  if (!this._have) return
  if (this._lastMs == null) { this._lastMs = nowMs; return }
  const dt = Math.max(0, (nowMs - this._lastMs) / 1000); this._lastMs = nowMs
  const target = this._srvTick + ((nowMs - this._srvAtMs) / 1000) * this.tickRate
  const diff = target - this.clock
  if (Math.abs(diff) > this.buffer + 30) this.clock = target
  else this.clock += dt * this.tickRate * Math.max(0.9, Math.min(1.1, 1 + 0.1 * diff))
  if (this.predict) this._runPredict(dt)
}

Each number in an object's state is eased between the two updates either side of the render tick, and anything else takes the later value. Things that move in a straight line, like rocks and bullets, aren't eased at all: their state is where they started, their velocity and the tick they started on, so the render body works out where they are from the clock.

Objects in the browser

Each kind of object can have three bodies of its own that run in the browser. An enter body runs once when it comes into view, to choose anything that should stay fixed while it's on screen, like the random puffs of an explosion, and keep it in the object's bag. An exit body runs once when it goes. And a smooth body runs every frame and can change how the object is eased: both games use one to snap a player to where they respawn, rather than sliding them across the map.

Your own object is the one that matters most, because anything between you pressing a key and seeing it happen feels like lag. A predict body runs your own object forward from your live input, with the same physics as the server, and keeps nudging it towards the server's position, so your ship moves the moment you do. For a one-off action, a command can also carry a guess at its result, which is drawn over the game until the server says it has applied the command. Gamehoster's front-end guide and front-end reference go through all of it.

The binary protocol

The server and the client talk over one WebSocket, in binary. There are four messages:

Nothing says which message is which: the first one each way is the hello or the welcome, and every one after that is a command or an update. Nothing about the schemas is sent either, not the names of the fields or their types. Both sides built their tables from the same folder, and the hash in the hello and the welcome is how they check that they agree. If they don't, the socket is closed. Gamehoster's protocol reference has the layout of every message.

Packing values

Every message is a single stream of bits, and each value takes exactly as many bits as it needs, so a value that's smaller than a byte stays smaller than a byte. Both sides know from the schema how many bits each value takes:

So a ship's x in Asteroids, 0 to 3,600 to one decimal place, is 36,001 steps, which fit in 16 bits. Its y, 0 to 2,400, takes 15, and its angle, −3.15 to 3.15 to three decimal places, takes 13. Declaring a tight range, and only the decimal places that matter, is what makes a value cheap. Counts and lengths are written in groups of 7 bits, each with a bit to say whether another group follows, so anything under 128 takes a byte.

What each player can see

Every update is made for one player. The view body says what they can see now, and the server keeps a list for each player of the objects it has already sent them. Comparing the two gives the three lists that make up an update: objects they can see that they haven't been sent, which go with all their state; objects they've been sent that they can no longer see; and, for the rest, the fields that have been written since the last send:

buildUpdate(inst, p) {
  const T = inst.tick
  const vis = this.visibleTo(inst, p.id)
  const sent = p.sent || (p.sent = new Set())
  const appear = [], changed = [], gone = []
  for (const id of vis) {
    if (!sent.has(id)) { const entry = this.objEntry(inst, id, p.id); if (entry) { appear.push(entry); sent.add(id) } continue }
    const dirty = inst.dirty.get(id); if (!dirty || !dirty.size) continue
    const fields = this.changedFor(inst, id, p.id, dirty)
    if (fields && Object.keys(fields).length) changed.push({ id, fields })
  }
  for (const id of sent) if (!vis.has(id)) { gone.push(id); sent.delete(id) }
  return { t: 'update', tick: T, ack: p.ackSeq || 0, appear, changed, gone }
}

That's all the server remembers about each player: which objects they hold, not what it told them about each one. An object that leaves your view and comes back is simply sent again in full. And an object that's never written after it's made, like a rock or a key, is sent once when it comes into view and never again.

Ids that aren't sent

The update needs some way to say which object a change is for, but the objects' own ids are never sent. Because the server knows exactly what each player is holding, both sides can number the objects the same way without being told.

Each player's objects are numbered from 0, with no gaps. Object 0 is always the instance and object 1 is always your own player. An object coming into view takes the next number, on both sides, so it doesn't need to send one. When an object goes, the last object in the list moves into its number, and both sides do exactly the same:

remove(netId) {
  const id = this.arr[netId], last = this.arr.length - 1
  if (netId === last) { this.arr.pop(); this.map.delete(id); return { removed: netId, pop: true } }
  const moved = this.arr[last]; this.arr[netId] = moved; this.map.set(moved, netId); this.arr.pop(); this.map.delete(id)
  return { removed: netId, from: last, to: netId }
}

When several objects go at once, they're removed from the highest number down, so the object that moves is always one that's staying. Since the numbers have no gaps, a number only needs as many bits as there are objects: 6 bits names any of 64. The count even changes within one update. The objects that have gone are numbered with the count as the update starts, and the changes with the count after the new objects have been added, so each number is as short as it can be at that point.

Commands aren't numbered either. The client counts the commands it sends and the server counts the ones it receives, so they agree, and each update carries in 8 bits how many of your commands the server has applied.

An update

An update starts with 8 bits for how many ticks have passed since the last one, rather than the tick itself, and 8 bits for the command count. Then come the three lists, each starting with its count. An object coming into view is 1 bit for whether it's a player or an entity, the kind of entity in just enough bits for the game's kinds, and then all its public fields. A changed object is its number, a bit for each of its fields to say whether it's included, and then just the fields that are.

A player with nothing new to be told isn't sent anything at all, except an occasional empty update so the 8-bit tick count can't run past 255. And everything the server keeps for itself, from velocities to bots' plans to spatial grids, is in the persistent bags rather than the schemas, so it never takes a single bit.