A scalable real-time signal system in PHP and Node.js

For the Frenzi taxi application there are many different situations where a change in the PHP backend needs to be signalled immediately to one or more mobile devices. For example, to a taxi driver when a new booking comes in, or to a passenger when the taxi is arriving. This needs to scale up to support tens of thousands of devices online at any time, and also handle a rapidly growing set of situations where these real-time updates are needed, as new features like food delivery come online.

After many iterations we settled on a solution which supports horizontal scaling as traffic increases, is resilient to failures and disconnections, and can be applied generically to any existing standard HTTP API endpoint with minimal extra work.

Potential solutions

There are quite a few ways of getting a change from a server to a phone. Here are the ones worth considering, and what's wrong with each of them for this job.

Polling

The app asks the API for the latest data every few seconds.

The drawbacks are that every update arrives up to a whole interval late, and the load grows with every device and every screen even when nothing has changed, which also costs phones battery and data.

Long polling

The app makes a request that the server holds open until something changes or a timeout passes, and then asks again.

The drawbacks are that each waiting device ties up a PHP worker for the whole wait, and PHP has no good way to be woken up when the change happens, so each held request ends up checking the database in a loop.

Server-sent events

PHP keeps a response open to each device and writes an event into it whenever something changes.

The drawbacks are the same as long polling: a PHP process held open for every device that's online, and nothing to wake it up.

WebSockets in PHP

A long-running PHP server built on an event loop (Ratchet, ReactPHP or Swoole, say) holds the sockets itself.

The drawbacks are that it runs outside the normal request model, so the API's code and sessions don't simply carry over, and as soon as there's more than one of them they need a message bus between them, so that a change made through one reaches the devices connected to another.

Push notifications

Send a silent push through Apple's and Google's push services whenever something changes.

The drawbacks are that delivery can be delayed, and frequent silent pushes get throttled. They're made for alerting the person holding the phone, not for keeping what's on the screen up to date.

A hosted service

Use a hosted real-time service such as Pusher, Ably or Firebase, and publish to it from PHP.

The drawbacks are the cost with tens of thousands of devices connected, since you pay by the connection and the message, and that every update now passes through a third party.

Sending the data itself

Push the changed data to the device over a socket, so it doesn't need to ask for it.

The drawbacks are that every kind of update needs its own message format and its own permission checks, repeating what the API already does, and a device that misses a message is left showing the wrong thing with no way of knowing.

The solution we settled on

We eventually settled on making the real-time part as simple as possible.

From the mobile device, we make a request to the backend, which returns the data along with a token for that data's channel. Then we subscribe to the signal server, a small Node.js service, using that token. When PHP changes something, it signals on the channel, and anyone listening to that channel gets the signal.

The signal doesn't contain any information itself, except that this channel has changed. It's up to the device to go and fetch the latest data from the same API. This is one of the key things that keeps it simple: we don't send any data through the signal server, so it knows nothing about the data, the formats or who's allowed to see what, and every endpoint works in exactly the same way.

The other key thing is resilience. If the device gets disconnected from the signal server, it reconnects, and if its token has run out, it goes back to PHP for a new one. The signal service isn't stateless, but everything it knows is put back by the clients reconnecting, so when one goes down and comes back up, everything carries on. The signal services don't communicate with each other either. Each one just deals with its own subset of the channels, and adding more of them is how it scales.

This is the design that runs in Frenzi. The code below is a separate, self-contained version of it, with the same protocol and the same way of authenticating.

How it works

1. The app fetches data from an API endpoint as usual. Along with the data, the endpoint returns a token for its channel and the address of the signal service.
2. The app opens a WebSocket to the signal service with the token. The service checks the token and subscribes the connection to the channel.
3. When PHP changes something in the database, it knows which endpoints that affects, works out their channels, and sends an HTTP request to the signal service for each of them.
4. The signal service sends the channel id to every connection listening on that channel.
5. The app fetches from the same endpoint again, which gives it the new data and a fresh token.

Here's what happens when a driver accepts a ride. The passenger app is showing the ride while it waits for a driver, and is listening on the ride's channel. The driver taps accept, which is an ordinary POST to the API. PHP updates the ride in the database, then signals the passenger's channel. The signal service passes the signal on, and the passenger app fetches the ride again and shows the driver on the way.

Channels

A channel is named by a 32 character hex id, worked out from the endpoint's name and whatever it's scoped to, such as a passenger's id. It's the first half of a SHA-256 hash of those parts and a secret that only PHP and the signal services know:

define('SIGNAL_SECRET', getenv('SIGNAL_SECRET'));

function signalChannelId(array $parts): string {
    $json = json_encode($parts, JSON_UNESCAPED_SLASHES);
    return substr(hash('sha256', $json . "\n" . SIGNAL_SECRET), 0, 32);
}

$channelId = signalChannelId(['passenger/current-ride', '42']);

With the secret example-secret, that's 46eb4f2a3e53a20d5fbf44c79a276720. The endpoint and the code that sends signals call the same function with the same parts, so they always agree on the channel. Pass the ids as strings, so that 42 and '42' don't make two different channels.

Because of the secret, nobody can work out the channel for somebody else's ride. But the real protection is the token: only PHP can make one, and it only makes one in an endpoint that has already checked who's asking.

Tokens

A token says that this session may listen on this channel until a certain time. It's JSON, signed with a hash of its fields and the secret:

function signalToken(string $sessionId, string $channelId, int $ttlSeconds = 600): string {
    $payload = [
        'channelId' => $channelId,
        'expiryTimestamp' => time() + $ttlSeconds,
        'sessionId' => $sessionId,
    ];
    $canonical = json_encode($payload, JSON_UNESCAPED_SLASHES);
    $payload['hash'] = hash('sha256', $canonical . "\n" . SIGNAL_SECRET);
    return json_encode($payload, JSON_UNESCAPED_SLASHES);
}

The fields are in alphabetical order and encoded without spaces, so the signal service can rebuild exactly the same text and check the hash. Here's a token for passenger 42's ride:

{"channelId":"46eb4f2a3e53a20d5fbf44c79a276720","expiryTimestamp":1793004000,"sessionId":"k3j9d2l0b7v6q1m8","hash":"314b02322c04b79f7aad0c76e92ff059e8e8512021f5c623ad9ac0bbd5610882"}

The signal service never has to ask PHP whether a token is valid. It has the secret, so it can check the hash itself, and the expiry tells it whether the token is still current. Tokens last ten minutes, and they're only checked when a connection opens, so that's how long a device has to connect, or reconnect, before it needs a new one.

Signal endpoints

Any GET endpoint whose data can change becomes a signal endpoint by returning its channel along with its data. A small wrapper adds the token and the address of the signal service:

function signalUrl(string $channelId, string $scheme): string {
    return $scheme . '://signal-' . $channelId[0] . '.example.com';
}

function signalEndpoint(callable $handler): void {
    [$channelId, $payload] = $handler();

    header('Content-Type: application/json');
    echo json_encode([
        'signalUrl' => signalUrl($channelId, 'wss'),
        'token' => signalToken(session_id(), $channelId),
        'payload' => $payload,
    ], JSON_UNESCAPED_SLASHES);
}

The handler is the endpoint as it was, except that it also says which channel its data belongs to:

//  GET /api/passenger/current-ride
signalEndpoint(function () {
    $passengerId = loggedInPassengerId();
    $channelId = signalChannelId(['passenger/current-ride', (string)$passengerId]);

    return [$channelId, findCurrentRide($passengerId)];
});

That's the only change. The endpoint still checks who's logged in, loads its data in the same way and returns it as JSON, and the app finds two more fields beside it:

{
    "signalUrl": "wss://signal-4.example.com",
    "token": "{\"channelId\":\"46eb4f2a3e53a20d5fbf44c79a276720\",\"expiryTimestamp\":1793004000,...}",
    "payload": {"rideId": 5120, "status": "waiting", "pickup": "Leeds station"}
}

The first character of the channel id picks which signal service to use. That's how it scales, and it's explained under Scaling below.

Sending a signal

PHP sends a signal with one HTTP request to the signal service, authorised with the secret:

function signalTrigger(string $channelId): void {
    $curl = curl_init(signalUrl($channelId, 'https') . '/?channelId=' . $channelId);
    curl_setopt_array($curl, [
        CURLOPT_CUSTOMREQUEST => 'PUT',
        CURLOPT_HTTPHEADER => ['Authorization: ' . SIGNAL_SECRET],
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_TIMEOUT => 2,
    ]);
    curl_exec($curl);
    $status = curl_getinfo($curl, CURLINFO_RESPONSE_CODE);

    if ($status !== 204) {
        error_log("signalTrigger: channel $channelId got status $status");
    }
}

PUT /?channelId=46eb4f2a3e53a20d5fbf44c79a276720 HTTP/1.1
Host: signal-4.example.com
Authorization: example-secret

HTTP/1.1 204 No Content

If the signal can't be sent, it's logged and the request carries on. The change has already been made, and the devices will see it the next time they fetch.

The part that takes some thought is knowing which endpoints a change affects. When a ride changes, the passenger's current ride and the driver's current ride are both different, so both channels are signalled:

function onRideChanged(PDO $db, int $rideId): void {
    $ride = findRide($db, $rideId);

    signalTrigger(signalChannelId(['passenger/current-ride', (string)$ride->passengerId]));
    if ($ride->driverId !== null) {
        signalTrigger(signalChannelId(['driver/current-ride', (string)$ride->driverId]));
    }
}

Anything that changes a ride calls this once it's done. Here's the driver accepting one:

//  POST /api/driver/accept-ride
$db->beginTransaction();
$db->prepare("UPDATE ride SET driver_id = ?, status = 'accepted' WHERE id = ? AND status = 'waiting'")
   ->execute([$driverId, $rideId]);
$db->commit();

onRideChanged($db, $rideId);

Signal after the commit. If the signal goes out first, the passenger app can fetch the ride before the change can be seen, and get the old one.

In Frenzi the code that changes rides doesn't call the signals directly. The ride module announces that a ride has changed, and the passenger and driver modules each listen for that and signal their own channels. A new feature can listen for changes in the same way, without the ride code having to know about it.

The signal service

The signal service is a small Node.js program, using the ws package for the WebSockets. It keeps one map, from each channel id to the connections listening on it:

const http = require('http');
const crypto = require('crypto');
const { WebSocketServer } = require('ws');

const SECRET = process.env.SIGNAL_SECRET;
const PORT = Number(process.env.PORT);

//  Channel id -> the set of sockets listening on it
const channels = new Map();

//  PHP signals a channel: send its id to everyone listening
const server = http.createServer((request, response) => {
    const url = new URL(request.url, 'http://localhost');
    const channelId = url.searchParams.get('channelId');
    if (request.method !== 'PUT' || request.headers.authorization !== SECRET) {
        response.writeHead(403);
        response.end();
        return;
    }
    const sockets = channels.get(channelId);
    if (sockets !== undefined) {
        for (const socket of sockets) {
            socket.send(channelId);
        }
    }
    response.writeHead(204);
    response.end();
});

//  An app connects with a token, which subscribes it to one channel
const wss = new WebSocketServer({ server });

wss.on('connection', (socket, request) => {
    const url = new URL(request.url, 'http://localhost');
    const token = verifyToken(url.searchParams.get('token'));
    if (token === null) {
        socket.close(1008, 'Token rejected');
        return;
    }

    const channelId = token.channelId;
    if (!channels.has(channelId)) {
        channels.set(channelId, new Set());
    }
    channels.get(channelId).add(socket);

    socket.on('close', () => {
        const sockets = channels.get(channelId);
        sockets.delete(socket);
        if (sockets.size === 0) {
            channels.delete(channelId);
        }
    });

    //  Signal once straight away, to cover anything missed while connecting
    socket.send(channelId);
});

server.listen(PORT);

The token comes in the query string, because browsers can't add headers to a WebSocket request. Checking it is the same as making it, the other way round: rebuild the text PHP signed, hash it with the secret, and compare.

function verifyToken(text) {
    let token;
    try {
        token = JSON.parse(text);
    } catch (e) {
        return null;
    }
    if (typeof token !== 'object' || token === null) return null;
    if (typeof token.sessionId !== 'string' || token.sessionId === '') return null;
    if (typeof token.channelId !== 'string' || !/^[0-9a-f]{32}$/.test(token.channelId)) return null;
    if (typeof token.expiryTimestamp !== 'number') return null;
    if (typeof token.hash !== 'string' || !/^[0-9a-f]{64}$/.test(token.hash)) return null;
    if (Date.now() / 1000 > token.expiryTimestamp) return null;

    //  The same fields, in the same order, as PHP signed them
    const canonical = JSON.stringify({
        channelId: token.channelId,
        expiryTimestamp: token.expiryTimestamp,
        sessionId: token.sessionId,
    });
    const expected = crypto.createHash('sha256').update(canonical + '\n' + SECRET).digest('hex');
    if (!crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(token.hash))) return null;
    return token;
}

A bad or expired token gets close code 1008, which tells the app to fetch a new token rather than keep trying the same one. A good one is added to its channel, and straight away it's sent the channel id once. That makes the app fetch again, which covers anything that changed between it fetching the data and the connection being ready.

Each connection listens on exactly one channel, so there are no subscribe or unsubscribe messages: the token says it all. And the whole of the service's state is that one map, which is rebuilt from nothing every time it starts, as the apps reconnect. In production it sits behind TLS, so the apps connect with wss://.

On the device

In the apps all of this is hidden inside the call that fetches data. Screen code asks for an endpoint and gets data that keeps itself up to date: it never sees a token or a socket. Here's the same thing as a plain JavaScript function:

function liveGet(url, onData) {
    let signalUrl = null;
    let token = null;
    let socket = null;
    let attempt = 0;
    let latest = 0;
    let stopped = false;

    //  Wait 1, 2, 4, 8, 16, then 32 seconds between tries
    function retry(next) {
        const delay = Math.min(1000 * 2 ** attempt, 32000);
        attempt++;
        setTimeout(next, delay);
    }

    function load() {
        if (stopped) return;
        const number = ++latest;
        fetch(url, { credentials: 'include' })
            .then((response) => {
                if (!response.ok) throw new Error('HTTP ' + response.status);
                return response.json();
            })
            .then((body) => {
                if (stopped || number !== latest) return;   //  a newer fetch has started
                signalUrl = body.signalUrl;
                token = body.token;
                onData(body.payload);
                connect();
            })
            .catch(() => {
                if (number === latest) retry(load);
            });
    }

    function connect() {
        if (stopped || socket !== null) return;
        if (Date.now() / 1000 > JSON.parse(token).expiryTimestamp - 10) {
            load();   //  the token has run out: fetch again for a new one
            return;
        }
        socket = new WebSocket(signalUrl + '/?token=' + encodeURIComponent(token));
        socket.onopen = () => {
            attempt = 0;
        };
        socket.onmessage = () => {
            load();
        };
        socket.onclose = (event) => {
            socket = null;
            if (stopped) return;
            if (event.code === 1008) {
                retry(load);      //  the token was turned down: fetch again for a new one
            } else {
                retry(connect);   //  the connection dropped: try again
            }
        };
    }

    load();

    return () => {
        stopped = true;
        if (socket !== null) socket.close();
    };
}

const stop = liveGet('/api/passenger/current-ride', (ride) => {
    showRide(ride);
});

Each call fetches the endpoint, opens one WebSocket for its channel, and fetches again whenever the channel's id arrives. If a fetch comes back after a newer one has started, it's thrown away, so a slow response can't overwrite a newer one. The socket stays open while the data is fetched again; the new token is kept for the next time it has to connect.

Resilience

Connections drop all the time on phones, and the signal service gets restarted for updates. Either way the app does the same thing: it reconnects, waiting 1, 2, 4, 8 and 16 seconds between tries and then every 32 seconds, for as long as it takes.

Before it connects, it looks at the token's expiry. If the token has run out, or is about to, it goes back to the API instead, which gives it fresh data and a new token in one request. If the service turns the token down anyway, with close code 1008, it does the same.

Nothing is lost while it's disconnected. Signals sent in that time go nowhere, but the service sends the channel id as soon as the app is back, and the app fetches the latest data.

So the resilience is pretty simple: if a service goes down, when it comes back up everything reconnects to it. There's nothing to back up or copy between servers, because the only state is which connections are listening on which channels, and the apps put that back by reconnecting. If a server is gone for good, point its names at another one and its apps reconnect there.

Scaling

Because the signal is so simple, the services don't need to talk to each other. All that matters is that PHP sends each signal to the same service the app is listening on, so each channel needs exactly one home.

The channel id is a hash, so its characters are spread evenly. The first character picks one of 16 names in DNS, signal-0.example.com to signal-f.example.com, and DNS says which server each name is on:

Both sides get the name from signalUrl: PHP when it sends a signal, and the endpoint wrapper when it tells the app where to listen.

To start with, all 16 names can point at one server. When it gets busy, move half of the names to a second server, then spread them over four, and so on up to sixteen. Past that, the first two characters give 256 names. Moving a name is just a change in DNS. Once the change has spread, restart the service on the old server, so that its apps reconnect, find the new server, and fetch again.

Another easy option is to rewrite the service in something more efficient, such as Rust. That hasn't been necessary at this stage, because Node.js with WebSockets can handle thousands of active connections on a single server. Nothing else would have to change: PHP and the apps only know the protocol.

Things to know

• Each WebSocket is one channel, so a screen showing three live endpoints opens three connections. That's what lets every channel live on its own server, and a connection that just sits waiting costs very little.
• The token is only checked when the connection opens. An open connection keeps listening after its token has run out, and logging out doesn't close it. That's safe because a signal tells the device nothing, and when it fetches, the API checks who's asking as usual.
• A ride that changes three times in a second can make the passenger app fetch three times. Each fetch gets the latest data, so it's always right, just a little more work than it needs to be.
• Both sides must encode the token in exactly the same way. Keep its fields to plain ASCII strings and whole numbers, in alphabetical order, with slashes unescaped on the PHP side.
• PHP and every signal service share the secret. Keep it in the servers' configuration, never in the app.