Type safety at the boundaries

At SketchUp we reviewed a variety of different libraries for turning raw data, which comes over an API connection for instance, into type checked TypeScript objects. Each of them required us to write an additional schema, and they generally had limitations which made them incompatible with our system. The amount of work to customise a library was more than the amount of work it would take to make one ourselves, specifically for our needs.

Or more precisely, to not have a library at all, and just have a pattern for importing data. This article describes that pattern: immutable data objects, each with a FromJson function, and a very small assert library that does the checking.

The problem

TypeScript's types only exist while the code is being compiled. Data that arrives at runtime, from an API, a WebSocket, local storage or a file, has whatever shape the other end gave it, and TypeScript will happily believe whatever type you tell it:

const project = (await response.json()) as ProjectData;

//  Compiles, but nothing has checked that any of this is true
const count = project.models.length;

If the API sends the ID as a string, or leaves the list of models out, the mistake doesn't show up here. It shows up later, somewhere else, when something odd appears on the screen. The types are only as good as the data coming in, so the data has to be checked as it comes in.

The libraries

There were plenty of libraries for this at the time:

• io-ts: runtime types built from combinators, with the TypeScript type worked out from them, and results as an Either from fp-ts.
• runtypes: a similar idea, with a check that throws instead of an Either.
• class-validator with class-transformer: decorators on a class's properties, with the transformer creating the class and the validator checking its fields.
• Ajv: a fast JSON Schema validator, with the TypeScript types written separately or generated from the schema.
• Yup and Joi: schema validation from the world of forms and Node.js servers.
• Superstruct and the newer Zod: small schema libraries that work out the TypeScript type from the schema.

They all start with a schema. Here's a project, with its owner and its models, in io-ts:

import * as t from 'io-ts';
import { isLeft } from 'fp-ts/lib/Either';
import { PathReporter } from 'io-ts/lib/PathReporter';

const Member = t.type({
    id: t.number,
    name: t.string,
    role: t.keyof({ viewer: null, editor: null, owner: null }),
});

const Model = t.type({
    id: t.number,
    name: t.string,
    sizeMb: t.number,
});

const Project = t.type({
    id: t.number,
    name: t.string,
    description: t.union([t.string, t.null, t.undefined]),
    tags: t.array(t.string),
    owner: Member,
    models: t.array(Model),
});

type Project = t.TypeOf<typeof Project>;

function decodeProject(json: unknown): Project {
    const result = Project.decode(json);
    if (isLeft(result)) {
        throw new Error(PathReporter.report(result).join('\n'));
    }
    return result.right;
}

It works, and it's well thought out, but it didn't fit. What comes out is a plain object of the schema's type, and our data lived in immutable data objects, classes with their fields in the constructor and functions for the calculations based on them. So every type would need a schema, a class, and a step converting one into the other, all the way down through the owner and the models. The decorator libraries go the other way and start from the class, but class-transformer creates it with no constructor arguments and then sets its fields one by one, which doesn't work for a class whose fields are all readonly and set in the constructor.

Each of them could have been bent to fit, but by then we'd have been maintaining the bending, as well as a schema for every type.

The pattern

Instead, each immutable data object gets a static FromJson function. It takes a value of unknown type, checks it, and calls the constructor. The checks come from a small assert library, where every check takes a value and either returns it, typed, or throws an AssertException:

export class ProjectData {
    constructor(
        readonly id: number,
        readonly name: string,
        readonly description: string | undefined,
        readonly tags: string[],
        readonly owner: MemberData,
        readonly models: ModelData[],
    ) {}

    static FromJson(json: unknown) {
        const object = Assert.Object(json);
        return new ProjectData(
            Assert.Integer(object.id),
            Assert.String(object.name),
            Assert.Optional(Assert.String)(object.description),
            Assert.Array(Assert.String)(object.tags),
            MemberData.FromJson(object.owner),
            Assert.Array(ModelData.FromJson)(object.models),
        );
    }

    get totalSizeMb() {
        const sizes = this.models.map((model) => model.sizeMb);
        const total = sizes.reduce((a, b) => a + b, 0);
        return total;
    }
}

The function first checks that what it was given is actually an object, then constructs the data object, checking one field per line, in the same order as the constructor. There's no schema: the constructor's arguments are the description of the type, and the FromJson function sits right underneath them, so it's easy to see that every field is covered. If a field is added to the constructor and not to FromJson, the compiler says so.

Once the object is built, it's type safe all the way through. Nothing after this point needs to check anything again.

Objects inside objects

The owner and the models are data objects too, each with a FromJson of its own:

export class MemberData {
    constructor(
        readonly id: number,
        readonly name: string,
        readonly role: 'viewer' | 'editor' | 'owner',
    ) {}

    static FromJson(json: unknown) {
        const object = Assert.Object(json);
        return new MemberData(
            Assert.Integer(object.id),
            Assert.String(object.name),
            Assert.OneOf(['viewer', 'editor', 'owner'] as const)(object.role),
        );
    }
}

export class ModelData {
    constructor(
        readonly id: number,
        readonly name: string,
        readonly sizeMb: number,
    ) {}

    static FromJson(json: unknown) {
        const object = Assert.Object(json);
        return new ModelData(
            Assert.Integer(object.id),
            Assert.String(object.name),
            Assert.Number(object.sizeMb, 0),
        );
    }
}

A FromJson function has the same shape as a check, a value of unknown type in and a typed value out, so it can go anywhere a check can go. For a single object, ProjectData.FromJson calls MemberData.FromJson directly. For a list of them, it goes inside Assert.Array.

That works because the assert library is set up with curried functions, the same way as in Composable typechecking in JavaScript. Assert.Array is called once with the check for each element, and gives back a check for the whole array, so Assert.Array(Assert.String) is a check for an array of strings, and Assert.Array(ModelData.FromJson) is a check for an array of models. It returns a new array of whatever the element check returned, so the result really is a ModelData[], made of real ModelData objects, and TypeScript knows it. It works to any depth: Assert.Array(Assert.Array(Assert.Number)) checks a grid of numbers.

Missing values

Assert.Optional takes a check and gives back one that also allows the value to be missing. APIs send missing values in different ways, as null or by leaving the field out, so the optional check turns both into undefined. That way the rest of the application only ever has to deal with one kind of missing value, which is the convention in Angular best practices for large teams, and the boundary is where it's enforced.

Fixed sets of strings

A member's role is one of three strings. Assert.OneOf checks for one of them, and with as const on the list, TypeScript gives the result the type 'viewer' | 'editor' | 'owner' rather than just string, which is exactly the type the constructor wants.

When it fails

A check that fails throws an AssertException saying what it expected and what it got. Checks inside an array or a dictionary add where they were, and Assert.Enter can add a description of your own around anything:

Assert.Enter('decoding project 42', () => ProjectData.FromJson(json));

//  AssertException: Expected number >= 0; got -1
//    while: decoding project 42
//    while: checking element 1

The exception is thrown as the data arrives, from the code that received it, which is the best possible place to find out about it.

At the boundary

The only reliable way to deal with type safety is to do it right at the boundary, where data comes into the application. If values are passed around unchecked and only checked just before they're used, every place that uses them has to check them, and by the time one fails nobody knows where it came from.

So FromJson is called exactly once for each piece of data, at the point it arrives. In an Angular application where every request goes through one communications library, it can go in the library itself: each request carries a reference to the FromJson function for decoding its data, as described in The type safety boundary, and no component ever sees anything that hasn't been checked.

Nothing about the pattern is specific to Angular. In React, the same call goes wherever the data is fetched, such as a hook:

function useProject(projectId: number) {
    const [project, setProject] = useState<ProjectData | undefined>(undefined);
    useEffect(() => {
        const url = '/api/projects/' + projectId;
        fetch(url)
            .then((response) => response.json())
            .then((json) => setProject(ProjectData.FromJson(json)));
    }, [projectId]);
    return project;
}

And in plain JavaScript the checks work exactly the same way. There are no types for the compiler to check afterwards, but the data is still known to be the right shape from the moment it arrives.

And back again

The other half of serialisation is turning the objects back into JSON to send. That's a utility function on the data object, alongside its calculations:

toJson() {
    const owner = this.owner.toJson();
    const models = this.models.map((model) => model.toJson());
    return {
        id: this.id,
        name: this.name,
        description: this.description,
        tags: this.tags,
        owner,
        models,
    };
}

Going this way there's nothing to check, because the object is already type safe. JSON.stringify leaves out the undefined description, and FromJson reads it back as undefined, so a project survives the round trip unchanged.

Between them, FromJson and toJson are the whole serialisation layer: two small functions on each data object, written in the same style as the rest of the code, with no schema, no decorators and no library beyond the assertions.

The assert library

The library is one TypeScript file, in the source code below. Every check returns the value it was given, typed, or throws an AssertException.

Alongside them, Assert.Check(condition, message?) fails unless a condition is true, Assert.Fail(message) fails straight away, and Assert.Enter(description, callback) adds a description to any failure inside the callback.

A check is just a function from unknown to a type, so anything with that shape can be passed to Array, Dictionary or Optional, including every FromJson. That's what lets the library stay this small: it doesn't need to know anything about the application's types, because the data objects describe themselves.