Self-describing code

Articles

It is possible to write code in a manner that allows you to choose whether to execute it or grab a description of what the code does as an Abstract Syntax Tree. All using the same code.

The purpose of grabbing a Syntax Tree might be as a step towards converting the code into a different language, either as a one-off process to migrate to a different platform, as part of the build process for your application, or even at runtime to allow the same underlying code to run in different environments depending on the circumstances.

This behaviour can be implemented with a small library (which I have called `Meta`) and a certain way of working when writing code.

This assumes your code is composed from the following basic building blocks:

  • Type definitions

. Enumerated Types . Object Types . Array Types . Optional Types . Scalar Types

  • Function definitions

. Global function definitions . Local (closure) function definitions . Function Inputs

. Value Inputs
. Closure Inputs ("Holes")

. Calls to functions . Assignment to local variables . Returning values . Native function definitions

This covers typesafe functional programming and is enough to make a wide range of applications.

Notice there are no control flow constructs such as `if`, `while`, and `for`. These can all be achieved with functions, as we'll see below.

We will build up a subset of JavaScript which can be considered a domain-specific language (DSL) and can be used to express a wide variety of applications.

Lets start by imagining that no JavaScript constructs at all are allowed, and then add specific allowed patterns one by one.

Module layout

The first thing to add is a way to organise code. You need more than just a single global namespace for most applications.

A simple JavaScript object provides a neat way to create namespaces, putting every type and function you will define into a hierarchy:

const MyModule = {
    MySubModule: {
        ... contents of submodule ...
    },
    ... other contents of module ...
}

As this is the first construct introduced, there is nothing defined to go inside the modules with the exception of submodules. The set of all allowed programs currently consists of all possible trees of modules and submodules with all possible names, and none of these programs can 'do' anything.

Modules in this form can be easily packaged up as a NodeJS module or can be served to a web browser in exactly the same format and used in the same way. This will intentionally be the case all the way through, to produce so called "isomorphic" JavaScript.

If this format was to be stored in a data structure, it would look like this (these types are created using JavaScript in the format described in this document) :

//  This lives in `Code.Meta.Structure` module
Module: (...values) => Meta.Object( Code.Meta.Structure.Module, values, {
    name: Meta.String,
    subModules: Meta.Array(Code.Meta.Structure.Module)
})

This format is also easy to parse, here is a parse rule for it (this rule is JavaScript in the format described in this document, and uses a Parse module for text parsing) :

//  This parsing rule lives in the module `Code.Meta.Parse`
Module: () => Meta.Function( Code.Meta.Structure.Module, () => {
    const constKeyword = Parse.Match.Text.Exact('const')
    Code.Meta.Parse.Whitespace()  //  This counts comments as whitespace
    const moduleName = Code.Meta.Parse.Identifier()
    Code.Meta.Parse.OptionalWhitespace()  //  Counts comments as whitespace
    Parse.Match.Text.Exact('=')
    Code.Meta.Parse.OptionalWhitespace()
    Parse.Match.Text.Exact('{')
    Code.Meta.Parse.OptionalWhitespace()
    const subModules = Code.Meta.Parse.SubModules()
    Code.Meta.Parse.OptionalWhitespace()
    Parse.Match.Text.Exact('}')
    return Code.Meta.Structure.Module( moduleName, subModules )
},

SubModules: () => Meta.Function(Code.Meta.Structure.ModuleContent, () => {
    const subModules = Parse.Match.List(
        //  List separator, this is a comma with optional whitespace around
        Meta.Closure( () => {
            Code.Meta.Parse.OptionalWhitespace()
            Parse.Match.Text.Exact(',')
            Code.Meta.Parse.OptionalWhitespace()
        }),
        //  List item, this is currently allowed to be a submodule
        Meta.Closure( () => {
            const submoduleName = Code.Meta.Parse.Identifier()
            Code.Meta.Parse.OptionalWhitespace()
            Parse.Match.Text.Exact(':')
            Code.Meta.Parse.OptionalWhitespace()
            Parse.Match.Text.Exact('{')
            Code.Meta.Parse.OptionalWhitespace()

            Code.Meta.Parse.OptionalWhitespace()
            Parse.Match.Text.Exact('}')
        }),
    )
    //  Optional comma is allowed after the list of submodules, but only if there were some
    const anySubModules = Collection.Array.NotEmpty( subModules )
    Logic.If( anySubModules,
        Meta.Closure( () => {
            Parse.Match.Optional(
                Meta.Closure( () => {
                    Code.Meta.Parse.OptionalWhitespace()
                    Parse.Match.Text.Exact(',')
                })
            )
        })
    )
    return Code.Meta.Structure.ModuleContent( subModules )
}

Types

There are various kinds of type which can be composed to describe any arbitrary data:

  • Scalar Types

. Meta.Integer . Meta.Float . Meta.String . Meta.Boolean . Meta.Buffer

  • Higher-order Types

. Array . Dictionary . Object . Enumerated Type

  • Binary Data Buffers

Object Types

For an object you need to specify what the fields are, and define a constructor function:

const MyModule = {
    MemberData: (...value) => Meta.Object( MyModule.MemberData, value, {
        name: Meta.String,
        isUpgraded: Meta.Boolean,
    })
}

//  Use the constructor to create an instance. Bob has upgrades.
const member = MyModule.MemberData( 'Bob', true )

Enumerated Types

A Enumerated Type can be a value of one of a set of possible types.

This example defines `MyModule.MemberType`, which can either contain a `MyModule.BasicMemberData` or a `MyModule.ProMemberData`:

const MyModule = {
    MemberType: (value) => Meta.Enum( MyModule.MemberType, value, {
        basic: MyModule.BasicMemberData,
        pro: MyModule.ProMemberData,
    })
}

The form above is sometimes known as a Sum Type, and often enums in programming languages are simpler, allowing one of a specific set of fixed values instead of allowing more complex data types.

For a simple case where you want one of a set of possible values, use `Meta.Void`:

const MyModule = {
    MemberLoggedIn: (value) => Meta.Enum( MyModule.MemberType, value, {
        yes: Meta.Void,
        no: Meta.Void,
    })
}

How it looks

Lets start by having a look at the final result of applying some principles to JavaScript code, and then look at each principle in turn.