15 March 2023

JavaScript Node.js Performance Algorithms

Common to NodeJS and browser: ArrayBuffer

There is an ArrayBuffer object, which is the underlying binary data storage for a set of other array types:

  • DataView: This provides a way to read and write arbitrary data from an ArrayBuffer.
  • Uint8Array: This allows access to individual bytes, as unsigned values from 0 to 255
  • Int8Array: This allows access to individual bytes, as signed values from -128 to 127
  • Uint16Array: This allows access to individual 16-bit elements, as unsigned values from 0 to 65,535
  • Int16Array: This allows access to individual 16-bit elements, as signed values from -32,768 to 32,767
  • Uint32Array: This allows access to individual 32-bit elements, as unsigned values from 0 to 4,294,967,295
  • Int32Array: This allows access to individual 32-bit elements, as signed values from -2,147,483,648 to 2,147,483,647
  • Float32Array: This allows access to individual 32-bit elements, as floating-point values in IEEE 754 format
  • Float64Array: This allows access to individual 64-bit elements, as floating-point values in IEEE 754 format
  • SharedArrayBuffer: A type of ArrayBuffer that can be shared between multiple threads or processes

These objects are created from an ArrayBuffer like this:

const myArrayBuffer = new ArrayBuffer(10)  //  10 bytes
const bytes = new Uint16Array( myArrayBuffer )
A 10-byte ArrayBuffer viewed as an array of 16-bit values

Objects only available in NodeJS

NodeJS has Buffer, which is what you get back from reading a file or a socket. A Buffer is a Uint8Array with extra methods for converting to and from strings, so it can be used anywhere a Uint8Array can.

There is something to be very careful of with Buffer which will surprise you if you are not aware of it, and that is that your data might not be the only thing in the Buffer's underlying ArrayBuffer, and might not be at the start. Small Buffers are allocated out of a shared pool, so if you go to the buffer property to get at the raw ArrayBuffer, you get the whole pool:

const other = Buffer.from( "Other data" )
const myBuffer = Buffer.from( "Hello" )

const everything = new Uint8Array( myBuffer.buffer )

console.log( everything.length )
console.log( myBuffer.byteOffset, myBuffer.byteLength )
console.log( everything.slice( 0, 24 ).toString() )  //  Surprise!
Looking at the ArrayBuffer behind a small Buffer, after another small Buffer has been made

This prints out something like this:

65536
16 5
79,116,104,101,114,32,100,97,116,97,0,0,1,0,0,0,72,101,108,108,111,0,0,0
The output in Node.js 26: a 64k pool, with "Other data" at the start and "Hello" at offset 16

The proper way to get at just your data is to make the view with the Buffer's byteOffset and byteLength:

const justMine = new Uint8Array( myBuffer.buffer, myBuffer.byteOffset, myBuffer.byteLength )

console.log( justMine.toString() )  //  72,101,108,108,111
A view of only the Buffer's own five bytes, using its byteOffset and byteLength

The same goes for any typed array: two views can share one ArrayBuffer, so whenever you pass .buffer on to something else, pass the offset and length with it.

Objects only available in browser

In the browser, binary data usually arrives as a Blob: a block of bytes with a MIME type, which can't be read directly. A File, from an <input type="file"> or a drag and drop, is a Blob with a name and a date. To get at the bytes, ask for an ArrayBuffer, which is asynchronous:

const file = document.querySelector( 'input[type=file]' ).files[0]

const bytes = new Uint8Array( await file.arrayBuffer() )
Reading the bytes of a file the user has picked, as a Uint8Array

Going the other way, new Blob( [ bytes ], { type: 'application/octet-stream' } ) wraps typed arrays back up, and URL.createObjectURL() turns a Blob into a URL that can be downloaded or shown in the page. The older FileReader object does the same job as arrayBuffer() with callbacks.