binary-packet

Lightweight and hyper-fast, zero-dependencies, TypeScript-first, schema-based binary packets serialization and deserialization library.
Originally made to be used for an ultrafast WebSockets communication with user-defined type-safe messages between client and server, with the smallest bytes usage possible.

Supports serializing into and deserializing from DataViews, ArrayBuffers and Buffers (NodeJS/Bun only).
To achieve the maximum performance is it always advised to use node Buffer(s) when available.

Installation

Node:
npm install binary-packet

Bun:
bun add binary-packet

Features & Specification

Define the structure of the packets through unique Packet IDs and "schema" objects.
This "schema" object is simply called Definition and defines the shape of a packet: specifically its fields and their types.

Fields / Data types

Currently, these kinds of fields are supported:

Type	Description	Values	Size (bytes)
Field.UNSIGNED_INT_8	8 bits unsigned integer	0 - 255	1
Field.UNSIGNED_INT_16	16 bits unsigned integer	0 - 65535	2
Field.UNSIGNED_INT_32	32 bits unsigned integer	0 - 4294967295	4
Field.INT_8	8 bits signed integer	-128 - 127	1
Field.INT_16	16 bits signed integer	-32768 - 32767	2
Field.INT_32	32 bits signed integer	-2147483648 - 2147483647	4
Field.FLOAT_32	32 bits IEEE754 floating-point		4
Field.FLOAT_64	64 bits IEEE754 floating-point		8
BinaryPacket	BinaryPacket "subpacket"	BinaryPacket	size(BinaryPacket)
FieldString	ASCII or single octet utf-8 chars string	Up to 65536 chars	2 + length
FieldArray	Dynamically-sized array of one of the types above	Up to 256 elements	1 + length * size(Element)
FieldFixedArray	Statically-sized array of one of the types above	Any pre-defined numbers of elements	length * size(Element)
FieldBitFlags	Boolean flags packed into a single 8 bits integer	Up to 8 boolean flags	1
FieldOptional	Optional BinaryPacket "subpacket"	BinaryPacket | undefined	1 + size(BinaryPacket)

As shown, both arrays and nested objects ("subpackets") are supported.
Note: FieldFixedArray is much more memory efficient and performant than FieldArray, but require a pre-defined length.

Pattern matching

The library exposes an easy way to "pattern match" packets of a yet-unknown-type in a type-safe manner through a visitor pattern.
For an example, search for "pattern matching" in the examples below.

Usage Examples

Example: (incomplete) definition of a simplistic board game

import { BinaryPacket, Field, FieldArray } from 'binary-packet'

// Suppose we have a game board where each cell is a square and is one unit big.
// A cell can be then defined by its X and Y coordinates.
// For simplicity, let's say there cannot be more than 256 cells, so we can use 8 bits for each coordinate.
const Cell = {
  x: Field.UNSIGNED_INT_8,
  y: Field.UNSIGNED_INT_8
}

// When done with the cell definition we can create its BinaryPacket writer/reader.
// NOTE: each BinaryPacket needs an unique ID, for identification purposes and error checking.
const CellPacket = BinaryPacket.define(0, Cell)

// Let's now make the definition of the whole game board.
// You can also specify arrays of both "primitive" fields and other BinaryPackets.
const Board = {
  numPlayers: Field.UNSIGNED_INT_8,
  otherStuff: Field.INT_32,
  cells: FieldArray(CellPacket)
}

// When done with the board definition we can create its BinaryPacket writer/reader.
// NOTE: each BinaryPacket needs an unique ID, for identification purposes and error checking.
const BoardPacket = BinaryPacket.define(1, Board)

//////////////////
// WRITING SIDE //
//////////////////
const buffer = BoardPacket.writeNodeBuffer({
  numPlayers: 1,
  otherStuff: 69420,
  cells: [
    { x: 0, y: 0 },
    { x: 1, y: 1 }
  ]
})

// ...
// sendTheBufferOverTheNetwork(buffer)
// ...

//////////////////
// READING SIDE //
//////////////////
import assert from 'assert'

// ...
// const buffer = receiveTheBufferFromTheNetwork()
// ...

const board = BoardPacket.readNodeBuffer(buffer)

assert(board.numPlayers === 1)
assert(board.otherStuff === 69420)
assert(board.cells.length === 2)
assert(board.cells[0].x === 0)
assert(board.cells[0].y === 0)
assert(board.cells[1].x === 1)
assert(board.cells[1].y === 1)

Example: pattern matching

import assert from 'assert/strict'
import { BinaryPacket, Field } from 'binary-packet'

// Packet A definition
const A = BinaryPacket.define(1)

// Packet B definition: This is the kind of packets that we care about in this example!
const B = BinaryPacket.define(2, { data: Field.UNSIGNED_INT_8 })

// Packet C definition
const C = BinaryPacket.define(3)

// Assume the following packet comes from the network or, for some other reason, is a buffer we do not know anything about.
const buffer = B.writeNodeBuffer({ data: 255 })

BinaryPacket.visitNodeBuffer(
  buffer,

  A.visitor(() => assert(false, 'Erroneously accepted visitor A')),

  B.visitor(packet => {
    // Do something with the packet
    assert.equal(packet.data, 255)
    console.log('Accepted visitor B:', packet)
  }),

  C.visitor(() => assert(false, 'Erroneously accepted visitor C'))
)

Benchmarks & Alternatives

Benchmarks are not always meant to be taken seriously.
Most of the times the results of a benchmark do not actually show the full capabilities of each library.
So, take these "performance" comparisons with a grain of salt; or, even better, do your own benchmarks with the actual data you need to serialize/deserialize.

This library has been benchmarked against the following alternatives:

• msgpackr - A very popular, fast and battle-tested library. Currently offers more features than binary-packet, but it always appears to be slower and is also less type-safe.
• restructure - An older, popular schema-based library, has some extra features like LazyArrays, but it is much slower than both binary-packet and msgpackr. And, sadly, easily crashes with complex structures.

The benchmarks are executed on three different kinds of packets:

• EmptyPacket: basically an empty javascript object.
• SimplePacket: objects with just primitive fields, statically-sized arrays and a string.
• ComplexPacket: objects with primitives, statically-sized arrays, dynamically-sized arrays, bitflags, a string, an array of strings and other nested objects/arrays.

You can see and run the benchmarks yourself if you:

• Clone the repository.
• Launch npm run benchmark.

Disclaimer

This library is still very new, thus not "battle-tested" in production enough, or may still have missing important features.
If you plan on serializing highly sensitive data or need to guarantee no crashes, use an alternative like msgpackr until this library becomes 100% production-ready.

Contribute

Would like to have more complex, but still hyper-fast and memory efficient, features?
Contribute on GitHub yourself or, alternatively, buy me a coffee!