Specification and Bindings for Swift Binary Protocol

• Installing sbp2json, json2sbp, json2json and related tools
• Building / installing
  • Using Docker
    • Fetching the prebuilt image from DockerHub
    • Creating your own image
    • Using the docker image
  • Installing from package managers
  • Installing development Python versions
  • Adding development version as a pip dependency
  • Installing from source
• SBP Development Procedures
• SBP Protocol Specification
• JSON Schema Definitions
• Kaitai Struct Format Descriptions
• LICENSE

The Swift Navigation Binary Protocol (SBP) is a fast, simple, and minimal binary protocol for communicating with Swift devices. It is the native binary protocol used by the Piksi GPS receiver to transmit solutions, observations, status and debugging messages, as well as receive messages from the host operating system, such as differential corrections and the almanac.

This project provides language-agnostic specification and documentation for messages used with SBP, a compiler for generating message bindings, and client libraries in a variety of languages. This repository is organized into the following directory structure:

• docs: Protocol documentation and message definitions.
• spec: Machine readable protocol specification in YAML.
• generator: Simple, template-based generator for different languages.
• python: Python client and examples.
• c: C client library and examples.
• haskell: Haskell client and examples.
• java: Java client library and examples.
• javascript: JavaScript client library and examples.
• rust: Rust client library and examples.
• sbpjson: Tools for parsing SBP-JSON.
• kaitai: Kaitai Struct Format Description and generated code.

Except for the generator, all of the above are generated and should not be modified directly.

Installing sbp2json, json2sbp, json2json and related tools

This repository also provides the following utilities for comprehending and inspecting SBP data:

• sbp2json: converts SBP binary into a JSON representation, in which field names and values are expanded into JSON objects, common fields such as "message type" and "payload" are included as well.

• json2sbp: uses the "message type", "payload" and related fields from an SBP JSON stream to reconstruct the binary representation.

• json2json: some tools (notably the Swift GUI Console) produce abbreviated JSON logs with only common fields such as "message type" and "payload", the json2json tool expands these JSON objects to include fields specific the individual message.

To install a released version of these tools, visit the releases page and download an archive for your platform.

To install from source, you can use Rust's cargo tool (first install Rust), then run:

cargo install --git https://github.com/swift-nav/libsbp.git --bins

There's also a Haskell version available which can be installed by running stack install in the ./haskell directory of a checkout of this repo (after installing stack) or by visiting the releases by and downloading an sbp_tools_haskell archive. This variant of the tools predate the Rust and Python versions, and also includes an sbp2yaml tool as well as a sbp2prettyjson tool.

Finally, a Python version of the sbp2json tool exists, which is installable on any platform that supports Python via pip, e.g.:

pip3 install sbp

The tool can then be invoked as follows:

python3 -m sbp2json <sbp.bin

The performance of the Python version is significantly slower than Rust and Haskell, but works on all platforms that Python itself supports.

Building / installing

Before you start, run

git pull --tags

in you local libsbp repository to pull the tags. This will ensure the correct version number is generated.

Using Docker

Before you begin, make sure you have Docker installed. Start Docker desktop.

Fetching the prebuilt image from DockerHub

The quickest method to get going is to just pull a prebuilt copy from DockerHub (no guarantees on freshness) by running the following on your command line:

docker run --rm -v $PWD:/mnt/workspace -i -t swiftnav/libsbp-build:2023-12-19 /bin/bash

This will mount your local copy of the libsbp repository onto the image.

Check this link for newer tags. Alternatively, you could run

docker run --rm -v $PWD:/mnt/workspace -i -t swiftnav/libsbp-build:latest-master /bin/bash

if you are facing issues with compilation and the tags are out of date as well.

Creating your own image

Otherwise, the Dockerfile will create a docker image that contains all the necessary dependencies to build libsbp. You can make a local image fresh from this file by running docker build as such:

docker build -t libsbp-build - <Dockerfile

Reading the Dockerfile from STDIN prevents docker from pulling in the whole repostory into the build context (which is then immediately discarded anyway). You can customize the UID of the user that's created with the docker image by passing the desired UID value to the build:

docker build -t libsbp-build --build-arg UID=1234 - <Dockerfile

You can then make this image operate on your local workspace like this:

docker run --rm -v $PWD:/mnt/workspace -i -t libsbp-build:latest /bin/bash

Using the docker image

Once in the image, to overcome issues with git trying to determine ownership of the repository, run git config --global --add safe.directory /mnt/workspace. Then, simply type make all to generate all the libsbp bindings. This could take several hours to run. Alternately, the docker image will run the make all command by default, so you can kick off the make all process by simply running the following command:

docker run --rm -v $PWD:/mnt/workspace -i -t libsbp-build:2023-12-19

To speed up this process you can attempt to run Python environment tests in paralell with:

docker run --rm -v $PWD:/mnt/workspace -i -t -e SBP_TOX_PARALLEL=auto libsbp-build:2023-12-19

When you are finished, quit Docker so that it would not unnecessarily use up resources on your machine.

If you run into issues during the generation process, try running make clean. Alternatively, you could recompile from a clean, newly-cloned libsbp repository on your machine, which would minimize the chance of running into compilation issues from an old build.

Installing from package managers

Some bindings are available on package managers:

• python: available on pip
• haskell: available on Hackage
• javascript: available on NPM

Installing development Python versions

To install the Python binding from source (using pip) run the following command:

pip install 'file:///path/to/libsbp#subdirectory=python'

Or via setuptools directly:

cd /path/to/libsbp
cd python
python setup.py

Adding development version as a pip dependency

Run the following command:

pip install "git+https://github.com/swift-nav/libsbp@<GIT_REVISION>#egg=sbp&subdirectory=python"

Or add this to requirements.txt:

git+https://github.com/swift-nav/libsbp@<GIT_REVISION>#egg=sbp&subdirectory=python

Installing from source

You can build one binding at a time or update all at once:

make python

or

make all

are both valid. To see a list of all valid targets, run make help.

Python version notes:

1. By default the Python targets make python and make test-python (as well as make all) run tests on all Python versions officially supported by the libsbp Python bindings, currently 3.6-3.9, skipping any versions not installed. To run tests on just specific Python version(s), specify the TOXENV environment variable, e.g., TOXENV=py37 make python. Travis runs Python tests on all supported versions.
2. By default the code generators are run on the system's (or virtual env's) default Python interpreter. Currently Python versions 2.7, 3.5, and 3.7 are officially supported, other versions may or may not work. The generated libsbp bindings should be the same on all supported Python versions. To use a different version than your default Python interpreter, specify the GENENV environment variable, e.g., GENENV=py27 make all (you must have that version of Python installed beforehand).
3. To run both the generator and the Python tests on specific Python versions, specify both envs, e.g., GENENV=py37 TOXENV=py37 make python

SBP Development Procedures

See the "how to" page for instructions on adding new messages, updating the documentation, and releasing new versions of this library.

SBP Protocol Specification

SBP consists of two pieces: (i) an over-the-wire message framing format and (ii) structured payload definitions. As of Version 1.0, the packet consists of a 6-byte binary header section, a variable-sized payload field, and a 16-bit CRC value. SBP uses the CCITT CRC16 (XMODEM implementation) for error detection.

Please see the docs for a full description of the packet structure and the message types. Developer documentation for the language-specific sbp libraries is here. Please refer to the changelog for more information about the evolution of the library and its messages.

JSON Schema Definitions

For web clients processing SBP in JSON form, JSON schema definitions are provided. Libraries for JavaScript, TypeScript, and Elm generated by the QuickType tool are provided. See the HOWTO for instructions on updating these schemas.

Kaitai Struct Format Descriptions

Kaitai Struct is a declarative language used to describe binary data structures. KSY files containing format specifications for all supported SBP messages can be generated using the kaitai target. The resulting files can be found in the kaitai/ksy directory. This target also runs the Kaitai Struct compiler to generate bindings for perl and python. These bindings (along with test cases) can be found in the kaitai/perl and kaitai/python subdirectories respectively.

The Kaitai Struct format description files can also potentially be used to generate bindings for the following targets:

• graphviz
• csharp
• rust
• java
• go
• cpp_stl
• php
• lua
• nim
• html
• ruby
• construct
• javascript

Notes on python bindings

The python bindings generated by the Kaitai Struct compiler allow for significantly faster parsing in comparison to the construct-based bindings included in libsbp. However, due to the nature of Kaitai Struct, these bindings do not allow for serialisation (i.e. construction of SBP objects). Serialisation for java and python targets is supported as an experimental feature in the development version of Kaitai Struct (> 0.10) so this restriction should hopefully be removed in the future.

It should also be noted that the names used by the KSY bindings are not 1:1 identical to the construct bindings due constraints which Kaitai Struct places on various token types (e.g. it is forbidden to use upper case characters in field names). The test cases in the kaitai/ksy/tests directory (particularly those in test_parsers.py) can be used as a starting point for working with these bindings, as well as demonstrating the minor differences between the output generated by each set of bindings.

LICENSE

Copyright © 2015-2023 Swift Navigation

Distributed under the MIT open source license.