diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index eff804fb..8f8418ef 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,4 +1,4 @@ -name: Continuous Integration +name: Build on: push: pull_request: diff --git a/README.md b/README.md index 4725dbff..3addcd18 100644 --- a/README.md +++ b/README.md @@ -1,49 +1,46 @@ # RMP - Rust MessagePack -RMP is a pure Rust [MessagePack](http://msgpack.org) implementation. +RMP is a complete pure-Rust [MessagePack](http://msgpack.org) implementation. MessagePack a compact self-describing binary serialization format. -[![Build Status](https://travis-ci.org/3Hren/msgpack-rust.svg?branch=master)](https://travis-ci.org/3Hren/msgpack-rust) -[![Coverage Status][coveralls-img]][coveralls-url] +This project consists of three crates: -This repository consists of three separate crates: the RMP core and two implementations to ease serializing and -deserializing Rust structs. - - crates.rs | API Documentation | --------------------------------------------|---------------------------------| - [![rmp][crates-rmp-img]][crates-rmp-url] | [RMP][rmp-docs-url] | - [![rmps][crates-rmps-img]][crates-rmps-url] | [RMP Serde][rmps-docs-url] | - [![rmpv][crates-rmpv-img]][crates-rmpv-url] | [RMP Value][rmpv-docs-url] | +* [RMP-Serde][crates-rmps-url] ([Documentation][rmps-docs-url]) — easy serializing/deserializing via [Serde](https://serde.rs). +* [RMP-Value][crates-rmpv-url] ([Documentation][rmpv-docs-url]) — a universal `Value` enum that can hold any MessagePack type. Allows deserializing arbitrary messages without a known schema. +* [RMP][crates-rmp-url] ([Documentation][rmp-docs-url]) — low-level functions for reading/writing encoded data. ## Features -- **Convenient API** +- **Convenient and powerful APIs** - RMP is designed to be lightweight and straightforward. There are low-level API, which gives you - full control on data encoding/decoding process and makes no heap allocations. On the other hand - there are high-level API, which provides you convenient interface using Rust standard library and - compiler reflection, allowing to encode/decode structures using `derive` attribute. + RMP is designed to be lightweight and straightforward. There is a high-level API with support for Serde, + which provides you convenient interface for encode/decode Rust's data structures using `derive` attribute. + There are also low-level APIs, which give you full control over data encoding/decoding process, + with no-std support and without heap allocations. - **Zero-copy value decoding** - RMP allows to decode bytes from a buffer in a zero-copy manner easily and blazingly fast, while Rust - static checks guarantees that the data will be valid as long as the buffer lives. - -- **Clear error handling** - - RMP's error system guarantees that you never receive an error enum with unreachable variant. + RMP allows to decode bytes from a buffer in a zero-copy manner. Parsing is implemented in safe Rust. -- **Robust and tested** +- **Robust, stable and tested** This project is developed using TDD and CI, so any found bugs will be fixed without breaking existing functionality. +## Why MessagePack? + +Smaller and simpler to parse than JSON. Supports the same types as JSON, plus binary data, all float values, and 64-bit numbers. +Encoded data is self-describing and extensible, without requiring a schema definition. + ## Requirements -- Rust 1.53.0 or later +- Rust 1.56.0 or later [rustc-serialize]: https://github.com/rust-lang-nursery/rustc-serialize [serde]: https://github.com/serde-rs/serde +[ci-img]: https://github.com/3Hren/msgpack-rust/actions/workflows/ci.yml/badge.svg +[ci-url]: https://github.com/3Hren/msgpack-rust/actions/workflows/ci.yml + [coveralls-img]: https://coveralls.io/repos/3Hren/msgpack-rust/badge.svg?branch=master&service=github [coveralls-url]: https://coveralls.io/github/3Hren/msgpack-rust?branch=master @@ -51,11 +48,9 @@ deserializing Rust structs. [rmps-docs-url]: https://docs.rs/rmp-serde [rmpv-docs-url]: https://docs.rs/rmpv -[crates-rmp-img]: https://img.shields.io/crates/v/rmp.svg [crates-rmp-url]: https://lib.rs/crates/rmp - -[crates-rmps-img]: https://img.shields.io/crates/v/rmp-serde.svg [crates-rmps-url]: https://lib.rs/crates/rmp-serde - -[crates-rmpv-img]: https://img.shields.io/crates/v/rmpv.svg [crates-rmpv-url]: https://lib.rs/crates/rmpv + + +[![Build][ci-img]][ci-url] [![Coverage Status][coveralls-img]][coveralls-url] diff --git a/rmp-serde/README.md b/rmp-serde/README.md deleted file mode 120000 index 32d46ee8..00000000 --- a/rmp-serde/README.md +++ /dev/null @@ -1 +0,0 @@ -../README.md \ No newline at end of file diff --git a/rmp-serde/README.md b/rmp-serde/README.md new file mode 100644 index 00000000..d7724ba8 --- /dev/null +++ b/rmp-serde/README.md @@ -0,0 +1,60 @@ +# MessagePack + Serde + +This crate connects Rust MessagePack library with [`serde`][serde] providing an ability to +easily serialize and deserialize both Rust built-in types, the standard library and custom data +structures. + +## Motivating example + +```rust +let buf = rmp_serde::to_vec(&(42, "the Answer")).unwrap(); + +assert_eq!( + vec![0x92, 0x2a, 0xaa, 0x74, 0x68, 0x65, 0x20, 0x41, 0x6e, 0x73, 0x77, 0x65, 0x72], + buf +); + +assert_eq!((42, "the Answer"), rmp_serde::from_slice(&buf).unwrap()); +``` + +## Type-based Serialization and Deserialization + +Serde provides a mechanism for low boilerplate serialization & deserialization of values to and +from MessagePack via the serialization API. + +To be able to serialize a piece of data, it must implement the `serde::Serialize` trait. To be +able to deserialize a piece of data, it must implement the `serde::Deserialize` trait. Serde +provides an annotation to automatically generate the code for these +traits: `#[derive(Serialize, Deserialize)]`. + +## Examples + +```rust +use std::collections::HashMap; +use serde::{Deserialize, Serialize}; +use rmp_serde::{Deserializer, Serializer}; + +#[derive(Debug, PartialEq, Deserialize, Serialize)] +struct Human { + age: u32, + name: String, +} + +fn main() { + let mut buf = Vec::new(); + let val = Human { + age: 42, + name: "John".into(), + }; + + val.serialize(&mut Serializer::new(&mut buf)).unwrap(); +} +``` + +## Efficient storage of `&[u8]` types + +MessagePack can efficiently store binary data. However, Serde's standard derived implementations *do not* use binary representations by default. Serde prefers to represent types like `&[u8; N]` or `Vec` as arrays of objects of arbitrary/unknown type, and not as slices of bytes. This creates about a 50% overhead in storage size. + +Wrap your data in [`serde_bytes`](https://lib.rs/crates/serde_bytes) to store blobs quickly and efficiently. Alternatively, [configure an override in `rmp_serde` to force use of byte slices](https://docs.rs/rmp-serde/latest/rmp_serde/encode/struct.Serializer.html#method.with_bytes). + +[serde]: https://serde.rs/ diff --git a/rmp-serde/src/encode.rs b/rmp-serde/src/encode.rs index 64b26d34..f42b20b7 100644 --- a/rmp-serde/src/encode.rs +++ b/rmp-serde/src/encode.rs @@ -268,6 +268,14 @@ impl Serializer { /// This reduces overhead of binary data, but it may break /// decodnig of some Serde types that happen to contain `[u8]`s, /// but don't implement Serde's `visit_bytes`. + /// + /// ```rust + /// use serde::ser::Serialize; + /// let mut msgpack_data = Vec::new(); + /// let mut serializer = rmp_serde::Serializer::new(&mut msgpack_data) + /// .with_bytes(rmp_serde::config::BytesMode::ForceAll); + /// vec![255u8; 100].serialize(&mut serializer).unwrap(); + /// ``` #[inline] pub fn with_bytes(mut self, mode: BytesMode) -> Serializer { self.config.bytes = mode; diff --git a/rmp-serde/src/lib.rs b/rmp-serde/src/lib.rs index b561a411..406a257a 100644 --- a/rmp-serde/src/lib.rs +++ b/rmp-serde/src/lib.rs @@ -1,55 +1,4 @@ -//! This crate connects Rust MessagePack library with [`serde`][serde] providing an ability to -//! easily serialize and deserialize both Rust built-in types, the standard library and custom data -//! structures. -//! -//! ## Motivating example -//! -//! ``` -//! let buf = rmp_serde::to_vec(&(42, "the Answer")).unwrap(); -//! -//! assert_eq!( -//! vec![0x92, 0x2a, 0xaa, 0x74, 0x68, 0x65, 0x20, 0x41, 0x6e, 0x73, 0x77, 0x65, 0x72], -//! buf -//! ); -//! -//! assert_eq!((42, "the Answer"), rmp_serde::from_slice(&buf).unwrap()); -//! ``` -//! -//! # Type-based Serialization and Deserialization -//! -//! Serde provides a mechanism for low boilerplate serialization & deserialization of values to and -//! from MessagePack via the serialization API. -//! -//! To be able to serialize a piece of data, it must implement the `serde::Serialize` trait. To be -//! able to deserialize a piece of data, it must implement the `serde::Deserialize` trait. Serde -//! provides an annotation to automatically generate the code for these -//! traits: `#[derive(Serialize, Deserialize)]`. -//! -//! # Examples -//! -//! ``` -//! use std::collections::HashMap; -//! use serde::{Deserialize, Serialize}; -//! use rmp_serde::{Deserializer, Serializer}; -//! -//! #[derive(Debug, PartialEq, Deserialize, Serialize)] -//! struct Human { -//! age: u32, -//! name: String, -//! } -//! -//! fn main() { -//! let mut buf = Vec::new(); -//! let val = Human { -//! age: 42, -//! name: "John".into(), -//! }; -//! -//! val.serialize(&mut Serializer::new(&mut buf)).unwrap(); -//! } -//! ``` -//! -//! [serde]: https://serde.rs/ +#![doc = include_str!("../README.md")] #![forbid(unsafe_code)] #![warn(missing_debug_implementations, missing_docs)] diff --git a/rmp/README.md b/rmp/README.md index 87da98cb..3638553f 100644 --- a/rmp/README.md +++ b/rmp/README.md @@ -4,8 +4,10 @@ RMP is a pure Rust [MessagePack](http://msgpack.org) implementation of an effici serialization format. This crate provides low-level core functionality, writers and readers for primitive values with direct mapping between binary MessagePack format. -**Warning** this library is still in rapid development and everything may change until 1.0 -comes. +[Looking for Serde support](https://lib.rs/crates/rmp-serde)? + +This crate represents the very basic functionality needed to work with MessagePack format. +Ideologically it is developed as a basis for building high-level abstractions. ### Usage @@ -13,22 +15,20 @@ To use `rmp`, first add this to your `Cargo.toml`: ```toml [dependencies.rmp] -rmp = "^0.8" +rmp = "0.8" ``` ### Features -- **Convenient API** +- **Low-level API** - RMP is designed to be lightweight and straightforward. There are low-level API, which gives you - full control on data encoding/decoding process and makes no heap allocations. On the other hand - there are high-level API, which provides you convenient interface using Rust standard library and - compiler reflection, allowing to encode/decode structures using `derive` attribute. + RMP is designed to be lightweight and straightforward. There are low-level APIs, which give you + full control over the encoding/decoding process. `no-std` environments are supported. - **Zero-copy value decoding** - RMP allows to decode bytes from a buffer in a zero-copy manner easily and blazingly fast, while Rust - static checks guarantees that the data will be valid until buffer lives. + RMP allows to decode bytes from a buffer in a zero-copy manner, without any heap allocations. + easily and blazingly fast. Rust static checks guarantee that the data will be valid until buffer lives. - **Clear error handling** @@ -41,9 +41,6 @@ rmp = "^0.8" ### Detailed -This crate represents the very basic functionality needed to work with MessagePack format. -Ideologically it is developed as a basis for building high-level abstractions. - Currently there are two large modules: encode and decode. More detail you can find in the corresponding sections. @@ -140,6 +137,6 @@ assert_eq!([0xcb, 0x40, 0x9, 0x21, 0xfb, 0x54, 0x44, 0x2d, 0x18], buf[..]); assert_eq!(pi, rmp::decode::read_f64(&mut &buf[..]).unwrap()); ``` -[read_int]: decode/fn.read_int.html - License: MIT + +[read_int]: https://docs.rs/rmp/latest/rmp/decode/fn.read_int.html diff --git a/rmp/src/lib.rs b/rmp/src/lib.rs index f9d69403..91bb862a 100644 --- a/rmp/src/lib.rs +++ b/rmp/src/lib.rs @@ -1,146 +1,4 @@ -//! # The Rust MessagePack Library -//! -//! RMP is a pure Rust [MessagePack](http://msgpack.org) implementation of an efficient binary -//! serialization format. This crate provides low-level core functionality, writers and readers for -//! primitive values with direct mapping between binary MessagePack format. -//! -//! **Warning** this library is still in rapid development and everything may change until 1.0 -//! comes. -//! -//! ## Usage -//! -//! To use `rmp`, first add this to your `Cargo.toml`: -//! -//! ```toml -//! [dependencies.rmp] -//! rmp = "^0.8" -//! ``` -//! -//! ## Features -//! -//! - **Convenient API** -//! -//! RMP is designed to be lightweight and straightforward. There are low-level API, which gives you -//! full control on data encoding/decoding process and makes no heap allocations. On the other hand -//! there are high-level API, which provides you convenient interface using Rust standard library and -//! compiler reflection, allowing to encode/decode structures using `derive` attribute. -//! -//! - **Zero-copy value decoding** -//! -//! RMP allows to decode bytes from a buffer in a zero-copy manner easily and blazingly fast, while Rust -//! static checks guarantees that the data will be valid until buffer lives. -//! -//! - **Clear error handling** -//! -//! RMP's error system guarantees that you never receive an error enum with unreachable variant. -//! -//! - **Robust and tested** -//! -//! This project is developed using TDD and CI, so any found bugs will be fixed without breaking -//! existing functionality. -//! -//! ## Detailed -//! -//! This crate represents the very basic functionality needed to work with MessagePack format. -//! Ideologically it is developed as a basis for building high-level abstractions. -//! -//! Currently there are two large modules: encode and decode. More detail you can find in the -//! corresponding sections. -//! -//! Formally every MessagePack message consists of some marker encapsulating a data type and the -//! data itself. Sometimes there are no separate data chunk, for example for booleans. In these -//! cases a marker contains the value. For example, the `true` value is encoded as `0xc3`. -//! -//! ``` -//! let mut buf = Vec::new(); -//! rmp::encode::write_bool(&mut buf, true).unwrap(); -//! -//! assert_eq!([0xc3], buf[..]); -//! ``` -//! -//! Sometimes a single value can be encoded in multiple ways. For example a value of `42` can be -//! represented as: `[0x2a], [0xcc, 0x2a], [0xcd, 0x00, 0x2a]` and so on, and all of them are -//! considered as valid representations. To allow fine-grained control over encoding such values -//! the library provides direct mapping functions. -//! -//! ``` -//! let mut bufs = vec![vec![]; 5]; -//! -//! rmp::encode::write_pfix(&mut bufs[0], 42).unwrap(); -//! rmp::encode::write_u8(&mut bufs[1], 42).unwrap(); -//! rmp::encode::write_u16(&mut bufs[2], 42).unwrap(); -//! rmp::encode::write_u32(&mut bufs[3], 42).unwrap(); -//! rmp::encode::write_u64(&mut bufs[4], 42).unwrap(); -//! -//! assert_eq!([0x2a], bufs[0][..]); -//! assert_eq!([0xcc, 0x2a], bufs[1][..]); -//! assert_eq!([0xcd, 0x00, 0x2a], bufs[2][..]); -//! assert_eq!([0xce, 0x00, 0x00, 0x00, 0x2a], bufs[3][..]); -//! assert_eq!([0xcf, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x2a], bufs[4][..]); -//! ``` -//! -//! But they aren't planned to be widely used. Instead we often need to encode bytes compactly to -//! save space. In these cases RMP provides functions that guarantee that for encoding the most -//! compact representation will be chosen. -//! -//! ``` -//! let mut buf = Vec::new(); -//! -//! rmp::encode::write_sint(&mut buf, 300).unwrap(); -//! -//! assert_eq!([0xcd, 0x1, 0x2c], buf[..]); -//! ``` -//! -//! On the other hand for deserialization it is not matter in which representation the value is -//! encoded - RMP deals with all of them. -//! -//! Sometimes you know the exact type representation and want to enforce the deserialization process -//! to make it strongly type safe. -//! -//! ``` -//! let buf = [0xcd, 0x1, 0x2c]; -//! -//! assert_eq!(300, rmp::decode::read_u16(&mut &buf[..]).unwrap()); -//! ``` -//! -//! However if you try to decode such bytearray as other integer type, for example `u32`, there will -//! be type mismatch error. -//! -//! ``` -//! let buf = [0xcd, 0x1, 0x2c]; -//! rmp::decode::read_u32(&mut &buf[..]).err().unwrap(); -//! ``` -//! -//! But sometimes all you want is just to encode an integer that *must* fit in the specified type -//! no matter how it was encoded. RMP provides [`such`][read_int] function to ease integration with -//! other MessagePack libraries. -//! -//! ``` -//! let buf = [0xcd, 0x1, 0x2c]; -//! -//! assert_eq!(300i16, rmp::decode::read_int(&mut &buf[..]).unwrap()); -//! assert_eq!(300i32, rmp::decode::read_int(&mut &buf[..]).unwrap()); -//! assert_eq!(300i64, rmp::decode::read_int(&mut &buf[..]).unwrap()); -//! assert_eq!(300u16, rmp::decode::read_int(&mut &buf[..]).unwrap()); -//! assert_eq!(300u32, rmp::decode::read_int(&mut &buf[..]).unwrap()); -//! assert_eq!(300u64, rmp::decode::read_int(&mut &buf[..]).unwrap()); -//! ``` -//! -//! ## API -//! -//! Almost all API are represented as pure functions, which accepts a generic `Write` or `Read` and -//! the value to be encoded/decoded. For example let's do a round trip for π number. -//! -//! ``` -//! let pi = std::f64::consts::PI; -//! let mut buf = Vec::new(); -//! rmp::encode::write_f64(&mut buf, pi).unwrap(); -//! -//! assert_eq!([0xcb, 0x40, 0x9, 0x21, 0xfb, 0x54, 0x44, 0x2d, 0x18], buf[..]); -//! assert_eq!(pi, rmp::decode::read_f64(&mut &buf[..]).unwrap()); -//! ``` -//! -//! [read_int]: decode/fn.read_int.html +#![doc = include_str!("../README.md")] #![cfg_attr(not(feature = "std"), no_std)] extern crate alloc;