Skip to content
This repository was archived by the owner on Aug 20, 2023. It is now read-only.

alshapton/TinyMP

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ed83b65 · Dec 23, 2017

History

94 Commits
Nov 13, 2017
Dec 23, 2017
Dec 23, 2017
Nov 13, 2017
Nov 17, 2017
Dec 23, 2017
Nov 24, 2017
Nov 26, 2017
Nov 13, 2017
Nov 19, 2017
Nov 19, 2017
Nov 13, 2017
Nov 19, 2017
Dec 23, 2017
Dec 23, 2017
Nov 14, 2017
Nov 18, 2017
Dec 23, 2017
Nov 18, 2017
Dec 23, 2017
Dec 23, 2017

Repository files navigation

artwork/tinymplogo.png

https://travis-ci.org/alshapton/TinyMP.svg?branch=master

TinyMP is a storage backend for TinyDB https://github.com/msiemens/tinydb which is based around the MessagePack compressed JSON format (https://msgpack.org/index.html)

Syntax :

TinyMP extends the syntax of the tinydb class using one of the optional kwargs as follows:

Values for **kwargs
Value Effect
storage=MsgPackStorage Default option, will use the MsgPack library
storage=MsgPackStorage,Lib='msgpack' Will use the MsgPack library
storage=MsgPackStorage,Lib='umsgpack' Will use the U-MsgPack Library

Example Usage:

from tinydb import TinyDB, Query
from tinymp import *

db = TinyDB('data.msg',storage=MsgPackStorage)

def dbins():
   db.insert({'type': 'apple', 'count': 7})

dbins()

As you can see, it's a simple drop-in replacement for any storage engine and it can be nested and cached. Don't forget, you will need to install as a minimum, the msgpack-python library using pip install msgpack-python and the U-MsgPack library from https://github.com/vsergeev/u-msgpack-python in order to use that option.

Example Usage using alternative MessagePack Library:

from tinydb import TinyDB, Query
from tinymp import *

db = TinyDB('data.msg',storage=MsgPackStorage,Lib='umsgpack')

def dbins():
   db.insert({'type': 'apple', 'count': 7})

dbins()

As you can see, it's a simple drop-in replacement for any storage engine and it can be nested and cached.

Why would I use this?

Looking at the statistics below, it's apparent that compared to the "standard" JSON Storage mechanism, MessagePack isn't as quick, however, the filesizes on disc are smaller - consider the table below, with 1,000 JSON documents of minute size - clearly, the MessagePack compressed format is smaller than the JSON format. Whether you choose the default MsgPack library, which is marginally slower than the U-MsgPack library (at the cost of a small increase in storage footprint with U-MsgPack) is dependent on your use case.

Timings (seconds)
Format Run 1 Run 2 Run 3 FileSize
JSON Write: 2.147 2.011 2.040 37.0 Kb
MsgPack Write: 9.562 9.732 9.716 21.1 Kb
U-MsgPack Write: 9.354 9.066 8.949 24.1 Kb

Changes

  • Version 1.0.0-Beta5 - XX/XX/XXXX
    • Added PiPy compatibility
    • PEP-8 Compliant code style
  • Version 1.0.0-Beta4 - 11/26/2017
    • Added descriptions of benchmarking and tidied up repo
    • Further added information to README about usage
  • Version 1.0.0-Beta3 - 11/24/2017
    • Added support for u-msgpack-python alternative msgpack library
  • Version 1.0.0-Beta2 - 11/21/2017
    • Fixed Testing issues
  • Version 1.0.0-Beta - 11/19/2017
    • Tidied up repo ready for Beta release
  • Version 0.2.0 - 11/18/2017
    • Tidied up repo
    • Added initial benchmarking and example
    • Added Github Community files.
  • Version 0.1 - 11/13/2017
    • Initial Release.

References: