-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support uint16/uint32/uint64 markers and ND array header (BJData draft 1) #14
Conversation
the support to |
Dear @fangq, thank your for your submitted PR and apologies for the delay in responding - I'm pretty busy at the moment. It's nice to hear that you've taken on evolving what started out as UBJSON and I'm glad that my condensed specification document in this repository was useful. However, based on changes in the new BJData 0.5 Draft 1 specification that you propose, I would recommend having a separate package which handles BJData (e.g. by forking this repo as a starting point). If it also happens to be able to handle (older) UBJSON specifications, then even better. I've listed some reasons below for why I think that is the best course of action. Backwards compatibilityMost people install py-ubjson without specifying a version requirement, i.e. they will download the latest available version. Introducing new markers means that objects encoded with the new proposed version will not be decodable with older/existing/stable versions, nor will they be decodable with other ubjson libraries of the same (Draft 12) level. At this point in time I don't think this would be acceptable given UBJSON and py-ubjson have been around for a while and (most) established libraries provide the same version support. Publishing this as a new independent package on the other hand would signal that BJData is not strictly backwards compatible but a real format in its own right. If you also offered a mode to encode/decode against the (now old) UBJSON spec, that would be very useful and enable people to more easily transition to BJData, if they wanted. Also, you could then choose to only support Python 3, which would make the codebase considerably cleaner. Ongoing BJData evolutionWe currently use ubjson in a production environment where stability & maturity are key. Given BJData is currently versioned at 0.5 Draft 1 and has only appeared last month on GitHub, I assume that further iterations on the spec might happen. (E.g. are your expecting it to be peer-reviewed? Might there be additional changes worth adding?) LicensingAs I'm sure you've seen, py-ubjson has an Apache 2.0 license and thus you are free to take all resources herein and use them as you see fit (e.g. to create a BJData package), preserving the copyright/licenses notices to show historic origins. If you'd still like detailed feedback on the changes in this PR, let me know. I'm very happy to have a look through it. A few high-level suggestions I have from the brief look I've taken:
|
@vtermanis, thanks for the detailed feedback. I will create a py-bjdata package from my fork of your py-ubjson, and maintain it from there (thanks again for making this package in the first place).
there is an interest to incorporate these new features to the next draft of the UBJSON spec, but the discussion is still on-going, see ubjson/universal-binary-json#109 regardless, I will be happy to maintain the bjdata spec as well as the derived python package separately, until there is sufficient interest in the community/users to adopt these new additions in the future.
these are very good suggestions, and I will try to make the package backward compatible by introducing new flags. May come back to you for questions if any specific function that I am not sure. closing this PR for now. |
hi team, as indicated in this UBJSON's issue tracker, I am interested in continuing the stalled UBJSON development and its future maintenance.
I derived a MarkDown version of the Draft 12 of the UBJSON spec from your documentation (thank you for that wonderful summary!), and added a few long-desired new features, i.e.
uint16 (u), uint32 (m), uint64 (M)
andhalf/float16 (h)
data#
followed by a 1D integer array construct)The improved UBJSON format, now named as "Binary JData Specification" (BJData) will be officially maintained at
https://github.com/OpenJData/bjdata
The BJData spec is largely backward compatible (i.e. support all existing UBJSON markers), with the only exception that it no longer maps
NaN/+infinity/-infinity
toZ
.In the past few days, I read through your py-ubjson encoder/decoder, and added the initial support to the new markers and count construct.
I am wondering if you can
the updated C parser already contains a macro
USE__BJDATA
to enable all added features. If desired, a similar option flag can be added to the pure python encoder/decoder to support UBJSON and BJData explicitly.