Suggestion: Minimal Program to run Specs

[tisba]

I was thinking about using your specification to improve my at best mediocre golang skills. It would be nice, if there was a test suite, like a bunch of json files with a validation tool. So that you can compare and verify 3rd party implementations.

I haven't thought this completely through, but I could imaging something like this:

validator mybinary

Where validator would be the "official" binary using the library which does through a spec/*.json folder, calls mybinary with the path to the example, expects the result on stdout and then compares the result.

This way it would be very easy – not only for implementors, but also for you to verify 3rd party implementation for correctness and completeness.

[jsteemann]

There is some golang implementation of velocypack already:
https://github.com/arangodb/go-velocypack

I don't want to discourage your development by that, I justed wanted to provide some pointer to it.

Regarding the original question, do you mean we should provide a binary for velocypack validation, which could be invoked like validator <filename> and which should return an exit code with the validation result (status or failure)? If so, that binary could easily be invoked from a shell script and be executed on any file in a sub-directory and be used to validate the generated velocypack data from third-party implementations. If not, could you please elaborate a bit? Thanks!

[tisba]

Nah, okay, I overlooked the golang implementation. Maybe I'll pick another language then, or I'll take a look at doing this with golang anyway, just as an exercise :)

What you describe would probably also do the job. Like I said initially I'm not 100% sure on how this should work exactly. The goal would be to be able to compare generated velocypack data using an official tool that can run over JSON (e.g. from https://github.com/arangodb/velocypack/tree/master/tests/jsonSample) to data generated by a 3rd party tool. Maybe you could even use that as part of CI to test your own, non-C++ implemented libraries.

[jsteemann]

Ok, got it.
We already provide a tool that converts JSON to VPack. It is named json-to-vpack and will be built if the VelocyPack repository is built with the CMake option -DBuildTools=On. The json-to-vpack tool will output the binary vpack result to stdout or into an output file.
Making comparisons with other implementations on a byte-by-byte level is a bit tricky, as there will be different valid vpack representation for the same data. For example, objects can be sorted by attribute name or unsorted, arrays can have index tables or not etc.
But what can simply be done is to run the other existing tool vpack-to-json on the third-party VelocyPack data to see if it can be converted back to JSON.
We can probably add more options to the tools to control their behavior (e.g. to control building of sorted/unsorted objects, arrays with or without index tables etc.). Do you think that would help?