Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meta issue: Design and implement (protocol buffer) file format #19

Open
Lathreas opened this issue Apr 29, 2021 · 2 comments
Open

Meta issue: Design and implement (protocol buffer) file format #19

Lathreas opened this issue Apr 29, 2021 · 2 comments
Labels
discussion Discussions about implementation or features to do This should be implemented

Comments

@Lathreas
Copy link
Collaborator

For storing any anatomy files, we need to store the data that represents each species or character. We should decide on how exactly to store this data and implement the save functions.

It might be a good idea to store a list of objects and serialize them easily using the stl System.Text.Json to keep things easy and clear.

@Lathreas Lathreas added to do This should be implemented discussion Discussions about implementation or features labels Apr 29, 2021
@AdamNorberg
Copy link
Collaborator

I strongly oppose JSON for this use because it is an incompletely-defined format (the spec is, approximately, "do what JavasScript does"), it does not allow declarative types providing consistency or tools enforcing what fields are expected, and its numeric type is always implied to be double, which cannot represent every long integer and certainly not every unsigned long integer.

I strongly prefer protocol buffers: https://developers.google.com/protocol-buffers/docs/csharptutorial for the C# version. It's a strictly written standard, it uses schemas with proper validation, it has a very compact binary representation, and it has been tuned to an extraordinary degree for encoding and decoding efficiently. Of course, I'm biased, because I work with them on a daily basis...

JSON has the nasty problem that it's easy to write something that looks like it's working but isn't. Protocol buffers are more opinionated about their contents, while being lax enough to allow fields to be added during development and correctly passed on by systems not familiar with those fields if that system is simply transferring messages, because it is actively designed to be "forwards compatible" and allow unknown fields to be transferred and reserialized.

Protocol buffer definition files run through protoc and generate native C# code. One of the benefits to this is that the C# compiler can (and will) stop you from using the wrong name or type for a field because it is a formally-defined type. The downside is that now the build process goes from "compile" to "generate, compile" but it is the nature of all nontrivial software projects to get an increasingly complicated build pipeline over time...

@Lathreas
Copy link
Collaborator Author

Lathreas commented May 3, 2021

Let me take a look at protocol buffers! They look very interesting so far, and I definitely see their benefits. Especially the fact that it handles floating point numbers much better than JSON is a big plus.

Since it uses a precompiler, I wonder how well it works with Godot, though. Internally Godot uses mono to handle C# compilation, so maybe it is possible to smoothly pass the precompiler (or generator?) to Godot's current build command, but that's something we should investigate.

To be fair, I must admit I am personally not too fond of adding too many precompilation steps if we can avoid it, so let's see what we can do there.

@Lathreas Lathreas changed the title Design and implement (JSON) file format Design and implement (protocol buffer) file format Dec 5, 2021
@Lathreas Lathreas changed the title Design and implement (protocol buffer) file format Meta issue: Design and implement (protocol buffer) file format Dec 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion Discussions about implementation or features to do This should be implemented
Projects
None yet
Development

No branches or pull requests

2 participants