[feature] Embed YAML data into PNG

[jacobian91]

PNGs have the ability to add arbitrary data to them in 'chunks'. This is inspired by diagram software diagrams.net. The XML that creates the diagram is stored in the zTXT chunk of the PNG. For WireViz, the equivalent would be putting the YAML data.

In addition to storing the data inside, it is most useful if WireViz can then use it as the input file, and ideally output the YAML file as well.

[formatc1702]

Hi,
sorry for ignoring this issue and the associated PR for this long.
This is an interesting idea... can you explain a real-life situation where this might come in handy?

It's nice that the PNG is "self-documenting", but I'm wondering in which situation you would want to use a PNG as an input file directly?

[jacobian91]

This is especially nice for document control systems in companies that produce hardware. Those systems typically require 2 files to be uploaded for something like this, rendered and source. That way the parties who need each can access them in a single controlled location. However, every time there is an update both of the files need to be kept in sync. Doc-control systems are notoriously manual and so files getting out of sync is a matter of human error. Helping prevent this issue reduces time/maintenance costs for companies who use it.

[kvid]

The "YAML data" might be the sum of a potentially large prepended file followed by the main input file. What should then be stored in the PNG file? If storing a copy of the prepended file, then it'll be disconnected from later changes to a prepended file that might be common to many different main YAML files. Is that a wanted feature?

[jacobian91]

@kvid I had originally kept only the complete yaml because of the structure of the code and I couldn't separate the prepend from the primary yaml file. I added some arguments to the main parse function now that give that ability so that the png now contains the yaml data file and the prepend yaml file, and when reading from a png file both will be saved to the directory as _out files.

This way it is easy to ignore the prepend file and still recover the diagram-specific yaml. Thanks for the suggestion!

[kvid]

@jacobian91 I like your solution to my scenario above. Splitting the prepended and main YAML into separate embedded sections will indeed make it easier to restore the original YAML files - even in cases with a common prepended file shared with multiple main files.

However, if you allow an option to conceal_input, then you might consider 4 values of this option: True, False, PrependedOnly, MainOnly - to handle cases where only one of the YAML sections need to be concealed. I admit that the latter of these 4 values might not be used much, but you might consider it for completeness. It might be easier for the user if you find some shorter value strings, or maybe split them into two boolean options conceal_prepended and conceal_main with conceal_input as an alias for both. When concealing only a section of he YAML input, it might be useful to embed a placeholder for the concealed section so the user knows the embedded YAML is not the complete input.

Be aware, though, that multiple (an ordered list of) prepended YAML inputs might be supported in a future version of this software.

[jacobian91]

@kvid nice suggestions, I think those a very valid use-cases.

As far as ordered list of more prepended files, I didn't add anything into this right now that would allow that, but nothing I did explicitly prevents it. I think one way it could be expanded is another argument could be added to list a range or specific prepend files to conceal maybe by index number.

[kvid]

Any connector and cable might contain an image read from an image file. A YAML input that contains such image filenames cannot be processed without access to those image files. Are you able to store a copy of all such input image files as well? If possible, it should perhaps be best to store only those image files actually used in the harness.

If a user wants to conceal a YAML input file, is it then possible to also conceal the image files from that YAML input?

Be aware that the current dev branch now contains code that supports multiple prepend files, and also input HTML template files. As with prepend files, image files and template files might also be shared between several harnesses. Future versions might support even more additional input files.

[jacobian91]

I think those are valid ideas for expansion. I won't be adding those to an already larger scoped PR than originally planned though. As for this feature, the YAML data alone was the goal.