Add a callback to select which element are written

[robUx4]

And remove the ForceNoDefault() which is not needed anymore.

There's a EBML_WRITE_FILTER define to tell between libmatroska versions with this feature or not.

[mbunkus]

Interesting, thanks. I haven't tried converted MKVToolNix yet & cannot say how much work supporting both types of interfaces would be & whether I like it or not. Fact is that this type of interface change does require a substantial amount of changes, whereas the simple flag/toggle I suggested to add in the elements wouldn't. On the other hand callback-type systems are more flexible. But do we really need that flexibility?

Have to think about it some more.

[mbunkus]

I don't understand the intent behind this change. What does keeping the position have to do with the question whether or not to write the element?

[robUx4]

This is just the translation of the previous call into the new system. bKeepPosition would be used as the bWithDefault value. So if it's true it writes elements even if they are the default value (WriteAll), if it's false it only writes the elements if they are not the default value (SkipDefault).

The original logic may be flawed and it should be fixed separately. In fact when overwriting you'd probably want to tell which elements you want to write (the least possible).

This whole API is a recipe for broken files (which originally was a welcome possibility in libebml to be able to generate broken files to test against). It seems mkvtoolnix doesn't use it, there's only a test calling OverwriteHead(). So I'd rather remove the API altogether.

[mbunkus]

Ah, I see. Thanks for the explanation. Yeah, MKVToolNix (or rather: mkvpropedit) only uses OverwriteHead for cases where it has to cover a one-byte space that's suddenly left by other operations; in such a case it forces the element size field to be one byte longer than necessary.

It never uses OverwriteData, though. If it has to overwrite a whole element, I think it simply uses Render & sets the file position manually.

[mbunkus]

I… don't like this #define. Consumers can already use the version number #defines in order to differentiate between v1 & v2 of the library.

Not sure how I'll handle it in MKVToolNix. Will have to look into how many places will have to be changed as I'm generally not a fan of a lot of conditional compilation anyway. Maybe I'll just throw the switch & require v2 as soon as I add support for it.

[robUx4]

The define is there to tell v2 users which API they can expect. Once v2 is stable we can remove it and everyone should assume it's there for v2 code. In other words it's to make your life easier if you want to support the old and new API in your codebase for a while. I don't mind removing it and you handle it on your side.

[mbunkus]

Makes sense, but isn't needed, at least not for me. As there's no official release of v2 yet, I won't support different snapshots of the upcoming v2 API in MKVToolNix. I'll try my best to support v1 & the latest v2 simultaneously, but even that is in question with this RFC.

[robUx4]

whereas the simple flag/toggle I suggested to add in the elements wouldn't. On the other hand callback-type systems are more flexible. But do we really need that flexibility?

There's either the ForceNoDefault() we currently have which tells a specific instance it's not it's default value (even when it does, completely changing the meaning of the IsDefault test). Even if you change the value after. Or we keep the IsDefault meaning intact and change the logic to tell if an element has to be written. Maybe we can still add a flag in each instance that would replace the ForceNoDefault(). But IMO it's intrusive and wastes space in each and every instance. While passing the "algorithm" to decide how to store it when we do seems less intrusive.

Being less intrusive you don't have to have a kax_block_add_id_c class in mkvtoolnix anymore.

[robUx4]

Also this kind of "agnostic" filter means we can also select which element to write based on the Matroska version targeted. When writing a v2 file, you don't want to end up writing v4 or v5 elements. Once we have the supported version inside each element class, we can easily do that.

[mbunkus]

There's either the ForceNoDefault() we currently have which tells a specific instance it's not it's default value (even when it does, completely changing the meaning of the IsDefault test). Even if you change the value after.

Hmmm I somehow wasn't aware of ForceNoDefault(). I'll try to look into it some more before Christmas, including how much of a change MKVToolNix would need to support this type of callback API.

[mbunkus]

Also this kind of "agnostic" filter means we can also select which element to write based on the Matroska version targeted. When writing a v2 file, you don't want to end up writing v4 or v5 elements. Once we have the supported version inside each element class, we can easily do that.

Not sure if that would actually help me. MKVToolNix does it the other way around: it first writes v2 as the DocTypeVersion & updates that field depending on which features are activated & used further down the line (e.g. after writing the track headers it potentially updates the EBMLHead with the actual profile values used if they differ from what's been written initially).

If the library routines need to know what to use before the whole file is written, a lot would have to changed in MKVToolNix. Or I'd simply have to use a filter callback that ignores the profiles altogether so that the existing code continues to work. For me this type of feature be an advantage.

[robUx4]

Not sure if that would actually help me. MKVToolNix does it the other way around: it first writes v2 as the DocTypeVersion & updates that field depending on which features are activated & used further down the line (e.g. after writing the track headers it potentially updates the EBMLHead with the actual profile values used if they differ from what's been written initially).

If the library routines need to know what to use before the whole file is written, a lot would have to changed in MKVToolNix. Or I'd simply have to use a filter callback that ignores the profiles altogether so that the existing code continues to work. For me this type of feature be an advantage.

I wasn't sure if I should add an opaque pointer (or if there's a cleaner way in C++ for such callbacks) but it might be useful in your case. During the write phase each element written goes through your filter which could collect the highest profile used.

Not that you'd have to implement it this way, but it might be useful as a generic solution.

[mbunkus]

I've finished the required modifications in MKVToolNix. All in all I'm fine with this change if the callback type is changed to the aforementioned std::function<void(const EbmlElement &)> instead. That allows me to use std::bind() in order to have a member function be a callback, simplifying what I have to do. It's just way more flexible than a plain old C-style function pointer.

On top of that the only remaining change I'd like to ask for is the other aforementioned one, getting rid of that one unused variable warning.

Thanks.

[mbunkus]

I've pushed branch callback_select_write_enhanced to this repo: it's your PR rebased onto current master with the two additional changes I asked for. This is what I've tested the support in MKVToolNix against.