Decoding containers with (potentially)-empty elements

[jsbean]

We have a use case where we need to decode an array of potentially empty elements.

Currently, unkeyed and keyed containers holding onto empty (e.g., a struct with no fields) or a potentially empty (e.g., a struct with all optional fields) elements does not decode.

Say we've got a struct like this:

struct Empty: Equatable, Codable { }

If we have some XML like this:

let xml = """
<container>
    <empty/>
    <empty/>
    <empty/>
</container>
"""

It would be nice to decode like this:

let decoded = try XMLDecoder().decode([Empty].self, from: xml.data(using: .utf8)!)
// Throws "Expected Empty but found null instead."

The poetics are strong.

Wrapping it up a bit, consider this:

struct EmptyArray: Equatable, Codable {
    enum CodingKeys: String, CodingKey { case empties = "empty" }
    let empties: [Empty]
}

Given the xml from above, we can try to decode it like this:

let decoded = try XMLDecoder().decode(EmptyArray.self, from: xml.data(using: .utf8)!)
// Throws "Expected Array<Empty> value but found null instead."

Lastly, given this container:

struct EmptyWrapper {
    let empty: Empty
}

And we have some XML like this:

let xml = """
<wrapper>
    <empty/>
</wrapper>
"""

And we try to decode it like this:

let decoded = try XMLDecoder().decode(EmptyWrapper.self, from: xml.data(using: .utf8)!)
// Throws "Expected Empty value but found null instead"

We get the same outcomes if you substitute the Empty type with something like this:

struct MaybeEmpty {
    var idunno: Int?
}

If this seems like something that should theoretically be supported, I can give a go at it. It smells like it might touch similar neighborhoods of the codebase as #122.

[MaxDesiatov]

Thank you for reporting this. I previously also had an attempt at this, or a similar case related to optionals and empty elements, and I wasn't able to fix and at the same time to keep all existing tests passing. If you see a way to make this work well, please go ahead!

[JUSTINMKAUFMAN]

@MaxDesiatov Any word on this?

I have the same issue decoding the following xml (see the part with <qtcodec/>):

<codec>
    <appspecificdata>
        <appname>Final Cut Pro</appname>
        <appmanufacturer>Apple Inc.</appmanufacturer>
        <data>
            <qtcodec/>
        </data>
    </appspecificdata>
</codec>

If there is no official answer to this as of yet, I'm curious to know what you're doing @jsbean to handle this case in the interim (both for decoding and encoding)?

Thanks, and kudos on a great library!

[jsbean]

Hi @JUSTINMKAUFMAN,

We made a patch for this in this PR on our fork. We haven't submitted it here yet because it changes an expectation in one test (@bwetherfield mentions it in the PR), and because we have been focused on other things in the meantime. Our patch is working great for us, though!

What kinda value is qtcodec supposed to decode into? We have also changed things a little on our fork so that an empty element like <qtcodec/> could represent "".

Perhaps give our fork on master a try. If this works for folks other than us (and @MaxDesiatov accepts a slight change in semantics), we will gladly submit a PR here!

[JUSTINMKAUFMAN]

Thanks @jsbean that works perfectly!

[MaxDesiatov]

Thanks folks! My main reason for being cautious with this is the effort to maintain widest possible compatibility in CoreXLSX. But I need to double check, maybe the fork works fine in most important cases and slight breakage in backward compatibility would be ok. I'll keep the issue open until that's verified, and as usual I'm open to any possible improvements.