Use of JSONEncoder or JSONDecoder massively increases final binary size

[bitwit]

Example Code:

import Foundation
let decoder = JSONDecoder()

Result
Without optimization, browser reports wasm file size of ~27MB, and ~11MB without any code

Proposed solution:
Update wasm book: https://book.swiftwasm.org/getting-started/porting.html

The Swift Foundation and Dispatch should probably include JSONEncoder/Decoder with some sort of flag that it's not worth using even if it compiles.

If this is a valuable improvement, I'd be happy to make that PR myself. I think it could help prevent other developers from going down the same rabbit hole

[yonihemi]

not worth using even if it compiles is highly subjective, and is not the correct tradeoff for every project. For example, for several years before ABI stability was introduced on Apple platforms merely using any Swift would increase the binary size by more than 10mb, and most seemed fine with the tradeoff.

You're welcome to extract JSONDecoder to a separate module, build one yourself in Swift, or use JavaScript interop to rely on the browser.

Optimization is critical for WebAssembly. Using carton bundle will produce the optimal result.

[bitwit]

Thanks. I think this seems to have arraybuffer implications and it was the difference between my code working or not so I think documentation would have value.

I agree my comment is subjective but I also think it's worth noting the impact somewhere. Honestly I'm happy if someone Googles arraybuffer swiftwasm and finds this issue in case it is helpful for others.

I wish I had know to look out for JSON Codable in my module and now I know the impact.

[bitwit]

I'd also like to note that even with the optimized release version my code still didn't work so binary size alone isn't the problem here alone.

[yonihemi]

JSONDecoder does work in a browser environment. If you find that it doesn't, please create a minimum reproducible project.

[bitwit]

It's not that it doesn't work, it's that the browser is highly constrained and including JSONDecoder will consume a huge portion of what fits, it seems. You mention that historically swift devs didn't care about binary size but that was for native mobile apps and it didn't affect the max number of other structures you could import. This is more impactful.

It could be that I'm not explaining this well but my purpose is not to claim there's a bug but just to have a discussion about what devs should know coming into this project.

There's a section in the book about unsupported types but maybe a section about types to look out for would be valuable too.

In any case, I think my best course of action is just to make a PR against the documentation and it can be discussed from there what developers might find useful when first getting their feet wet.

[MaxDesiatov]

Do you see binary size increase with any Foundation type or just these two?

[bitwit]

Hi @MaxDesiatov , I realized all my code is open source so I've made an example that clearly demonstrates the issue:

https://github.com/bitwit/swift-wasm-example

If you swap between the 2 branches in the 'RedECS' package you'll see a massive change in binary size and an array buffer error in the browser when you run the branch that includes the lines with the JSONCoding work.

I'm sorry also if the tone of this issue initially comes off as a trashing of the project, as my intention is the opposite. I'd love to help contribute and help new developers get through the gotchas.

I don't actually believe this is a bug in SwiftWasm either, but just a limitation of the browsers right now, so it is important for Swift devs to understand those differences coming in.

Lastly just want to say this project BLEW my expectations away and I'm elated with the results so far. I can't believe my random dungeon generator code is working in a browser and all I had to do was hook in to canvas. AMAZING 🎉 https://twitter.com/kylnew/status/1486556895653646343

[MaxDesiatov]

Great, thanks for your feedback!

If you use any Foundation other than JSONDecoder or JSONDecoder in your project, say Date, does that still have any impact on binary size? What I'm trying to understand is whether the increase is caused by linking Foundation in general, or by using any JSON-related types specifically.

[bitwit]

Its after I use the JSONDecoder and JSONEcoder, from what I saw. Import alone didn't seem to trigger it.

main diff here: RedECSEngine/RedECS@13e1a2d

[MaxDesiatov]

Import alone didn't seem to trigger it.

Yes, import alone does not trigger linking, that's why I'm asking about using specific types. Is the binary size increased when you start using Date or Data, or any other Foundation type?

[bitwit]

I will try to do some more investigations later too and see if there are other structures that do this, such as Date. when I tried Date it wasn't working at all.
Let me get back to you later 👍

[MaxDesiatov]

I'll reopen the issue in the meantime until we reach a consensus how our documentation should be updated to clarify the binary size concerns.

[bitwit]

Okay so this was an interesting investigation:

1. You are correct that using Data and Date both also had this impact on the binary size. It's not JSONDecoder/Encoder specifically, that just happens to be what I was using. I think I also had a misunderstanding that Codable protocol is part of Foundation (and when you go to the Apple docs they don't delineate Swift Standard Library from Foundation very well either). So it appears the implications of this are that my modules I've been importing may not be using Foundation at all.

2. With this understand I was able reduce my minimum example to simply importing ONLY my Geometry module, plus using Data() and getting TypeError: Underlying ArrayBuffer has been detached from the view

3. I tried to copy paste all my Geometry structures into main.swift directly because I wanted to see if I could find out exactly what the breaking point was, but I couldn't!! Even when the binary size exceeded that of the one where I used the module, it wasn't causing the Array buffer error. Why is this? Could this be not a size issue alone?

So from this I have a few more thoughts (under an assumption that my questions in point no. 3 doesn't reframe the conversation):

• For my use case Foundation seems not worth the overhead. It was the difference between none of my modules working and being able to load my Geometry, Graphs, Dungeon Generator and Game Engine into the browser. Once I realized this I was over the moon with excitement. This has me really motivated to continue playing with SwiftWasm for game development purposes.
• As a good Swift package citizen, I like my packages to be as cross platform friendly as possible, so knowing that Foundation is a lot for the browser helps inform my own package design. In the case of my game engine I think putting JSON work into a separate module makes total sense and now I know that separating out Foundation-dependant functions into a separate module could be beneficially
• It looks like JavascriptKit helps bridge some Foundation gaps and we shouldn't forget that we are inside of a browser where there is other context to help us bridge the Foundation gaps. If all you need to do is Decode some JSON, then JavascriptKit does that without the whole Foundation overhead, for example.

So when it comes to documentation I was wondering if maybe there is a way to target both app developers and package maintainers who want their code to be SwiftWasm friendly. Understanding the Foundation overhead a little bit better might helps folks weigh the pros and cons of if they need to import Foundation, and organize their packages accordingly. Getting into the mindset of Wasm binary constraints like this is a new concept to me compared to mobile and linux. So worth the costs though, I still can't believe I see my modules working in the browser!

[ephemer]

@bitwit note that we ran into a related issue, whereby Foundation was implicitly being imported: https://forums.swift.org/t/how-to-disable-implicit-foundation-imports/59678

Due to that bug, using any API from Foundation, even if you don't import Foundation will mean Foundation is added to the bundle. I worked around this by making a local copy of JavaScriptKit via swift package edit JavaScriptKit, navigating to Packages/JavaScriptKit/Package.swift and commenting out the resources: line from the JavaScriptKit target. That allows you to see which code is using Foundation without importing it – you can get that build working and then run swift package unedit JavaScriptKit again.

Regarding JSON, You might have more luck using XJSONDecoder from this package. I haven't tried it myself, but doing things that way should mean you only get JSON Decoding and not the entirety of Foundation.

FWIW, we removed Foundation from our stack and reduced our binary size on Wasm and Android by 70-90%

[bitwit]

@ephemer thanks for the reference. I love to look out for non-foundation-dependent swift packages

I managed to remove foundation and it was a huge difference in what was possible in the browser. I didn't personally encounter issues with JavaScript kit (though I'm not up to date on latest versions)