Update references to emscripten/emsdk

[rgaudin]

Within kiwix/overview#82, we looked at all repos for dependency on Docker Images that may migrate out of Docker Hub.

You use emscripten/emsdk:2.0.25 in compilation scripts. emscripten is a Community Organization.

While this specific image shall remain available (unless emscripten decides to), it is possible that emscripten decides to push future images to a different registry (alternatives being to pay a subscription to Hub or get accepted in their OSS Program).

You should probably find out what will happen and adjust your scripts accordingly.

[Jaifroid]

Thank you for checking this. I'll find out what Emscripten's plans are. This will probably affect WASM building in openzim/javascript-lbzim too.

[kelson42]

@Jaifroid what is the purpose od this container image?! I though that emscripyen happens now in kiwix-build!

[Jaifroid]

It is the method for building XZ and ZSTD ASM/WASM binaries that we still use for all ZIM access except full-text search in the JavaScript apps. We are unlikely to have to rebuild them very often, but it's important to document how to do it. It's not part of any CI, because the binaries are stable and in long-term use, so they are simply part of the code base.

[kelson42]

@Jaifroid OK. Any current blocker I should know about to use only libzim? AFAIK doing this would solve this ticket (and this seems the right way to me)?

[Jaifroid]

Yes, these are probably the main blockers- the first of these is the most serious, and the other two depend on resolving that one:

• Compile without Atomics / POSIX threads / pthreads openzim/javascript-libzim#43
• Try to compile to a lower target to support older browsers openzim/javascript-libzim#17
• Provide all the libzim javascript bindings openzim/javascript-libzim#11

We would also need to tweak for performance, as it's currently quite a bit slower than XZ / ZSTD access on a modern PC, and unacceptably slow on underpowered devices. But I imagine there are optimizations that could be done.

[kelson42]

@Jaifroid Two points:

• The two first bullet points seem to be really similar tickets. They basically talk about atomic/pthreads. This seems to me basically the same technical problem. My problem is that I have no clue about the quality/quantity of the user impact. What is clearly the impact for our audience (client portfolio in biz world) if we decide to not fix this problem?
• We could easily address the third bullet IMO. @mgautierfr you confirm?

[Jaifroid]

@kelson42 I agree the second point is similar, but it does involve building the app to the lowest supported browser target with the appropriate flags in Emscripten. I have experience doing this, so it's not a big issue once openzim/javascript-libzim#43 is resolved. The third one isn't a blocker for reading articles and assets because the binding needed for that has been done by mossroy and @mgautierfr. Once the first two are done, then the main focus would probably be speed optimization to be sure we don't significantly degrade user experience with the apps if we switch fully to libzim reading. But we are straying from this issue!

[kelson42]

@Jaifroid Let me try to phrase it differently: these old browsers have already kiwix-js and this will continue to work. Considering that they are really a small number of users, why not focusing on wasm from now, like it is?

[Jaifroid]

Yes, we have #770 to take forward the use of libzim for reading articles and assets, which I intend to integrate as an opt-in test version when I get the chance. But I do think that currently too few browsers and platforms support Atomic operations in JS, and that if we can get an option to deisable pthreads in libzim, it will widen the catchment significantly.

[kelson42]

But I do think that currently two few browsers and platforms support Atomic operations in JS, and that if we can get an option to deisable pthreads in libzim, it will widen the catchment significantly.

Sorry if I highjack a bit #770, but do we have numbers to support this last assertion?

[Jaifroid]

If you mean numbers of users affected, I don't know for sure how many users are stuck on such browsers or frameworks, but a good example would be all users of the Windows Store (UWP) app, especially the base app and WikiMed, which are the most used, who are currently unable to access the libzim reader due to lack of Atomics support.

[kelson42]

Yes, I mean number of users affected.

Regarding UWP, this is an old or a moderm framework? Do we have an alternative with similar features?

[Jaifroid]

UWP is a modern framework but HTML/JS apps use the Edge HTML engine (which doesn't support Atomics) instead of the new Edge Chromium engine. There are some solutions around for changing that, but they're all still experimental. Microsoft reports 30,000 acquisitions of Kiwix JS UWP (of which 6K in the last year) and 15,800 acquisitions of WikiMed by Kiwix UWP (of which 3K was in the last year). Wikivoyage is smaller.

NB I'm not saying that we must remain stuck in the past, but I'd like to keep code that supports legacy situations while introducing new features for users that can run them. It's been the philosophy of Kiwix JS and its derivatives for a long time, and it means everything is supported from IE11 and IceCat, Firefox OS, Edge Legacy, Windows Mobile, 32-bit Linux, Windows from XP to 11, Safari on iOS.... Mossroy was particularly proud of this wide support, and I'd really like to keep that legacy as long as possible, though I'm of course open to dropping things that are not used.

If we can't compile libzim for older targets without Atomics, then so be it, and libzim features will be for clients that can support them. For now, the main difference is better search with libzim.

[kelson42]

Then I guess we will have to live with this dependence for a while. I propose then to close this ticket.

[Jaifroid]

We should keep this ticket open till I have found out where Emscripten will be publishing its container images.

[rgaudin]

We should keep this ticket open till I have found out where Emscripten will be publishing its container images.

Docker Hub eventually decided to cancel its move so it's likely that emscripten will not do anything if they didn't already start a migration.

[Jaifroid]

I think it's safe to close this now.