Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compile the libzim js bindings in a separate module #768

Closed
mossroy opened this issue Oct 29, 2021 · 11 comments
Closed

Compile the libzim js bindings in a separate module #768

mossroy opened this issue Oct 29, 2021 · 11 comments
Labels
Milestone

Comments

@mossroy
Copy link
Contributor

mossroy commented Oct 29, 2021

Like it is done for python : https://github.com/openzim/python-libzim/
It should be run by kiwix-build, too : see kiwix/kiwix-build#503

It might allow this wasm version of libzim to be used in other javascript projects, like mwoffliner

@mossroy
Copy link
Contributor Author

mossroy commented Jun 6, 2022

The bindings I used for the prototype are in https://github.com/mossroy/libzim_wasm/blob/master/demo_file_api.cpp (done with the help of @mgautierfr )
But we only did a few bindings for what was necessary for the prototype.
The goal would be to expose all the libzim APIs with these bindings (or the same ones exposed by the Python bindings)

Note that kiwx-js only needs the read APIs. But the write APIs might be of some interest for some other projects, like the node.js scrapers

@mossroy
Copy link
Contributor Author

mossroy commented Jun 6, 2022

Note that I suspect there is a memory leak in the current version of these bindings: #872

@kelson42
Copy link
Collaborator

@mgautierfr is working on the wasm compilation of the libzim with Emscripten. This might be ready next week in libzim nightly and release (from libzim 8.1.0). But this won't make the libzim directly usable from kiwix JS, we will need the wrapper. So far I understand that https://github.com/mossroy/libzim_wasm is the best we have current regarding this. Do you confirm? If "yes", I would recommend to migrate it to openzim/javascript-libzim. Would that make sense to you?

@mossroy
Copy link
Contributor Author

mossroy commented Oct 27, 2022

Yes, it's the git repo where we've compiled libzim with emscripten.
I can transfer it under the openzim umbrella, or wherever you want.
However, there is not a lot of code in it either. Feel free to copy it for kiwix.
Maybe it would be worth having a visio meeting to discuss this and its usage in kiwix-js. So that you have all the info

@kelson42
Copy link
Collaborator

If this is ok to both of you, yes would probably be beneficial to move it to openzim so further dev can take place

@mossroy
Copy link
Contributor Author

mossroy commented Oct 27, 2022

I currently don't have the right to create a git repo under openzim, so can't move it.

It would be needed to publish on gh-pages the same pages as currently: at least https://mossroy.github.io/libzim_wasm/ that is useful for testing. And maybe https://mossroy.github.io/libzim_wasm/index-bigfile.html (which is now less useful as the issue has been fixed in emscripten)
It should be possible to enable github issues on it, even if it's a fork from a very old git repo of dattaz.

@kelson42
Copy link
Collaborator

@mossroy I have sent you an invitation to be owner, shoukd be possible then.

@mossroy
Copy link
Contributor Author

mossroy commented Nov 1, 2022

Done: repo transferred to https://github.com/openzim/javascript-libzim
I've enabled issues on it.
gh-pages are enabled, so the demo pages are available again at different URLs: https://openzim.github.io/javascript-libzim/ and https://openzim.github.io/javascript-libzim/index-bigfile.html
You should have the necessary rights on the repo. If not, feel free to ask me

@kelson42
Copy link
Collaborator

kelson42 commented Nov 1, 2022

@mossroy Thank you, I will check.

@kelson42
Copy link
Collaborator

kelson42 commented Nov 7, 2022

Looks good to me. I guess this ticket should be closed now as everything will be handled in https://openzim.github.io/javascript-libzim/

@mossroy
Copy link
Contributor Author

mossroy commented Nov 19, 2022

I've transferred what remains to be done in openzim/javascript-libzim#11

@mossroy mossroy closed this as completed Nov 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants