Warning

This documentation has not been fully written yet.

Important

If you want to test txiki modules you will have to use a separate branch since it has not been merged yet. this contains the module feature alongide few more patches.

A full list of available modules is here.

What are txiki modules?

External modules are a mechanism to simplify the definition of custom runtimes based on Txiki.
Any external module specified as a requirement will be integrated within the native build process of Txiki; this way, any bundle, artefact or documentation can be generated, exposed & tested alongside the core codebase.
Txiki modules are mostly independent of each other, and kept separate from the main repository; as such, aside from API breaking changes, the development of txiki, the custom modules and the final application are fully decoupled, without the hassle of maintaining divergent branches.
External modules are specifically designed not to interfere with the original repo content, and any file generated because of them will not be tracked.

For more context on why they exist, you can refer to saghul/txiki.js#514.

Getting started

Assuming you are using txiki as a submodule in your own repository, creating a custom distribution is as easy as placing a configuration json file outside the txiki submodule.
For example, a file modules.json in its parent folder. While you can have a file modules.json inside the repo, and that is the default location assumed by most of the CLI commands, such file is meant to be generated by some wrapper script or automatically copied over before usage. It is not tracked by txiki, and as such it will not be persistent.

The modules.json file should follow the schema described in /schemas/src/modules.schema.ts. For example:

{
  "demo": "https://github.com/KaruroChori/demo-txiki-module/releases/download/v2.1.0/module.tar.gz"
}

Each entry can either point to a filesystem location, or to a tar.gz archived hosted online.
There are precise specifications on how any module should have its folders organized.

At this point, from a terminal in the submodule you can either run bun run extra-modules-clone ../modules.json.
The ./extra-helper.mjs script offers few more commands, and they are all documented using the --help option.

Now you just have to build the runtime as you usually would, and you get the interface and functionality of the demo module exposed as part of its core features.

Ideally this process is not to be performed manually; helpers scripts in the parent repository are expected to streamline the build process.

Features

This is no roadmap, but a summary of what has been implemented and what not.
As it is, the current implementation is enough to properly operate (aside from possible bugs), so there is no rush to see the rest of these features being implemented anytime soon.

Basic features

• Automatic module retrieval via fs and URI.
• Module types and documentation as we already have from the core code.
• Native dependencies retrieved, built and linked (more testing needed).
• Command to refresh a single module.

Future improvements

• Peer module dependencies. Supported in the current schema, so they can be specified, but they are not enforced or tested yet by the helper script.
• Checks on versions for both the runtime and the schema format. Some boilerplate has been implemented, but no action is taken yet.
• Schema validation. Since we now have proper schemas for module.json and module.schema.json it would be nice to enforce them in the helper script.
• Some caching. At the moment the clone command will fully refresh all modules from scratch. The refresh command is just a mitiagtion.

How to make custom modules?

Custom modules can be either external libraries which have been ported so that they can be embedded in the runtime, or fully custom code.
In either case, they are made of three mandatory components:

• Some native C/C++ code, which can be as simple as an empty function in case of a pure JS module. In case you are using C++, be mindful of exporting C-like symbols.
• A single type definition file .d.ts for the JS interface/implementation.
• A single js file containing the interface/implementation.

While the js part requires to be bundled, you can use any process you want before distributing the final module.
A common workflow is to work with multiple typescript files, and generate both js and type declarations from them in a bundled format.

To get started, you might find this demo module helpful, as it provides a basic reference implementation for a trivial module.
I also covered the topic of porting existing JS libraries as modules & a broader range of issues you might encounter in here.

Resources

This repository is not just for documentation purposes, but it also hosts the schemas for the module list & the module declaration files. They are automatically generated on release as JSON schemas. Some editors like vscodium will automatically use the specified schemas to validate against the file you are editing, which can be helpful while writing custom modules.
If your application allows that, you can directly use the typebox-based schemas from which they are generated that you can find in ./schemas/src/*.