Skip to content

Commit

Permalink
Switch to WASM, rewrite build pipeline
Browse files Browse the repository at this point in the history
init

WIP

WIP!

Rewrite from scratch

Inline WASM

add github action

cleanup

Setup vite for debugging

Fix se;f reference in Node environment

Use ICU from EMSDK

cleanup

Build WASM module as sync CommonJS module

Cleanup

Cleanup

Final cleanup

cleanup
  • Loading branch information
stepankuzmin committed Jun 14, 2024
1 parent 08461ee commit 03884b3
Show file tree
Hide file tree
Showing 28 changed files with 3,316 additions and 20,535 deletions.
32 changes: 0 additions & 32 deletions .eslintrc

This file was deleted.

37 changes: 37 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: CI

on:
push:
branches: [ "main" ]

pull_request:
branches: [ "main" ]

jobs:
test:
runs-on: ubuntu-latest

strategy:
matrix:
node-version: [20.x]

steps:
- uses: actions/checkout@v4

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'

- run: npm ci
- run: npm test
- run: npm run lint
- run: npm run build

- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: mapbox-gl-rtl-text
path: dist
retention-days: 7
11 changes: 3 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,4 @@
.DS_Store
*~
/emsdk-portable
/icu
/icu-llvm
*.tgz
/node_modules
/build
/.nyc_output

/dist
/node_modules
95 changes: 0 additions & 95 deletions Makefile

This file was deleted.

69 changes: 35 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,58 +1,59 @@
# mapbox-gl-rtl-text.js

[![Build Status](https://circleci.com/gh/mapbox/mapbox-gl-rtl-text.svg?style=shield)](https://circleci.com/gh/mapbox/mapbox-gl-rtl-text)
[![CI](https://github.com/mapbox/mapbox-gl-rtl-text/actions/workflows/ci.yml/badge.svg)](https://github.com/mapbox/mapbox-gl-rtl-text/actions/workflows/ci.yml)

An [Emscripten](https://github.com/kripken/emscripten) port of a subset of the functionality of [International Components for Unicode (ICU)](http://site.icu-project.org/) necessary for [Mapbox GL JS](https://github.com/mapbox/mapbox-gl-js) to support [right to left text rendering](https://github.com/mapbox/mapbox-gl/issues/4). Supports the Arabic and Hebrew languages, which are written right-to-left. Mapbox Studio loads this plugin by default.
An [Emscripten](https://github.com/emscripten-core/emscripten) port of a subset of the functionality of [International Components for Unicode (ICU)](http://site.icu-project.org/) necessary for [Mapbox GL JS](https://github.com/mapbox/mapbox-gl-js) to support [right to left text rendering](https://github.com/mapbox/mapbox-gl/issues/4). Supports the Arabic and Hebrew languages, which are written right-to-left. Mapbox Studio loads this plugin by default.

**Requires [mapbox-gl-js](https://github.com/mapbox/mapbox-gl-js) (version 0.32.1 and up).**

A map that requires Arabic names should at a minimum install the `mapbox-gl-rtl-text` plugin. To display the actual place names, the map could use a specially modified style, manipulate the style at runtime, or install the [`mapbox-gl-language`](https://github.com/mapbox/mapbox-gl-language/) plugin for convenience. The `mapbox-gl-language` plugin displays Arabic name data (among other languages), while the `mapbox-gl-rtl-text` plugin adds support for displaying Arabic names.
A map that requires Arabic names should at a minimum install the `mapbox-gl-rtl-text` plugin. To display the actual place names, the map could use a specially modified style, manipulate the style at runtime, or install the [`mapbox-gl-language`](https://github.com/mapbox/mapbox-gl-language/) plugin for convenience. The `mapbox-gl-language` plugin displays Arabic name data (among other languages), while the `mapbox-gl-rtl-text` plugin adds support for displaying Arabic names.

## Using mapbox-gl-rtl-text

mapbox-gl-rtl-text exposes two functions:

### applyArabicShaping(unicodeInput)
### `applyArabicShaping(unicodeInput)`

Takes an input string in "logical order" (i.e. characters in the order they are typed, not the order they will be displayed) and replaces Arabic characters with the "presentation form" of the character that represents the appropriate glyph based on the character's location within a word.

### processBidirectionalText(unicodeInput, lineBreakPoints)
Takes an input string with characters in "logical order", along with a set of chosen line break points, and applies the [Unicode Bidirectional Algorithm](http://unicode.org/reports/tr9/) to the string. Returns an ordered set of lines with characters in "visual order" (i.e. characters in the order they are displayed, left-to-right). The algorithm will insert mandatory line breaks (`\n` etc.) if they are not already included in `lineBreakPoints`.
### `processBidirectionalText(unicodeInput, lineBreakPoints)`

`mapbox-gl-rtl-text.js`/`mapbox-gl-rtl-text.min.js` are built to be loaded directly by Mapbox GL JS using:
Takes an input string with characters in "logical order", along with a set of chosen line break points, and applies the [Unicode Bidirectional Algorithm](http://unicode.org/reports/tr9/) to the string. Returns an ordered set of lines with characters in "visual order" (i.e. characters in the order they are displayed, left-to-right). The algorithm will insert mandatory line breaks (`\n` etc.) if they are not already included in `lineBreakPoints`.

setRTLTextPlugin('mapbox-gl-rtl-text.js');
`mapbox-gl-rtl-text.js` is built to be loaded directly by Mapbox GL JS using:

You can use ICU JS directly:
```
var rtlText = require('mapbox-gl-rtl-text');
var arabicString = "سلام";
var shapedArabicText = rtlText.applyArabicShaping(arabicString);
var readyForDisplay = rtlText.processBidirectionalText(shapedArabicText, []);
```js
mapboxgl.setRTLTextPlugin('mapbox-gl-rtl-text.js');
```
## Building mapbox-gl-rtl-text

Running `build.sh` will:
You can use ICU JS directly:
```js
import rtlText from '@mapbox/mapbox-gl-rtl-text';
const {applyArabicShaping, processBidirectionalText} = await rtlText;

- Download Emscripten
- Download and compile ICU to LLVM bytecode
- Run `make all`
- Compile `ushape_wrapper.c` and `ubidi_wrapper.c` to LLVM bytecode
- Generate `wrapper.js`, exposing bytecode sources as Javascript
- Embed `wrapper.js` in `index.js` for use with Browserify, and `mapbox-gl-rtl-text.js` for loading directly as a GL JS plugin
const arabicString = "سلام";
const shapedArabicText = applyArabicShaping(arabicString);
const readyForDisplay = processBidirectionalText(shapedArabicText, []);
```

Build process only tested on MacOS 10.12 and Ubuntu Xenial.
## Building mapbox-gl-rtl-text

Running `npm test` will run unit tests in `test/*.test.js`. Use `npm test -- --cov` to generate code coverage stats.
* Running `npm start` will spin up a local server to test the plugin in a browser.
* Running `npm test` will run unit tests in `test.js`.
* Running `npm run build:icu` will rebuild ICU WASM module:
- Download Emscripten SDK Docker Image
- Compile `ushape_wrapper.c` and `ubidi_wrapper.c` to LLVM bytecode
- Generate `./src/icu.wasm.js`, exposing bytecode sources as WASM module

## Deploying mapbox-gl-rtl-text

- `./build.sh`
- `npm test`
- `npm version {patch|minor|major}`
- `git push --follow-tags`
- `mbx env`
- `aws s3 cp --acl public-read --content-type application/javascript mapbox-gl-rtl-text.min.js s3://mapbox-gl-js/plugins/mapbox-gl-rtl-text/v$(node --print --eval "require('./package.json').version")/mapbox-gl-rtl-text.js`
- `mbx npm publish`

## Experimental Web Assembly support
`make all` will now build a second version of the plugin built using Web Assembly. Once wasm support is widespread, the technology promises smaller package sizes and faster load times. The output file `mapbox-gl-rtl-text.wasm.js` will try to locally load a `wrapper.wasm.wasm`. To test the wasm version of the plugin, you need to somehow host `wrapper.wasm` and modify the JavaScript wrapper to pick it up.
```
npm test
npm version {patch|minor|major}
git push --follow-tags
mbx env
VERSION=v$(node -p "require('./package.json').version")
aws s3 cp --acl public-read --content-type application/javascript mapbox-gl-rtl-text.js s3://mapbox-gl-js/plugins/mapbox-gl-rtl-text/v$VERSION/mapbox-gl-rtl-text.js
mbx npm publish
```
Loading

0 comments on commit 03884b3

Please sign in to comment.