*: support compiling tidb to wasm #13069

lucklove · 2019-11-01T08:31:21Z

Signed-off-by: lucklove gnu.crazier@gmail.com

What problem does this PR solve?

This PR make it possible to compile tidb to wasm and use tidb in browser.

What is changed and how it works?

add a term to interact with tidb core directly instead of mysql protol
add wasm platform support for some low-level operation source file
add a web interface to interact with term
add polyfill for golang's missing fs operation in wasm

Check List

Tests

Manual test

Code changes

Side effects

Related changes

Release note

Signed-off-by: lucklove <gnu.crazier@gmail.com>

lucklove · 2019-11-01T08:35:10Z

This PR works now, but need a little work to do until I remove "WIP".

cd wasm
GOOS=js GOARCH=wasm go build -o main.css
python -m SimpleHTTPServer 8000

siddontang · 2019-11-02T12:35:37Z

util/mathutil/mathutil_js.go

+const MaxInt = int(MaxUint >> 1) 
+const MinInt = -MaxInt - 1
+
+func MaxUint64(x, y uint64) uint64 {


please add comments for the export functions and vars

Ok, thanks for reminding

Signed-off-by: lucklove <gnu.crazier@gmail.com>

go.mod

codecov · 2019-11-06T07:07:54Z

Codecov Report

Merging #13069 into master will increase coverage by 0.0008%.
The diff coverage is 0%.

@@               Coverage Diff                @@
##             master     #13069        +/-   ##
================================================
+ Coverage   80.1771%   80.1779%   +0.0008%     
================================================
  Files           483        482         -1     
  Lines        121516     120911       -605     
================================================
- Hits          97428      96944       -484     
+ Misses        16330      16234        -96     
+ Partials       7758       7733        -25

Signed-off-by: lucklove <gnu.crazier@gmail.com>

syrusakbary · 2019-11-06T22:45:29Z

GOOS=js GOARCH=wasm go build -o main.css

I just inspected https://play.tidb.io and was a bit weird seeing a 13Mb css file main.css.
When inspecting, it was actually a wasm file: https://play.tidb.io/main.css

Any reason on naming it main.css rather than main.wasm ?

Makefile

.gitignore

wasm/index.html.tpl

Deardrops

Please declare the minimum browser support version for running TiDB wasm in PR description.
If needed, add some check logic when load the index.html page, prompt some information about unsupported browser.

wasm/index.html.tpl

lucklove · 2019-11-07T02:31:14Z

@syrusakbary thanks for comment.
We use .css instead of .wasm because many CDNs don't compress .wasm file automatically and unfortunately our CDN is one of them. In fact, the size of main.css is not 13MB, it's about 80MB, after compression, it's 13MB.

syrusakbary · 2019-11-07T02:51:09Z

We use .css instead of .wasm because many CDNs don't compress .wasm file automatically and unfortunately our CDN is one of them. In fact, the size of main.css is not 13MB, it's about 80MB, after compression, it's 13MB.

Oh no! Thanks for the info :)
Note: It might be worth to add that in a comment somewhere in the code (in the Makefile?) for future readers

lucklove · 2019-11-07T03:35:08Z

@syrusakbary thanks for suggestion. I'll add a comment to makefile

lucklove · 2019-11-07T03:48:04Z

known issue: golang1.13 can't produce large wasm correctly
golang/go#34395

Signed-off-by: lucklove <gnu.crazier@gmail.com>

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

5kbpers · 2019-12-13T03:29:43Z

This PR is ready to merge, PTAL, thx! @zz-jason @siddontang @ngaut

5kbpers · 2019-12-13T03:31:15Z

go.mod

@@ -78,3 +80,5 @@ require (
 go 1.13

 replace github.com/pingcap/check => github.com/tiancaiamao/check v0.0.0-20191119042138-8e73d07b629d
+
+replace github.com/coreos/go-systemd => github.com/5kbpers/go-systemd v0.0.0-20191209150347-994f05092cc6


Note: here we forked go-systemd for building WASM.

Signed-off-by: lucklove <gnu.crazier@gmail.com>

lucklove · 2019-12-25T11:20:46Z

We have fixed TiDB's Wasm compatibility problem and moved this feature to a standalone repo, closing this PR.

lucklove requested a review from a team as a code owner November 1, 2019 08:31

ghost requested review from SunRunAway and wshwsh12 and removed request for a team November 1, 2019 08:31

lucklove changed the title ~~feature: support compile to wasm~~ feature: support compile to wasm (WIP) Nov 1, 2019

feature: support compile to wasm

85d64a8

Signed-off-by: lucklove <gnu.crazier@gmail.com>

lucklove force-pushed the feature/wasm branch from c29ee56 to 85d64a8 Compare November 1, 2019 08:34

SunRunAway removed request for SunRunAway and wshwsh12 November 1, 2019 08:37

siddontang reviewed Nov 2, 2019

View reviewed changes

zz-jason changed the title ~~feature: support compile to wasm (WIP)~~ *: support compiling tidb to wasm (WIP) Nov 4, 2019

makefile: add wasm target

ded7aa1

Signed-off-by: lucklove <gnu.crazier@gmail.com>

5kbpers reviewed Nov 6, 2019

View reviewed changes

go.mod Outdated Show resolved Hide resolved

bugfix: fix stat -c on linux

b8e1a35

Signed-off-by: lucklove <gnu.crazier@gmail.com>

syrusakbary reviewed Nov 6, 2019

View reviewed changes

Makefile Show resolved Hide resolved

syrusakbary reviewed Nov 6, 2019

View reviewed changes

Makefile Outdated Show resolved Hide resolved

syrusakbary reviewed Nov 6, 2019

View reviewed changes

.gitignore Show resolved Hide resolved

syrusakbary reviewed Nov 6, 2019

View reviewed changes

wasm/index.html.tpl Show resolved Hide resolved

Deardrops reviewed Nov 7, 2019

View reviewed changes

wasm/index.html.tpl Outdated Show resolved Hide resolved

wasm/index.html.tpl Outdated Show resolved Hide resolved

Comment why use main.css instead of main.wasm

8494787

Signed-off-by: lucklove <gnu.crazier@gmail.com>

syrusakbary mentioned this pull request Nov 7, 2019

TiDB in WebAssembly WASI #13236

Closed

use local file and remove scroll bar

483d264

Signed-off-by: lucklove <gnu.crazier@gmail.com>

lucklove added 2 commits December 3, 2019 13:55

workaround wasi

3af5a0c

Signed-off-by: lucklove <gnu.crazier@gmail.com>

wasm: adapt phone keyboard

70f1703

Signed-off-by: lucklove <gnu.crazier@gmail.com>

lucklove requested a review from a team as a code owner December 3, 2019 05:57

ghost requested review from francis0407 and alivxxx and removed request for a team December 3, 2019 05:57

alivxxx removed their request for review December 3, 2019 09:44

francis0407 removed their request for review December 3, 2019 10:20

5kbpers added 2 commits December 10, 2019 21:21

merge branch 'master' into 'feature/wasm'

96ceaaf

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

replace go-systemd

b3ff6e7

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

lucklove changed the title *: support compiling tidb to wasm (can review, but don't merge) *: support compiling tidb to wasm (can review) Dec 11, 2019

5kbpers removed the status/DNM label Dec 12, 2019

5kbpers changed the title *: support compiling tidb to wasm (can review) *: support compiling tidb to wasm Dec 12, 2019

5kbpers added 5 commits December 13, 2019 10:42

add copyright

dd247ff

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

Merge branch 'master' into feature/wasm

6842573

go fmt

ca7b4ef

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

fix test

a826e7a

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

add comments

6ebef5c

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

5kbpers requested review from zz-jason, ngaut and siddontang December 13, 2019 03:27

5kbpers reviewed Dec 13, 2019

View reviewed changes

Fix scrollbar

ad57699

Signed-off-by: lucklove <gnu.crazier@gmail.com>

lucklove force-pushed the feature/wasm branch from 9c50c3b to ad57699 Compare December 17, 2019 05:43

lucklove closed this Dec 25, 2019

lucklove deleted the feature/wasm branch December 26, 2019 11:40

RinatValiullov mentioned this pull request Nov 24, 2021

Темы эфира #141 (25 ноября) znprod/stream-topics#62

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: support compiling tidb to wasm #13069

*: support compiling tidb to wasm #13069

lucklove commented Nov 1, 2019

lucklove commented Nov 1, 2019

siddontang Nov 2, 2019

lucklove Nov 3, 2019

codecov bot commented Nov 6, 2019 •

edited

Loading

syrusakbary commented Nov 6, 2019

Deardrops left a comment •

edited

Loading

lucklove commented Nov 7, 2019

syrusakbary commented Nov 7, 2019 •

edited

Loading

lucklove commented Nov 7, 2019

lucklove commented Nov 7, 2019

5kbpers commented Dec 13, 2019

5kbpers Dec 13, 2019

lucklove commented Dec 25, 2019 •

edited

Loading

*: support compiling tidb to wasm #13069

*: support compiling tidb to wasm #13069

Conversation

lucklove commented Nov 1, 2019

What problem does this PR solve?

What is changed and how it works?

Check List

lucklove commented Nov 1, 2019

siddontang Nov 2, 2019

Choose a reason for hiding this comment

lucklove Nov 3, 2019

Choose a reason for hiding this comment

codecov bot commented Nov 6, 2019 • edited Loading

Codecov Report

syrusakbary commented Nov 6, 2019

Deardrops left a comment • edited Loading

Choose a reason for hiding this comment

lucklove commented Nov 7, 2019

syrusakbary commented Nov 7, 2019 • edited Loading

lucklove commented Nov 7, 2019

lucklove commented Nov 7, 2019

5kbpers commented Dec 13, 2019

5kbpers Dec 13, 2019

Choose a reason for hiding this comment

lucklove commented Dec 25, 2019 • edited Loading

codecov bot commented Nov 6, 2019 •

edited

Loading

Deardrops left a comment •

edited

Loading

syrusakbary commented Nov 7, 2019 •

edited

Loading

lucklove commented Dec 25, 2019 •

edited

Loading