Skip to content

Commit

Permalink
cherry pick outstanding commits (#7871)
Browse files Browse the repository at this point in the history 
* Fix bug in Transpose CUDA kernel (#7329)

* Fix permission error for ORTModule lock file (#7814)

* fix topo sort in quant tool (#7833)

* fix topo sort in quant tool

* add unit test and make the topo sort stable

* Relax tol for Conv1D fp16 test (#7844)

* Relax tol for Conv1D fp16 test

Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>

* Resolve issue with wrapped ORTModule load_state_dict (#7847)

* Encapsulate children modules inside a ModuleAccessor object to prevent erroneuos iteration over children while loading the state dictionary

* Add named_models, models, apply methods, change ModuleAccessor to ModuleMetadata and modify unit tests

* Change ModuleMetadata module getter logic, raise NotImplementedError for add_modules

* Add comment explaining why overriding _load_from_state_dict method is needed

* fixed bugs in packed mode and enable pack mode tests in ci (#7848)

* fixed bugs in packed mode and enable pack mode tests in ci

* removed unnecessary space

* pr comments

* pr comments

* disable an average pool test

* try disabling another avg pool

* disable more avg pool tests

* disable maxpool tests

* add environment variable to control default training package's local version (#7849)

* [js] update documents (#7852)

* [js] update documents

* escape double quotes

* update operators.md

* resolve comments

* Support bool type for Pad CPU (#7856)

* Initial commit

* update

* nit

* Include ORT C/C++ API headers in the ORT Mobile AAR package (#7858)

* Add header files of ort c/c++ api to aar package

* Move header file selection to cmake based on EP choice

* fix duplicated node name (#7865)

* Clean up CPU kernel definition for opset 13 Pad (#7867)

Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com>
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>
Co-authored-by: Sherlock <baihan.huang@gmail.com>
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: baijumeswani <bmeswani@microsoft.com>
Co-authored-by: Tixxx <tix@microsoft.com>
Co-authored-by: liqunfu <liqfu@microsoft.com>
Co-authored-by: Yulong Wang <yulongw@microsoft.com>
Co-authored-by: Guoyu Wang <62914304+gwang-msft@users.noreply.github.com>
Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
  • Loading branch information
@xzhu1900 @hariharans29
@yufenglee @SherlockNoMad @baijumeswani @liqunfu @fs-eire @guoyu-wang @tianleiwu
12 people authored May 28, 2021
1 parent c487824 commit a54716e
Show file tree
Hide file tree
Showing 36 changed files with 1,127 additions and 312 deletions.
47 changes: 32 additions & 15 deletions cmake/onnxruntime.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,25 @@ if (${CMAKE_SYSTEM_NAME} STREQUAL "iOS")
set(OUTPUT_STYLE xcode)
endif()

# This macro is to get the path of header files for mobile packaging, for iOS and Android
macro(get_mobile_api_headers _HEADERS)
# include both c and cxx api
set(${_HEADERS}
"${REPO_ROOT}/include/onnxruntime/core/session/onnxruntime_c_api.h"
"${REPO_ROOT}/include/onnxruntime/core/session/onnxruntime_cxx_api.h"
"${REPO_ROOT}/include/onnxruntime/core/session/onnxruntime_cxx_inline.h"
)

# need to add header files for enabled EPs
foreach(f ${ONNXRUNTIME_PROVIDER_NAMES})
file(GLOB _provider_headers CONFIGURE_DEPENDS
"${REPO_ROOT}/include/onnxruntime/core/providers/${f}/*.h"
)
list(APPEND ${_HEADERS} "${_provider_headers}")
unset(_provider_headers)
endforeach()
endmacro()

#If you want to verify if there is any extra line in symbols.txt, run
# nm -C -g --defined libonnxruntime.so |grep -v '\sA\s' | cut -f 3 -d ' ' | sort
# after build
Expand All @@ -39,21 +58,7 @@ if(WIN32)
"${ONNXRUNTIME_ROOT}/core/dll/onnxruntime.rc"
)
elseif(onnxruntime_BUILD_APPLE_FRAMEWORK)
# include both c and cxx api
set(APPLE_FRAMEWORK_HEADERS
"${REPO_ROOT}/include/onnxruntime/core/session/onnxruntime_c_api.h"
"${REPO_ROOT}/include/onnxruntime/core/session/onnxruntime_cxx_api.h"
"${REPO_ROOT}/include/onnxruntime/core/session/onnxruntime_cxx_inline.h"
)

# need to add header files for enabled EPs
foreach(f ${ONNXRUNTIME_PROVIDER_NAMES})
file(GLOB _provider_headers CONFIGURE_DEPENDS
"${REPO_ROOT}/include/onnxruntime/core/providers/${f}/*.h"
)
list(APPEND APPLE_FRAMEWORK_HEADERS "${_provider_headers}")
unset(_provider_headers)
endforeach()
get_mobile_api_headers(APPLE_FRAMEWORK_HEADERS)

# apple framework requires the header file be part of the library
onnxruntime_add_shared_library(onnxruntime
Expand Down Expand Up @@ -132,6 +137,18 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Android" OR (onnxruntime_MINIMAL_BUILD AND UNIX))
endif()
endif()

# we need to copy C/C++ API headers to be packed into Android AAR package
if(CMAKE_SYSTEM_NAME STREQUAL "Android" AND onnxruntime_BUILD_JAVA)
get_mobile_api_headers(ANDROID_AAR_HEADERS)
set(ANDROID_HEADERS_DIR ${CMAKE_CURRENT_BINARY_DIR}/android/headers)
file(MAKE_DIRECTORY ${ANDROID_HEADERS_DIR})
# copy the header files one by one
foreach(h_ ${ANDROID_AAR_HEADERS})
get_filename_component(HEADER_NAME_ ${h_} NAME)
configure_file(${h_} ${ANDROID_HEADERS_DIR}/${HEADER_NAME_} COPYONLY)
endforeach()
endif()

target_link_libraries(onnxruntime PRIVATE
onnxruntime_session
${onnxruntime_libs}
Expand Down
177 changes: 146 additions & 31 deletions js/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This directory contains multiple NPM projects:
- [onnxruntime-web](#onnxruntime-web)
- [onnxruntime-react-native](#onnxruntime-react-native)

### Development
## Development

This folder contains a `.vscode` folder for Visual Studio Code workspace configs. Using VSCode to open this folder
will allow code-formatting and linting features on typescript and C/C++ source code inside this folder. Following files
Expand All @@ -19,20 +19,45 @@ are used for code-formatting and linting features for developers:
- .eslintrc.js
- .clang-format

#### Using VSCode:
Please follow the steps described below to setup development environment.

1. in `<ORT_ROOT>/js`, run:
> npm ci
2. use VSCode to open folder `<ORT_ROOT>/js`
3. install VSCode extension if not installed yet:
- Clang-Format
- ESLint
### Prerequisites

To populate typescript type declarations, in each projects, run `npm ci`.
- Node.js (14.0+): https://nodejs.org/ - (Optional) Use nvm ([Windows](https://github.com/coreybutler/nvm-windows) / [Mac/Linux](https://github.com/creationix/nvm)) to install Node.js

#### Run code formatter and linter manually
- Python (2.7 or 3.6+): https://www.python.org/downloads/

in `<ORT_ROOT>/js`, use `npm run lint` to run ESLint , and use `npm run format` to run clang-format.
- python should be added to the PATH environment variable

- Visual Studio Code: https://code.visualstudio.com/

- **required** extension: [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint)
- **required** extension: [Clang-Format](https://marketplace.visualstudio.com/items?itemName=xaver.clang-format)
- **required** extension: [Debugger for Chrome](https://marketplace.visualstudio.com/items?itemName=msjsdiag.debugger-for-chrome)

- Chrome or Edge Browser

### Setup TypeScript development environment

In `<ORT_ROOT>/js`, run:

```
npm ci
```

This will install Clang-format and ESLint for code-formatting and linting features. This is a one-time setup unless a `git clean` is performed or folder `<ORT_ROOT>/js/node_modules` is removed manually.

### Using VSCode:

Use VSCode to open folder `<ORT_ROOT>/js`.

Make sure to open the correct folder to allow VSCode to load workspace configuration. Otherwise typescript and code formatter may not work as expected.

To populate typescript type declarations, in each project folder, run `npm ci`.

### Run code formatter and linter manually

In `<ORT_ROOT>/js`, use `npm run lint` to run ESLint , and use `npm run format` to run clang-format.

## onnxruntime-common

Expand Down Expand Up @@ -112,10 +137,6 @@ It should be able to consumed by from projects that uses NPM packages (through a
This project is a library for running ONNX models on browsers. It is the successor of [ONNX.js](https://github.com/Microsoft/onnxjs).

### Requirements

Node.js v12+ (recommended v14+)

### Build

1. Install NPM packages
Expand All @@ -124,21 +145,26 @@ Node.js v12+ (recommended v14+)
2. in `<ORT_ROOT>/js/common/`, run `npm ci`.
3. in `<ORT_ROOT>/js/web/`, run `npm ci`.

2. ~~Follow [instructions](https://www.onnxruntime.ai/docs/how-to/build.html#apis-and-language-bindings) for building ONNX Runtime WebAssembly. (TODO: document is not ready. we are working on it.)~~
2. ~~Follow [instructions](https://www.onnxruntime.ai/docs/how-to/build.html#apis-and-language-bindings) for building ONNX Runtime WebAssembly. (TODO: document is not ready. we are working on it. Please see steps described as below.)~~

in `<ORT_ROOT>/`, run either of the following commands to build WebAssembly:
in `<ORT_ROOT>/`, run one of the following commands to build WebAssembly:

```sh
# In windows, use 'build' to replace './build.sh'

# The following command build debug.
./build.sh --build_wasm

# The following command build debug with debug info.
./build.sh --build_wasm --skip_tests --enable_wasm_debug_info

# The following command build release.
./build.sh --config Release --build_wasm --skip_tests --disable_wasm_exception_catching --disable_rtti
```

To build with multi-thread support, append flag ` --enable_wasm_threads` to the command. Make sure to build both single-thread and multi-thread before next step.
To build with multi-thread support, append flag `--enable_wasm_threads` to the command. Make sure to build both single-thread and multi-thread before next step.

NOTE: You can also find latest build artifacts on [Windows WebAssembly CI Pipeline](https://dev.azure.com/onnxruntime/onnxruntime/_build?definitionId=161&_a=summary&repositoryFilter=1&branchFilter=4%2C4%2C4%2C4%2C4%2C4). Choose any build for master branch, download artifacts "Release_ort-wasm" and "Release_ort-wasm-threaded" and unzip.

3. Copy following files from build output folder to `<ORT_ROOT>/js/web/dist/`:

Expand All @@ -156,9 +182,91 @@ Node.js v12+ (recommended v14+)
npm run build
```

### Test

We use command `npm test` (test runner) and `npm run test:e2e` (E2E test) for tests in ONNXRuntime Web.

#### test runner

In folder `<ORT_ROOT>/js/web`,

- Run `npm test -- --help` for a full CLI instruction.
- Run `npm test -- <your-args> --debug` to run one or more test cases.

There are multiple levels of tests for ONNXRuntime Web:

- unit test: tests for individual components written in TypeScript. Launch unit test by:
```
npm test -- unittest
```
- model test: run a single model. The model folder should contains one .onnx model file and one or more folders for test cases, each folder contains several input*\*.pb and output*\*.pb as test data. Launch model test by:
```
npm test -- model <model_folder>
```
- op test: test a single operator. An op test is described in a `.jsonc` file which specify the operator type, its attributes and one or more test case(s), each includes a list of expected input tensor(s) and output tensor(s). The `.jsonc` file is located at `<ORT_ROOT>/js/web/test/data/ops`. Launch op test by:

```
npm test -- op <file_name>
```

- suite test: suite test includes unit test, a list of model tests and op tests. Launch suite test by:
```
npm test
```

#### E2E test

E2E test is for testing end-to-end package consuming. In this test, NPM packages for `onnxruntime-common` and `onnxruntime-web` are generated and a clean folder is used for installing packages. Then a simple mocha test is performed to make sure package can be consumed correctly.

To launch E2E test:

```
npm run test:e2e
```

### Debugging

#### Debugging TypeScript on Desktop/Chrome

To debug the code from test-runner on Chrome:

- Launch `npm test -- <your_args> --debug`. It opens an instance of Chrome browser.
- In the open Chrome browser, click the `DEBUG` button on the top-right of the page.
- In VSCode, click [side bar]->Run and Debug->select [Attach to Chrome]->click [Start Debugging] to attach.
- put breakpoints in source code, and Refresh the page to reload.

#### Debugging TypeScript on iOS/Safari

To debug on an Apple iOS device, please refer to the following steps:

- install [
RemoteDebug iOS WebKit Adapter](https://github.com/RemoteDebug/remotedebug-ios-webkit-adapter) by following its instructions.
- launch the adapter in commandline: `remotedebug_ios_webkit_adapter --port=9000`.
- in VSCode, select debug configuration `Remote Browser via Webkit Adaptor`.
- follow the steps above to debug.

#### Debugging TypeScript on Android/Chrome

To debug on an Android device, please refer to the following steps:

- Install [Android SDK Platform Tools](https://developer.android.com/studio/releases/platform-tools) and make sure `adb` is ready to use.
- Follow instructions in [Remote Debugging on Android](https://developer.chrome.com/devtools/docs/remote-debugging-legacy) to launch `adb`. Make sure to use port 9000 so that the existing debug configuration works.
- in VSCode, select debug configuration `Remote Browser via Webkit Adaptor`.
- follow the steps above to debug.

#### Debugging C/C++ for ONNX Runtime WebAssembly

To debug C/C++ code for ONNX Runtime WebAssembly, you need to build ONNX Runtime with debug info (see [Build](#Build-2)).

Currently debugging C/C++ code in WebAssembly is not supported in VSCode yet. Please follow [this instruction](https://developer.chrome.com/blog/wasm-debugging-2020/) to debug in browser devtool using extension [C/C++ DevTools Support (DWARF)](https://chrome.google.com/webstore/detail/cc%20%20-devtools-support-dwa/pdcpmagijalfljmkmjngeonclgbbannb).

### Generating Document

Use command `npm run build:doc` to generate the latest documents.

### Distribution

It should be able to consumed by both from projects that uses NPM packages (through a Node.js folder structure of `node_modules` folder that generated by `npm install onnxruntime-web`) and from a CDN service that serves a `.min.js` file and one or multiple `.wasm` file(s).
It should be able to consumed by both from projects that uses NPM packages (through a Node.js folder structure of `node_modules` folder that generated by `npm install onnxruntime-web`) and from a CDN service that serves a `ort.min.js` file and one or multiple `.wasm` file(s).

## onnxruntime-react-native

Expand Down Expand Up @@ -192,54 +300,61 @@ This project provides an ONNX Runtime React Native JavaScript library to run ONN
1. Set up an Android build environment referring to [instruction](https://www.onnxruntime.ai/docs/how-to/build.html#android)

2. In `<ORT_ROOT>`, run this python script to build ONNX Runtime Android archive file. In windows, this requires admin account to build. If an app uses a fixed set of models, refer to [instruction](https://www.onnxruntime.ai/docs/how-to/build.html#android) and build a mobile version package

```python
python tools/ci_build/github/android/build_aar_package.py js/react_native/scripts/aar_build_settings.json --config MinSizeRel --android_sdk_path <ANDROID_SDK_PATH> --android_ndk_path <ANDROID_NDK_PATH> --build_dir <BUILD_DIRECTORY>
```

3. This generates `onnxruntime-mobile-<version>.aar` in `<BUILD_DIRECTORY>/aar_out/MinSizeRel/com/microsoft/onnxruntime/onnxruntime-mobile/<version>`. Copy `aar` file into `<ORT_ROOT>/js/react_native/android/libs` and rename it as `onnxruntime.aar`

4. To verify, open Android Emulator and run this command from `<ORT_ROOT>/js/react_native/android`

```sh
adb shell am instrument -w ai.onnxruntime.react_native.test/androidx.test.runner.AndroidJUnitRunner
```

3. Build iOS ONNX Runtime package

1. Set up iOS build environment referring to [instruction](https://www.onnxruntime.ai/docs/how-to/build.html#ios).

2. Build ONNX Runtime library for iOS from `<ORT_ROOT>` using this command,

```sh
./build.sh --config MinSizeRel --use_xcode --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target 11
```

Copy `<ORT_ROOT>/build/iOS/MinSizeRel/MinSizeRel-iphoneos/libonnxruntime.<version>.dylib` file into `<ORT_ROOT>/js/react_native/ios/Libraries/onnxruntime/lib/iphoneos`

3. Clean up the previous build and build ONNX Runtime library for iOS Simulator from `<ORT_ROOT>`

```sh
./build.sh --config MinSizeRel --use_xcode --ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target 11
```

Copy `<ORT_ROOT>/build/iOS/MinSizeRel/MinSizeRel-iphonesimulator/libonnxruntime.<version>.dylib` file into `<ORT_ROOT>/js/react_native/ios/Libraries/onnxruntime/lib/iphonesimulator`

4. Edit `onnxruntime-react-native.iphoneos.podspec` and `onnxruntime-react-native.iphonesimulator.podsepc` in `<ORT_ROOT>/js/react_native` to change a version of ONNX Runtime library.

5. Copy ONNX Runtime header files

```sh
cp <ORT_ROOT>/include/onnxruntime/core/session/*.h <ORT_ROOT>/js/react_native/ios/Libraries/onnxruntime/include
```

6. To verify, open iOS Simulator and run this command from `<ORT_ROOT>/js/react_native/ios`. Change a destination to specify a running iOS Simulator.
```sh
pod install
export ONNXRUNTIME_VERSION=<version>; xcodebuild test -workspace OnnxruntimeModule.xcworkspace -scheme OnnxruntimeModuleTest -destination 'platform=iOS Simulator,name=iPhone 11,OS=14.5'
```
```sh
pod install
export ONNXRUNTIME_VERSION=<version>; xcodebuild test -workspace OnnxruntimeModule.xcworkspace -scheme OnnxruntimeModuleTest -destination 'platform=iOS Simulator,name=iPhone 11,OS=14.5'
```

4. Update a version in `package.json` to align with ONNX Runtime version.

5. Test an example for Android and iOS. In Windows, open Android Emulator first. From `<ORT_ROOT>/js/react_native`
```sh
yarn bootstrap
yarn example ios
yarn example android
```
```sh
yarn bootstrap
yarn example ios
yarn example android
```

### NPM Packaging

Expand All @@ -251,7 +366,7 @@ This project provides an ONNX Runtime React Native JavaScript library to run ONN
4. Run `npm publish <tgz> --dry-run` to see how it's going to be published

5. Run `npm publish <tgz>` to publish to npmjs
5. Run `npm publish <tgz>` to publish to npmjs. If it's for a dev, add flag `--tag dev`.
### Distribution
Expand Down
Loading

0 comments on commit a54716e

Please sign in to comment.