boa: support running Python code in non-blocking way #602

yorkie · 2020-09-30T06:49:31Z

This PR uses the Node.js worker_threads to start a new thread to run Python functions, and add SharedPythonObject to share(not copy) the Python object in sub-threads.

The following is an experimental API usage(example):

const { Worker, isMainThread, workerData, parentPort } = require('worker_threads');
const boa = require('@pipcook/boa');
const pybasic = boa.import('tests.base.basic');
const { SharedPythonObject, symbols } = boa;

class Foobar extends pybasic.Foobar {
  hellomsg(x) {
    return `hello <${x}> on ${this.test}(${this.count})`;
  }
}

if (isMainThread) {
  const foo = new Foobar();
  const worker = new Worker(__filename, {
    workerData: {
      foo: new SharedPythonObject(foo),
      // the `foo` is unavailable util the sub-thread exits.
      // it throws an error if trying to use `foo`.
    },
  });
  let alive = setInterval(() => {
    // this is always alive though it's blocking by `foo.sleep()` in sub-thread.
    console.log(`main: still training, ownership(${foo[symbols.GetOwnershipSymbol]()})`);
  }, 1000);

  worker.on('message', (state) => {
    if (state === 'done') {
      clearInterval(alive);
      // the process exits
    }
  });
} else {
  const { foo } = workerData; // the foo is a borrowed object from main thread by `SharedPythonObject`.
  foo.sleep(); // call Python's method `sleep()`, it would block the sub-thread 5s.
  setTimeout(() => {
    // this will 1) do ref the sub-thread 2000ms, 2) send a "done" message and 3) exit the sub-thread.
    parentPort.postMessage('done');
  }, 2000);
}

Note: an object could only be shared with 1 sub-thread at the same time.

TODO

improvements
- avoid to create SharedPythonObject twice and multiple times on a same object.
- ~~avoid crash(segfault) if the ownershipId is invalid.~~
unit tests
documentation

It uses the Node.js worker_threads to start new thread to run Python functions. Note that, we share the same Python interrupter.

…er-threads-support

yorkie · 2020-09-30T16:36:02Z

@FeelyChau Just mark this PR be ready for review :)

yorkie · 2020-09-30T18:05:42Z

Occurred the following error if running npm test:

* thread #1, stop reason = signal SIGSTOP
  * frame #0: 0x000000010554b27e libpython3.7m.dylib`PyErr_FormatV + 30
    frame #1: 0x000000010554b239 libpython3.7m.dylib`PyErr_Format + 137
    frame #2: 0x0000000105478590 libpython3.7m.dylib`_PyObject_GenericGetAttrWithDict + 848
    frame #3: 0x0000000105477b26 libpython3.7m.dylib`PyObject_GetAttrString + 134
    frame #4: 0x0000000105477c4c libpython3.7m.dylib`PyObject_HasAttrString + 12
    frame #5: 0x000000010526c589 boa.node`boa::PythonObject::HasAttr(Napi::CallbackInfo const&) [inlined] pybind11::hasattr(obj=<unavailable>, name=<unavailable>) at pytypes.h:397:12 [opt]
    frame #6: 0x000000010526c584 boa.node`boa::PythonObject::HasAttr(this=0x00000001053091e0, info=0x00007ffeefbfc4a0) at object.cc:315 [opt]
    frame #7: 0x0000000105271a9b boa.node`Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(this=<unavailable>)::'lambda'()::operator()() const at napi-inl.h:3838:12 [opt]
    frame #8: 0x00000001052719ca boa.node`Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(napi_env__*, napi_callback_info__*) [inlined] napi_value__* Napi::details::WrapCallback<Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(napi_env__*, napi_callback_info__*)::'lambda'()>(Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(napi_env__*, napi_callback_info__*)::'lambda'()) at napi-inl.h:73:12 [opt]
    frame #9: 0x00000001052719c5 boa.node`Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(env=0x000000010500b280, info=0x00007ffeefbfc580) at napi-inl.h:3831 [opt]
    frame #10: 0x000000010006346a node`v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) + 122
    frame #11: 0x00000001002580c8 node`v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) + 616
    frame #12: 0x000000010025765c node`v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) + 524
    frame #13: 0x0000000100256d82 node`v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) + 258
    frame #14: 0x0000000100a50f59 node`Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit + 57

yorkie · 2020-09-30T19:16:41Z

Occurred the following error if running npm test:

* thread #1, stop reason = signal SIGSTOP
  * frame #0: 0x000000010554b27e libpython3.7m.dylib`PyErr_FormatV + 30
    frame #1: 0x000000010554b239 libpython3.7m.dylib`PyErr_Format + 137
    frame #2: 0x0000000105478590 libpython3.7m.dylib`_PyObject_GenericGetAttrWithDict + 848
    frame #3: 0x0000000105477b26 libpython3.7m.dylib`PyObject_GetAttrString + 134
    frame #4: 0x0000000105477c4c libpython3.7m.dylib`PyObject_HasAttrString + 12
    frame #5: 0x000000010526c589 boa.node`boa::PythonObject::HasAttr(Napi::CallbackInfo const&) [inlined] pybind11::hasattr(obj=<unavailable>, name=<unavailable>) at pytypes.h:397:12 [opt]
    frame #6: 0x000000010526c584 boa.node`boa::PythonObject::HasAttr(this=0x00000001053091e0, info=0x00007ffeefbfc4a0) at object.cc:315 [opt]
    frame #7: 0x0000000105271a9b boa.node`Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(this=<unavailable>)::'lambda'()::operator()() const at napi-inl.h:3838:12 [opt]
    frame #8: 0x00000001052719ca boa.node`Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(napi_env__*, napi_callback_info__*) [inlined] napi_value__* Napi::details::WrapCallback<Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(napi_env__*, napi_callback_info__*)::'lambda'()>(Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(napi_env__*, napi_callback_info__*)::'lambda'()) at napi-inl.h:73:12 [opt]
    frame #9: 0x00000001052719c5 boa.node`Napi::ObjectWrap<boa::PythonObject>::InstanceMethodCallbackWrapper(env=0x000000010500b280, info=0x00007ffeefbfc580) at napi-inl.h:3831 [opt]
    frame #10: 0x000000010006346a node`v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) + 122
    frame #11: 0x00000001002580c8 node`v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) + 616
    frame #12: 0x000000010025765c node`v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) + 524
    frame #13: 0x0000000100256d82 node`v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) + 258
    frame #14: 0x0000000100a50f59 node`Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit + 57

This is addressed, the sub-thread would call to the finalize_interpreter() which causes the following calls to Python would crash, just added an extra state initialized to flag the finalization op must be in the main thread and Py_IsInitialized() is true.

yorkie · 2020-10-01T13:39:54Z

@FeelyChau See https://github.com/alibaba/pipcook/pull/602/checks?check_run_id=1192579536, it seems failed by daemon tests which are not related to this PR.

FeelyChau · 2020-10-02T09:32:44Z

@FeelyChau See https://github.com/alibaba/pipcook/pull/602/checks?check_run_id=1192579536, it seems failed by daemon tests which are not related to this PR.

OK, I'll check it out.

WenheLI

LGTM

packages/boa/lib/proxy.js

FeelyChau · 2020-10-10T06:59:23Z

packages/boa/src/core/object.cc

+  if (_ownership == nullptr) {
+    return true;
+  } else {
+    bool owned = _ownership->getOwned();


It seems not thread-safe here.

FeelyChau · 2020-10-10T09:49:53Z

packages/boa/src/core/object.h

+private:
+  mutable std::mutex _mtx;
+  PyObject *_pyobject;
+  bool _owned;


We can use std::atomic_flag instead to improve performance.

Co-authored-by: feely <feely@outlook.com>

FeelyChau · 2020-10-10T15:04:43Z

LGTM.

boa: support running Python code in non-blocking way

6845794

It uses the Node.js worker_threads to start new thread to run Python functions. Note that, we share the same Python interrupter.

yorkie added the boa Python related issues label Sep 30, 2020

yorkie requested a review from FeelyChau September 30, 2020 06:49

yorkie added 8 commits September 30, 2020 14:50

remove initialized

2ad8ed4

fix nits

1e68990

fix nits

8bfda7f

fix exit segmentfault

7439800

fix nits

500c8e4

fix nits

9d9782b

Merge branch 'master' of github.com:alibaba/pipcook into boa/add-work…

e7596fb

…er-threads-support

revert useless changes

4ccdb09

yorkie marked this pull request as ready for review September 30, 2020 16:35

yorkie requested a review from WenheLI September 30, 2020 16:38

yorkie added 4 commits October 1, 2020 01:04

fix nits

6f2ca6c

fix nits

7ca1ab3

fix c++ lint

b1f3b0e

fix nits

cd608e3

fix nits

7289482

add more testcase

9c52f07

add docs

5ae54c9

yorkie mentioned this pull request Oct 9, 2020

feat: add tvm wasm export #582

Closed

WenheLI previously approved these changes Oct 10, 2020

View reviewed changes

FeelyChau requested changes Oct 10, 2020

View reviewed changes

Update packages/boa/lib/proxy.js

ab7dce6

Co-authored-by: feely <feely@outlook.com>

yorkie dismissed WenheLI’s stale review via ab7dce6 October 10, 2020 11:55

yorkie added 2 commits October 10, 2020 21:06

fix nits

61179cb

fix nits

3551ca4

FeelyChau approved these changes Oct 10, 2020

View reviewed changes

FeelyChau merged commit 1b50e4e into master Oct 10, 2020

FeelyChau deleted the boa/add-worker-threads-support branch October 10, 2020 15:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

boa: support running Python code in non-blocking way #602

boa: support running Python code in non-blocking way #602

yorkie commented Sep 30, 2020 •

edited

Loading

yorkie commented Sep 30, 2020

yorkie commented Sep 30, 2020

yorkie commented Sep 30, 2020

yorkie commented Oct 1, 2020 •

edited

Loading

FeelyChau commented Oct 2, 2020

WenheLI left a comment

FeelyChau Oct 10, 2020

FeelyChau Oct 10, 2020

FeelyChau commented Oct 10, 2020

boa: support running Python code in non-blocking way #602

boa: support running Python code in non-blocking way #602

Conversation

yorkie commented Sep 30, 2020 • edited Loading

TODO

yorkie commented Sep 30, 2020

yorkie commented Sep 30, 2020

yorkie commented Sep 30, 2020

yorkie commented Oct 1, 2020 • edited Loading

FeelyChau commented Oct 2, 2020

WenheLI left a comment

Choose a reason for hiding this comment

FeelyChau Oct 10, 2020

Choose a reason for hiding this comment

FeelyChau Oct 10, 2020

Choose a reason for hiding this comment

FeelyChau commented Oct 10, 2020

yorkie commented Sep 30, 2020 •

edited

Loading

yorkie commented Oct 1, 2020 •

edited

Loading