Make driver take optional executor #8
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D30193354 |
3 similar comments
|
This pull request was exported from Phabricator. Differential Revision: D30193354 |
|
This pull request was exported from Phabricator. Differential Revision: D30193354 |
|
This pull request was exported from Phabricator. Differential Revision: D30193354 |
Summary: Pull Request resolved: #8 There are cases that may require callers to supply their own executor. Make it pluggable through the interface. Reviewed By: pedroerp Differential Revision: D30193354 fbshipit-source-id: ff1a37e43a855af89220e0e4dffb08b951e87954
|
This pull request was exported from Phabricator. Differential Revision: D30193354 |
|
This pull request has been merged in 84ab493. |
winningsix
pushed a commit
to winningsix/velox-1
that referenced
this pull request
Sep 27, 2021
* Address comments from PR facebookincubator#5 * Refactor to pass down IR to Operator level
winningsix
pushed a commit
to winningsix/velox-1
that referenced
this pull request
Sep 27, 2021
* Address comments from PR facebookincubator#5 * Refactor to pass down IR to Operator level
Closed
facebook-github-bot
pushed a commit
that referenced
this pull request
Mar 16, 2022
Summary:
There was a race condition that caused ASAN heap-use-after-free in
Driver::runInternal. It is possible for the Task to terminate (due to an error)
while Driver is off-thread sitting in the executor's queue waiting to be
scheduled to run on a thread. If this case, Task may have been terminated and
operators have been destroyed just before Driver::runInternal executes. In this
case `operators_[curOpIndex_]->stats().addRuntimeStat
("queuedWallNanos", ...);` call would try to access freed memory.
The fix is to (1) streamline Driver by removing task_ member variable and use
DriverCtx::task instead; (2) move logic for updating stats after the call to
task->enter(). If task has terminated, enter() will return
StopReason::kTerminate and no Driver will abort any further processing.
```AddressSanitizer: heap-use-after-free
#6 0x7f01c29ae9b9 in facebook::velox::exec::OperatorStats::addRuntimeStat(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long) velox/exec/Operator.h:144
#7 0x7f01c29afaf7 in facebook::velox::exec::Driver::runInternal(std::shared_ptr<facebook::velox::exec::Driver>&, std::shared_ptr<facebook::velox::exec::BlockingState>*) velox/exec/Driver.cpp:249
#8 0x7f01c29b4174 in facebook::velox::exec::Driver::run(std::shared_ptr<facebook::velox::exec::Driver>) velox/exec/Driver.cpp:415
#9 0x7f01c29cce5f in facebook::velox::exec::Driver::enqueue(std::shared_ptr<facebook::velox::exec::Driver>)::$_0::operator()() const velox/exec/Driver.cpp:161
```
Pull Request resolved: #1219
Test Plan:
$ buck test mode/dev-asan //presto_cpp/main/tests:presto_main_test -- --exact 'presto_cpp/main/tests:presto_main_test - TaskManagerTest.outOfQueryUserMemory' --run-disabled --stress-runs 1000
```
Summary
Pass: 1000
ListingSuccess: 1
Finished test run: https://www.internalfb.com/intern/testinfra/testrun/1125900137408748
```
Imported from GitHub, without a `Test Plan:` line.
Reviewed By: spershin, oerling
Differential Revision: D34904220
Pulled By: mbasmanova
fbshipit-source-id: d4b88cf9e3ec2884c77371c37c571297a59728b6
ZJie1
pushed a commit
to ZJie1/velox
that referenced
this pull request
Mar 18, 2022
Summary:
There was a race condition that caused ASAN heap-use-after-free in
Driver::runInternal. It is possible for the Task to terminate (due to an error)
while Driver is off-thread sitting in the executor's queue waiting to be
scheduled to run on a thread. If this case, Task may have been terminated and
operators have been destroyed just before Driver::runInternal executes. In this
case `operators_[curOpIndex_]->stats().addRuntimeStat
("queuedWallNanos", ...);` call would try to access freed memory.
The fix is to (1) streamline Driver by removing task_ member variable and use
DriverCtx::task instead; (2) move logic for updating stats after the call to
task->enter(). If task has terminated, enter() will return
StopReason::kTerminate and no Driver will abort any further processing.
```AddressSanitizer: heap-use-after-free
facebookincubator#6 0x7f01c29ae9b9 in facebook::velox::exec::OperatorStats::addRuntimeStat(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long) velox/exec/Operator.h:144
facebookincubator#7 0x7f01c29afaf7 in facebook::velox::exec::Driver::runInternal(std::shared_ptr<facebook::velox::exec::Driver>&, std::shared_ptr<facebook::velox::exec::BlockingState>*) velox/exec/Driver.cpp:249
facebookincubator#8 0x7f01c29b4174 in facebook::velox::exec::Driver::run(std::shared_ptr<facebook::velox::exec::Driver>) velox/exec/Driver.cpp:415
facebookincubator#9 0x7f01c29cce5f in facebook::velox::exec::Driver::enqueue(std::shared_ptr<facebook::velox::exec::Driver>)::$_0::operator()() const velox/exec/Driver.cpp:161
```
Pull Request resolved: facebookincubator#1219
Test Plan:
$ buck test mode/dev-asan //presto_cpp/main/tests:presto_main_test -- --exact 'presto_cpp/main/tests:presto_main_test - TaskManagerTest.outOfQueryUserMemory' --run-disabled --stress-runs 1000
```
Summary
Pass: 1000
ListingSuccess: 1
Finished test run: https://www.internalfb.com/intern/testinfra/testrun/1125900137408748
```
Imported from GitHub, without a `Test Plan:` line.
Reviewed By: spershin, oerling
Differential Revision: D34904220
Pulled By: mbasmanova
fbshipit-source-id: d4b88cf9e3ec2884c77371c37c571297a59728b6
facebook-github-bot
pushed a commit
that referenced
this pull request
Apr 28, 2022
Summary: Enhance printExprWithStats to identify common-sub expressions. For example, `c0 + c1` is a common sub-expression in `"(c0 + c1) % 5", " (c0 + c1) % 3"` expression set. It is evaluated only once and there is a single Expr object that represents it. That object appears in the expression tree twice. printExprWithStats does not show the runtime stats for second instance of that expression and instead annotates it with `[CSE https://github.com/facebookincubator/velox/issues/2]`, where CSE stands for common sub-expression and 2 refers to the first instance of the expression. ``` mod [cpu time: 50.49us, rows: 1024] -> BIGINT [#1] cast(plus as BIGINT) [cpu time: 68.15us, rows: 1024] -> BIGINT [#2] plus [cpu time: 51.84us, rows: 1024] -> INTEGER [#3] c0 [cpu time: 0ns, rows: 0] -> INTEGER [#4] c1 [cpu time: 0ns, rows: 0] -> INTEGER [#5] 5:BIGINT [cpu time: 0ns, rows: 0] -> BIGINT [#6] mod [cpu time: 49.29us, rows: 1024] -> BIGINT [#7] cast((plus(c0, c1)) as BIGINT) -> BIGINT [CSE #2] 3:BIGINT [cpu time: 0ns, rows: 0] -> BIGINT [#8] ``` Pull Request resolved: #1500 Reviewed By: Yuhta Differential Revision: D35994836 Pulled By: mbasmanova fbshipit-source-id: 6bacbbe61b68dad97ce2fd5f99610c4ad55897be
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
May 9, 2022
Arty-Maly
pushed a commit
to Arty-Maly/velox
that referenced
this pull request
May 13, 2022
…tor#1500) Summary: Enhance printExprWithStats to identify common-sub expressions. For example, `c0 + c1` is a common sub-expression in `"(c0 + c1) % 5", " (c0 + c1) % 3"` expression set. It is evaluated only once and there is a single Expr object that represents it. That object appears in the expression tree twice. printExprWithStats does not show the runtime stats for second instance of that expression and instead annotates it with `[CSE https://github.com/facebookincubator/velox/issues/2]`, where CSE stands for common sub-expression and 2 refers to the first instance of the expression. ``` mod [cpu time: 50.49us, rows: 1024] -> BIGINT [#1] cast(plus as BIGINT) [cpu time: 68.15us, rows: 1024] -> BIGINT [facebookincubator#2] plus [cpu time: 51.84us, rows: 1024] -> INTEGER [facebookincubator#3] c0 [cpu time: 0ns, rows: 0] -> INTEGER [facebookincubator#4] c1 [cpu time: 0ns, rows: 0] -> INTEGER [facebookincubator#5] 5:BIGINT [cpu time: 0ns, rows: 0] -> BIGINT [facebookincubator#6] mod [cpu time: 49.29us, rows: 1024] -> BIGINT [facebookincubator#7] cast((plus(c0, c1)) as BIGINT) -> BIGINT [CSE facebookincubator#2] 3:BIGINT [cpu time: 0ns, rows: 0] -> BIGINT [facebookincubator#8] ``` Pull Request resolved: facebookincubator#1500 Reviewed By: Yuhta Differential Revision: D35994836 Pulled By: mbasmanova fbshipit-source-id: 6bacbbe61b68dad97ce2fd5f99610c4ad55897be
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Jun 24, 2022
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Jun 30, 2022
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Jul 20, 2022
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Aug 2, 2022
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Aug 12, 2022
shiyu-bytedance
pushed a commit
to shiyu-bytedance/velox-1
that referenced
this pull request
Aug 18, 2022
Summary:
There was a race condition that caused ASAN heap-use-after-free in
Driver::runInternal. It is possible for the Task to terminate (due to an error)
while Driver is off-thread sitting in the executor's queue waiting to be
scheduled to run on a thread. If this case, Task may have been terminated and
operators have been destroyed just before Driver::runInternal executes. In this
case `operators_[curOpIndex_]->stats().addRuntimeStat
("queuedWallNanos", ...);` call would try to access freed memory.
The fix is to (1) streamline Driver by removing task_ member variable and use
DriverCtx::task instead; (2) move logic for updating stats after the call to
task->enter(). If task has terminated, enter() will return
StopReason::kTerminate and no Driver will abort any further processing.
```AddressSanitizer: heap-use-after-free
facebookincubator#6 0x7f01c29ae9b9 in facebook::velox::exec::OperatorStats::addRuntimeStat(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long) velox/exec/Operator.h:144
facebookincubator#7 0x7f01c29afaf7 in facebook::velox::exec::Driver::runInternal(std::shared_ptr<facebook::velox::exec::Driver>&, std::shared_ptr<facebook::velox::exec::BlockingState>*) velox/exec/Driver.cpp:249
facebookincubator#8 0x7f01c29b4174 in facebook::velox::exec::Driver::run(std::shared_ptr<facebook::velox::exec::Driver>) velox/exec/Driver.cpp:415
facebookincubator#9 0x7f01c29cce5f in facebook::velox::exec::Driver::enqueue(std::shared_ptr<facebook::velox::exec::Driver>)::$_0::operator()() const velox/exec/Driver.cpp:161
```
Pull Request resolved: facebookincubator#1219
Test Plan:
$ buck test mode/dev-asan //presto_cpp/main/tests:presto_main_test -- --exact 'presto_cpp/main/tests:presto_main_test - TaskManagerTest.outOfQueryUserMemory' --run-disabled --stress-runs 1000
```
Summary
Pass: 1000
ListingSuccess: 1
Finished test run: https://www.internalfb.com/intern/testinfra/testrun/1125900137408748
```
Imported from GitHub, without a `Test Plan:` line.
Reviewed By: spershin, oerling
Differential Revision: D34904220
Pulled By: mbasmanova
fbshipit-source-id: d4b88cf9e3ec2884c77371c37c571297a59728b6
shiyu-bytedance
pushed a commit
to shiyu-bytedance/velox-1
that referenced
this pull request
Aug 18, 2022
…tor#1500) Summary: Enhance printExprWithStats to identify common-sub expressions. For example, `c0 + c1` is a common sub-expression in `"(c0 + c1) % 5", " (c0 + c1) % 3"` expression set. It is evaluated only once and there is a single Expr object that represents it. That object appears in the expression tree twice. printExprWithStats does not show the runtime stats for second instance of that expression and instead annotates it with `[CSE https://github.com/facebookincubator/velox/issues/2]`, where CSE stands for common sub-expression and 2 refers to the first instance of the expression. ``` mod [cpu time: 50.49us, rows: 1024] -> BIGINT [facebookincubator#1] cast(plus as BIGINT) [cpu time: 68.15us, rows: 1024] -> BIGINT [facebookincubator#2] plus [cpu time: 51.84us, rows: 1024] -> INTEGER [facebookincubator#3] c0 [cpu time: 0ns, rows: 0] -> INTEGER [facebookincubator#4] c1 [cpu time: 0ns, rows: 0] -> INTEGER [facebookincubator#5] 5:BIGINT [cpu time: 0ns, rows: 0] -> BIGINT [facebookincubator#6] mod [cpu time: 49.29us, rows: 1024] -> BIGINT [facebookincubator#7] cast((plus(c0, c1)) as BIGINT) -> BIGINT [CSE facebookincubator#2] 3:BIGINT [cpu time: 0ns, rows: 0] -> BIGINT [facebookincubator#8] ``` Pull Request resolved: facebookincubator#1500 Reviewed By: Yuhta Differential Revision: D35994836 Pulled By: mbasmanova fbshipit-source-id: 6bacbbe61b68dad97ce2fd5f99610c4ad55897be
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Aug 22, 2022
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Sep 7, 2022
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Sep 26, 2022
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
This was referenced Oct 8, 2022
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Oct 26, 2022
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Nov 8, 2022
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Nov 8, 2022
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Nov 22, 2022
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Dec 15, 2022
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Jan 6, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Jan 12, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
PHILO-HE
pushed a commit
to PHILO-HE/velox
that referenced
this pull request
Feb 3, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Feb 24, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
liujiayi771
pushed a commit
to liujiayi771/velox
that referenced
this pull request
Mar 3, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
liujiayi771
pushed a commit
to liujiayi771/velox
that referenced
this pull request
Mar 9, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
pushed a commit
to rui-mo/velox
that referenced
this pull request
Mar 17, 2023
liujiayi771
added a commit
to liujiayi771/velox
that referenced
this pull request
Mar 18, 2023
liujiayi771
pushed a commit
to liujiayi771/velox
that referenced
this pull request
Apr 1, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
rui-mo
added a commit
to rui-mo/velox
that referenced
this pull request
Apr 23, 2023
…cebookincubator#7) Always compile Substrait (facebookincubator#8)
facebook-github-bot
pushed a commit
that referenced
this pull request
Aug 23, 2024
Summary: The default algorithm used is MD5. However, MD5 is not supported with fips and can cause a SIGSEGV. Set CRC32 instead which is a standard for checksum computation and is not restricted by fips. crc32 is also faster than md5. Internally at IBM, we hit the following SIGSEGV ``` 0x0000000000000000 in ?? () Missing separate debuginfos, use: dnf debuginfo-install openssl-fips-provider-3.0.7-2.el9.x86_64 xz-libs-5.2.5-8.el9_0.x86_64 (gdb) bt #0 0x0000000000000000 in ?? () #1 0x0000000004e5f89b in Aws::Utils::Crypto::MD5OpenSSLImpl::Calculate(std::istream&) () #2 0x0000000004efd298 in Aws::Utils::Crypto::MD5::Calculate(std::istream&) () #3 0x0000000004ef71b9 in Aws::Utils::HashingUtils::CalculateMD5(std::iostream&) () #4 0x0000000004e8ebe8 in Aws::Client::AWSClient::AddChecksumToRequest(std::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&) const () #5 0x0000000004e8ed15 in Aws::Client::AWSClient::BuildHttpRequest(Aws::AmazonWebServiceRequest const&, std::shared_ptr<Aws::Http::HttpRequest> const&) const () #6 0x0000000004e977f9 in Aws::Client::AWSClient::AttemptOneRequest(std::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&, char const*, char const*, char const*) const () #7 0x0000000004e9e1c0 in Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const () #8 0x0000000004ea15e8 in Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const () #9 0x0000000004ea1f70 in Aws::Client::AWSXMLClient::MakeRequest(Aws::AmazonWebServiceRequest const&, Aws::Endpoint::AWSEndpoint const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const () #10 0x0000000004de0933 in Aws::S3::S3Client::UploadPart(Aws::S3::Model::UploadPartRequest const&) const::{lambda()https://github.com/facebookincubator/velox/issues/1}::operator()() const () #11 0x0000000004de0b8c in std::_Function_handler<Aws::Utils::Outcome<Aws::S3::Model::UploadPartResult, Aws::S3::S3Error> (), Aws::S3::S3Client::UploadPart(Aws::S3::Model::UploadPartRequest const&) const::{lambda()https://github.com/facebookincubator/velox/issues/1}>::_M_invoke(std::_Any_data const&) () #12 0x0000000004e19317 in Aws::Utils::Outcome<Aws::S3::Model::UploadPartResult, Aws::S3::S3Error> smithy::components::tracing::TracingUtils::MakeCallWithTiming<Aws::Utils::Outcome<Aws::S3::Model::UploadPartResult, Aws::S3::S3Error> >(std::function<Aws::Utils::Outcome<Aws::S3::Model::UploadPartResult, Aws::S3::S3Error> ()>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, smithy::components::tracing::Meter const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () #13 0x0000000004d7cdcf in Aws::S3::S3Client::UploadPart(Aws::S3::Model::UploadPartRequest const&) const () #14 0x0000000004ca4aa6 in facebook::velox::filesystems::S3WriteFile::Impl::uploadPart (this=0x7fffec2f09a0, part=..., isLast=true) at /root/velox/velox/connectors/hive/storage_adapters/s3fs/S3FileSystem.cpp:380 ``` Pull Request resolved: #10801 Reviewed By: amitkdutta Differential Revision: D61671574 Pulled By: kgpai fbshipit-source-id: 34c7b777b3fde0659ef74c4fbfd93740fdfa3f7c
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are cases that may require callers to supply their own executor. Make it pluggable through the interface.
Differential Revision: D30193354