This repository has been archived by the owner on Sep 18, 2023. It is now read-only.
Leverage buffered write in shuffle #647
Labels
enhancement
New feature or request
Comments
zhouyuan
pushed a commit
to zhouyuan/native-sql-engine
that referenced
this issue
Jan 8, 2022
zhouyuan
added a commit
that referenced
this issue
Jan 9, 2022
* Use Hadoop 3.2 as default hadoop.version (#652) * [NSE-661] Add left/right trim in WSCG * [NSE-675] Add instr expression support (#676) * Initial commit * Add the support in wscg * [NSE-674] Add translate expression support (#672) * Initial commit * Add StringTranslate for subquery checking * Code refactor * Change arrow branch for unit test [will revert at last] * Revert "Change arrow branch for unit test [will revert at last]" This reverts commit bf74356. * Port the function to wscg * Change arrow branch for unit test [will revert at last] * Format native code * Fix a bug * Revert "Change arrow branch for unit test [will revert at last]" This reverts commit 3a53fa2. * [NSE-681] Add floor & ceil expression support (#682) * Initial commit * Add ceil expression support * Change arrow branch for unit test [will revert at last] * Revert "Change arrow branch for unit test [will revert at last]" This reverts commit 5fb2f4b. * [NSE-647] Leverage buffered write in shuffle (#648) Closes #647 * [NSE-400] Native Arrow Row to columnar support (#637) * Support ArrowRowToColumnar Optimization * Replace expired code * Add the code to convert recordbatch to columnarBatch * Add unit test on java size * Update the unit tests * Fix the bug when reading decimal value from unsafeRow * Use ArrowRowToColumnarExec instead of RowToArrowColumnarExec * Use clang-format to standardize the CPP code format * enable arrowRowToColumnarExec * Add the metrics for ArrowRowToColumnarExec * Add the metrics for ArrowRowToColumnarExec and unsupport Codegen * Add parameter 'spark.oap.sql.columnar.rowtocolumnar' to control ArrowRowToColumnarExec * Remove useless code * Release arrowbuf after return recordbatch * Fix the processTime metric for ArrowRowToColumnarExec * Refine the code of ArrowRowToColumnar operator * Add more metrics to detect the elapse time of each action * Small fix about allocating buffer for unsafeRow * Remove useless code * Remove useless metrics for ArrowRowToColumnarExec * Fall back to use java RowToColumnarExec when the row is not unsafeRow Type * Fix the bug for decimal format * fix format Co-authored-by: Yuan Zhou <yuan.zhou@intel.com> * fix leakage in rowtocolumn (#683) Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Wei-Ting Chen <weiting.chen@intel.com> Co-authored-by: PHILO-HE <feilong.he@intel.com> Co-authored-by: Hongze Zhang <hongze.zhang@intel.com> Co-authored-by: haojinIntel <hao.jin@intel.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
No description provided.
The text was updated successfully, but these errors were encountered: