Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL]Daily Update Velox Version (20231206) #3954

Merged
merged 1 commit into from
Dec 7, 2023
Merged

[VL]Daily Update Velox Version (20231206) #3954

merged 1 commit into from
Dec 7, 2023

Conversation

yma11
Copy link
Contributor

@yma11 yma11 commented Dec 6, 2023

What changes were proposed in this pull request?

Rebase velox with following changes:

commit 075cc92
Author: Jimmy Lu jimmylu@meta.com
Date: Tue Dec 5 15:35:32 2023 -0800

Back out "Add support for decimals in approx_distinct aggregate function" (#7889)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7889

Original commit changeset: 7b9cead380c5

Original Phabricator Diff: D51811791

Aggregate fuzzer is not able to resolve decimal types, back it out to unbreak the test

Reviewed By: mbasmanova

Differential Revision: D51863784

fbshipit-source-id: a822065d7c4b7cc7002bbb3dfb1bd121686da8f0

commit 8a9ae28
Author: yangchuan yangchuan.zy@alibaba-inc.com
Date: Tue Dec 5 15:34:11 2023 -0800

Fix memory leak of HdfsBuilder (#7579)

Summary:
ASAN report memory leak of HdfsBuilder, after checking libhdfs source code, `hdfsBuilderConnect` does not free hdfsBuilder as its doc claimed. https://github.com/apache/hawq/blob/e9d43144f7e947e071bba48871af9da354d177d0/depends/libhdfs3/src/client/Hdfs.cpp#L406

Gluten issue: https://github.com/oap-project/gluten/issues/3713

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7579

Reviewed By: Yuhta

Differential Revision: D51860195

Pulled By: kgpai

fbshipit-source-id: 6beab96f9f83465e25bb5fb7ef370e32ee82c743

commit 14dc44b
Author: Ge Gao gaoge@fb.com
Date: Tue Dec 5 15:19:50 2023 -0800

Change cast(double|real as varchar) to return scientific notation (#7602)

Summary:
Change cast(double as varchar) and cast(real as varchar) to return scientific
notation, when magnitude of the input value is greater than or equal to 10^7,
or less than 10^-3. Today they return standard notation in all cases.

Change to use fmt::format() instead of folly::to() to output standard notation,
for less noisy digits in outputs, when magnitude of the input value is greater
than or equal to 10^-3 and less than 10^7.

Use QueryConfig legacy_cast false to gate this new
behavior. To keep existing behavior, set legacy_cast to true.

The returned scientific notation uses a max of 17 significant decimal digits
for double and a max of 8 decimal digits for real, according to the double and
single precision supported by IEEE 754 floating point, as well as Presto
empirically.

Also add 4 benchmarks for casting double or real to scientific notation or standard
notation. The time of casting for legacy_cast = false, new implementation is
on par with the time for legacy_cast = true, today implementation.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7602

Reviewed By: kagamiori

Differential Revision: D51386874

Pulled By: gggrace14

fbshipit-source-id: 84286074864b8a817d0f3feb97d498b4fe7c6c66

commit a5d816e
Author: rui-mo rui.mo@intel.com
Date: Tue Dec 5 13:40:03 2023 -0800

Fix conversion documentation (#7629)

Summary:
Fix some inaccurate descriptions found in conversion.rst. Document the
limitation of cast from string to real. Add unit tests for cast from float to
bool, cast from string to float, and cast from timestamp with timezone to
timestamp.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7629

Reviewed By: zacw7

Differential Revision: D51837319

Pulled By: kagamiori

fbshipit-source-id: c2b35b2cdf94747858cf76d4ea7fce0d1e2515f5

commit 87c6c8c
Author: duanmeng macduan@tencent.com
Date: Tue Dec 5 12:54:35 2023 -0800

Remove the spilling after memory reservation fails in ensureOutputFits (#7879)

Summary:
Remove the unnecessary spilling in GroupingSet::ensureOutputFits
to keep the implementation simple.
This is a follow-up of https://github.com/facebookincubator/velox/issues/7846

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7879

Reviewed By: bikramSingh91

Differential Revision: D51855352

Pulled By: xiaoxmeng

fbshipit-source-id: d4a5b449853c39716dad706964eb75563ab5cdcb

commit fec3ff0
Author: Jia ke.a.jia@intel.com
Date: Tue Dec 5 11:00:26 2023 -0800

Add BaseVector::setType API (#7783)

Summary:
Sometimes it is useful to be able to change the logical type of a vector without making a copy.

The new BaseVector::setType API allows to change vector type to a compatible logical type, e.g.
change ROW("a" BIGINT) to ROW("b" BIGINT).

For example, in https://github.com/facebookincubator/velox/pull/6074

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7783

Reviewed By: xiaoxmeng

Differential Revision: D51847700

Pulled By: mbasmanova

fbshipit-source-id: bb5b30d3372239196b1f6f1dbb1dee9226bf306a

commit 928f378
Author: Jimmy Lu jimmylu@meta.com
Date: Tue Dec 5 10:43:58 2023 -0800

Copy input in json_parse to avoid ASAN error (#7858)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7858

`value` is put on offset 32 of the stack frame, and the total frame size is 88.  In case `value` is inlined, and SIMD register is 64 bytes (e.g. AVX512), simdjson is reading the memory from offset 36 to 100, which exceeds the frame boundary.

This caused some ASAN error, but no issue in production, because the padding data is not really used or changed.

Fix it by asking `simdjson::dom::parser::parse` to copy the input to internal padded memory owned by parser.  There is some minor performance loss but is probably not noticeable at query level.

Reviewed By: mbasmanova

Differential Revision: D51810542

fbshipit-source-id: cf2145e6fd0d0cb08a53a1bb1b9a2bdb0dfe2892

commit ce7ce23
Author: Orri Erling oerling@fb.com
Date: Tue Dec 5 10:34:56 2023 -0800

Make ByteStream::size() consistent across use cases (#7520)

Summary:
ByteStream::size is expected to have the semantics of the length of std::stringstream. It is the offset of the first unwritten byte from the start. This does not move backward when seeking to mid stream and writing, for example.  been written.

Adds a test to verify proper function when randomly overwriting a ByteStream backed by HashStringAllocator multipart entries.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7520

Reviewed By: xiaoxmeng

Differential Revision: D51270817

Pulled By: oerling

fbshipit-source-id: 7b772852663b933b600c1d52b400a37ea9429edc

commit 7f82f3e
Author: Open Source Bot generatedunixname499836121@fb.com
Date: Tue Dec 5 09:31:51 2023 -0800

Updating submodules

Summary:
GitHub commits:

https://github.com/facebook/cachelib/commit/6b792ab4dc4351f2e84ddec216c5af8cc3ef1325
https://github.com/facebook/fb303/commit/9e9d80d3bca5c06d0ad3fffade9276df8b5709fd
https://github.com/facebook/fbthrift/commit/b8cdb5a9408c2539321ce62c2ac0b063633af213
https://github.com/facebook/folly/commit/c9b275e8ac978251db847818e5b8921f8649086f
https://github.com/facebook/litho/commit/883d011a777492643c5cba8bc232769e38c0ebcd
https://github.com/facebook/mcrouter/commit/de94b8647bb5404971403e657593cc84ff4fbd78
https://github.com/facebook/mvfst/commit/20376c1b0e15a2fbb1b8d5dbd4d56d08e6eefbec
https://github.com/facebook/ocamlrep/commit/308d1e30778276e8b13e7ee7855d5225a498582b
https://github.com/facebook/proxygen/commit/262523a7ffdc53adbff6ec60fcb215ab074168ed
https://github.com/facebook/rocksdb/commit/2045fe4693dd8957796ccce3a4abbf3f689226b3
https://github.com/facebook/wangle/commit/26a4ebed5b876565e861b0412817c8f08e8f4322
https://github.com/facebook/watchman/commit/0fd5c51ccb0657ac68ce7c0ab5c8581464f14606
https://github.com/facebookexperimental/edencommon/commit/f0eab4143f03a1e6771a68bf0180812d6f78785d
https://github.com/facebookexperimental/rust-shed/commit/6f916d7421023392d32b05fce419f2a4f3f5022c
https://github.com/facebookincubator/fizz/commit/661b953302ee2c4b3b368dcd9631328dae090466
https://github.com/facebookincubator/katran/commit/19eb04d184a6f3e7b7c6ecf99d50112ffc912d55
https://github.com/facebookincubator/velox/commit/975ca3ac942472d6312b54280d8543a7b870d6b0
https://github.com/facebookresearch/vrs/commit/798e3b33d8a5ba250d448d2a94880b9595e5f763
https://github.com/pytorch/fbgemm/commit/dbc3157bf256f1339b3fa1fef2be89ac4078be0e
https://github.com/pytorch/kineto/commit/e8a37135a338ec03126f49d8da554544d1a164be

Reviewed By: jurajh-fb

fbshipit-source-id: 50be4041086b9771484e8d3a3a6020fd59df66a9

commit 59b56cd
Author: Karteekmurthys karteek@ahana.io
Date: Tue Dec 5 09:00:18 2023 -0800

Add support for decimals in approx_distinct aggregate function (#7794)

Summary:
This PR adds support for approx_distinct function on decimal types. Resolves https://github.com/prestodb/presto/issues/21331.

This function is needed to run ANALYZE sql command in Presto.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7794

Reviewed By: mbasmanova

Differential Revision: D51811791

Pulled By: Yuhta

fbshipit-source-id: 7b9cead380c56de18e2b0fb80d91fd7c0bc425b5

commit 9830814
Author: Christian Zentgraf kitgocz@gmail.com
Date: Tue Dec 5 08:58:54 2023 -0800

Fix writing dictionary data to parquet (#7025)

Summary:
In a chunked approach the partitioned data is sliced into
dictionary vectors where the partition vectors are shared
across chunks. A previous chunk row position is changed and
not applicable to the prior chunk when the dictionary content
is resolved. This results in incorrect data being written
to the partitioned files.

In addition, Arrow does not support NULL values in a
dictionary and throws an exception.
The exception is
NotImplemented: Writing DictionaryArray with null encoded in dictionary type not yet supported
See https://github.com/apache/arrow/blob/73589ddd60e4cbcd860102871692541989ea38c6/cpp/src/parquet/arrow/path_internal.cc#L752

To solve both issues, the dictionary vector representing
the partitioning is flattened into a FlatVector.
As a result the data is copied to persist it
across chunks.

To make the constant vectors in the TableWriterTest work
they are also flattened into flat vectors with the same
logic the dictionary vectors are.

The ArrowBridge has a new optional option structure to
indicate if dictionary and constant vectors should be flattened.

Resolves https://github.com/facebookincubator/velox/issues/5560

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7025

Reviewed By: mbasmanova

Differential Revision: D51760838

Pulled By: Yuhta

fbshipit-source-id: 1ef7d57af199ea0e0089200a06477759ac31a90a

commit 975ca3a
Author: Masha Basmanova mbasmanova@fb.com
Date: Tue Dec 5 03:29:20 2023 -0800

Add support for lambda functions to ExpressionFuzzer (#7834)

Summary:
The tricky part of supporting lambda functions is finding a way to generate
lambdas given both input types and the result type, i.e. generate
LambdaTypedExpr to match given FunctionType.  This implementation is very
basic. It looks for a matching function signature or signature template and if
none found uses a lambda expression that returns constant value.

For example, lambda for filter(array, function(T) -> boolean) can be
x -> is_null(x) or x -> cast(x as boolean) or x -> true.

This very simple implementation is quite effective. It helped discover a number
of bugs: https://github.com/facebookincubator/velox/issues/7821 #7824 https://github.com/facebookincubator/velox/issues/7828 #7829 https://github.com/facebookincubator/velox/issues/7831 #7851 https://github.com/facebookincubator/velox/issues/7861

This new logic cannot generate valid lambda for 'comparator' argument of
array_sort. Hence, added support for skipping individual function signatures
and used it to exclude this particular signature for array_sort.

Ran the Fuzzer multiple times for 10 minutes each and collected custom stats on
the lambda functions tested and lambda expressions generated. 20 functions
were tested using 154 different lambda expressions. More details in
https://gist.github.com/mbasmanova/1530e3cdfe2f6f4fef35a03f07ee4a39

```
velox_expression_fuzzer_test --logtostderr --velox_fuzzer_enable_complex_types --enable_variadic_signatures --lazy_vector_generation_ratio 0.2 --velox_fuzzer_enable_column_reuse --velox_fuzzer_enable_expression_reuse --max_expression_trees_per_step 2 --retry_with_try --enable_dereference --duration_sec 1200
```

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7834

Test Plan:
Imported from GitHub, without a `Test Plan:` line.

```
$ buck2 run stylus/xstream/query/functions/test:fuzzer_test

$ buck2 run stylus/xstream/query/functions/test:f3_fuzzer_test
```

Reviewed By: xiaoxmeng

Differential Revision: D51756700

Pulled By: mbasmanova

fbshipit-source-id: 4a60fd29e1e670f905e3dc8b5a2180e4440796e2

commit 156abe3
Author: xiaoxmeng xiaoxmeng@fb.com
Date: Tue Dec 5 00:39:43 2023 -0800

Document stats counters and stats macros rename (#7803)

Summary:
Add velox metrics counter documentation and code refactor:
rename stats to metric and update the macros for metric definition
and record:
DEFINE_METRIC/DEFINE_HISTOGRAM_METRICS: define metrics
RECORD_METRIC_VALUE/RECORD_HISTOGRAM_METRIC_VALUE: record metrics

The followup will migrate Prestissimo to use new apis and deprecate
the old ones.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7803

Reviewed By: mbasmanova

Differential Revision: D51696300

Pulled By: xiaoxmeng

fbshipit-source-id: 355c98d3a131cc74169cbee315320ef74c8aa1f7

commit 0b3407e
Author: xiaoxmeng xiaoxmeng@fb.com
Date: Mon Dec 4 23:06:51 2023 -0800

Remove the spilling after memory reservation fails (#7846)

Summary:
With memory arbitration enabled, we shall never need to spill after the memory
reservation fails. The memory arbitrator shall always trigger spilling if memory
reservation needs to grow the pool's capacity. Remove the unnecessary spilling
to keep the implementation simple.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7846

Reviewed By: oerling

Differential Revision: D51787778

Pulled By: xiaoxmeng

fbshipit-source-id: b24c0958de388d9945df4b5da49e88206c3d6d95

commit d5ddb45
Author: Masha Basmanova mbasmanova@fb.com
Date: Mon Dec 4 21:14:58 2023 -0800

Fix crash in FlatVector::hashAll (#7871)

Summary:
FlatVector::hashAll used to crash if nullCount metadata was missing or non-zero, but nulls was buffer was not present.

Fixes https://github.com/facebookincubator/velox/issues/7868

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7871

Reviewed By: xiaoxmeng, darxsys

Differential Revision: D51832284

Pulled By: mbasmanova

fbshipit-source-id: e1dbd22be9c94396ca4c96599db3940415b75c9e

commit 543a460
Author: Wei He weihe@meta.com
Date: Mon Dec 4 21:13:49 2023 -0800

Allow UNKNOWN key type in map_keys and map_values (#7702)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7702

This diff fixes https://github.com/facebookincubator/velox/issues/7489.

Reviewed By: kevinwilfong

Differential Revision: D51534119

fbshipit-source-id: f1709ff7dcb6a4aaa2d850e74914b2bbff722205

commit 261d0b0
Author: Pedro Eugenio Rocha Pedreira pedroerp@meta.com
Date: Mon Dec 4 18:14:16 2023 -0800

Support TimestampWithTimezons in format_datetime() (#7841)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7841

Support TimestampWithTimezons in format_datetime.

Reviewed By: amitkdutta, zacw7

Differential Revision: D51776777

fbshipit-source-id: 82fa7217e0e3789fd8cd2838ec2bc5f73fb11bc8

commit b9367aa
Author: Laith Sakka lsakka@meta.com
Date: Mon Dec 4 18:01:07 2023 -0800

Back out "make skipFunctions, onlyFunctions, and specialForms as ExpressionFuzzer options."

Summary:
Dont have time to investigate will backout and now and fix later.
Original commit changeset: c4bb6b830dc0

Original Phabricator Diff: D51408535

Reviewed By: kevinwilfong

Differential Revision: D51831341

fbshipit-source-id: fcbb80894cc5aecc189b982e9c572bcec74c2984

commit a5978ae
Author: Masha Basmanova mbasmanova@meta.com
Date: Mon Dec 4 18:00:31 2023 -0800

Remove BaseVector::clearIndices API (#7857)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7857

clearIndices API was always used together with resizeIndices API.

Fold the logic of clearIndices into resizeIndices to reduce API surface.

Reviewed By: laithsakka

Differential Revision: D51810625

fbshipit-source-id: 2fe8a2b6a887b9b007c13c74577a8d3b8b81b542

commit 444c5e3
Author: Masha Basmanova mbasmanova@fb.com
Date: Mon Dec 4 16:54:37 2023 -0800

Refactor ExpressionFuzzer to extract logic for generating random integer (#7850)

Summary:
Refactor `boost::random::uniform_int_distribution<uint32_t>(min, max)(rng_)` as `rand32(min, max)`.

Use VectorFuzzer::coinToss when appropriate.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7850

Reviewed By: xiaoxmeng

Differential Revision: D51805835

Pulled By: mbasmanova

fbshipit-source-id: 9b41deda5b5d28e71df51ae7ba5af476d8a9bd40

commit 850046b
Author: Krishna Pai kpai@fb.com
Date: Mon Dec 4 16:06:01 2023 -0800

Ensure signature bias'd functions always run and also surface errors only for Presto. (#7619)

Summary:
* We move the signature check error code to end of the function signature run for presto.
* To do the above, we make changes to the signature.py script so it always writes output to the bias file.
* Finally we ignore the function signature error output for spark , but still run the fuzzer for it when possible.

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7619

Reviewed By: bikramSingh91

Differential Revision: D51417742

Pulled By: kgpai

fbshipit-source-id: cd676cf95230502ba19589bfebb356d90ac2e77d

commit add8303
Author: Masha Basmanova mbasmanova@fb.com
Date: Mon Dec 4 15:26:38 2023 -0800

Fix fast path in map Presto function for input arrays with nulls (#7862)

Summary:
Fast path in the "map" Presto function could produce invalid MapVector if input
arrays had null rows with out-of-bounds  offsets/sizes.

Fixes https://github.com/facebookincubator/velox/issues/7861

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7862

Reviewed By: xiaoxmeng

Differential Revision: D51814196

Pulled By: mbasmanova

fbshipit-source-id: 791bfa01e4e34c164f83117c23c34f286b3ff881

commit e4b9def
Author: Laith Sakka lsakka@meta.com
Date: Mon Dec 4 15:23:55 2023 -0800

make skipFunctions, onlyFunctions, and specialForms as ExpressionFuzzer options. (#7615)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7615

More refactoring for the expression fuzzer.

1) Push the skipFunctions, onlyFunctions, and specialForms as ExpressionFuzzer options.
2) FuzzerRunner: just a tool that wrap ExpressionFuzzerVerifier into a unit test.
3) Move the comment from FuzzerRunner class to ExpressionFuzzerVerifier since it describes the later.

Next diff :
4) Move all the flags from ExpressionFuzzerVerifier to FuzzerRunner and pass them through ExpressionFuzzerVerifier::Options .

1. spark fuzzer used to only support and, or not it uses all of them "and,or,cast,coalesce,if,switch".

Reviewed By: kevinwilfong

Differential Revision: D51408535

fbshipit-source-id: c4bb6b830dc06db849b453741d67f3903804f4d0

commit 681c2fc
Author: Masha Basmanova mbasmanova@meta.com
Date: Mon Dec 4 14:09:15 2023 -0800

Fix spillSortTime for tiny queries (#7854)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7854

Fixes flaky AggregationTest.spillingForAggrsWithSorting which used to fail
asserting spillSortTime > 0. In release build, sometimes the test runs so fast
that spillSortTime is measured as zero. Report at least 1 in these cases.

Reviewed By: xiaoxmeng

Differential Revision: D51807666

fbshipit-source-id: 2f43e7d81a1f5afabd1db8c5e13a293cf53efdd3

commit c0fadb9
Author: Amit Dutta adutta@meta.com
Date: Mon Dec 4 13:43:14 2023 -0800

Make return type void during aggregate registration. (#7864)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7864

Aggregate function registration does not utilize return type. Some registrations are done with void and some are done with return type - where return type is unused. Making all of them void, similar to what is done in  https://github.com/facebookincubator/velox/pull/7689

Reviewed By: mbasmanova

Differential Revision: D51593723

fbshipit-source-id: 088e25e9c8547cd4bf6260621ced4e07d5dfb7aa

commit 4574284
Author: Harvey Hunt harveyhunt@meta.com
Date: Mon Dec 4 12:52:00 2023 -0800

getdeps: Don't use system deps on CentOS for gflags and glog

Summary:
Building projects using `--allow-system-packages` on CentOS for
projects that rely on `gflags` fails. I'm not completely sure why this is the
case and myself and kmancini have spent some time trying to investigate this.

Fix these builds by disabling the system packages on CentOS.

Reviewed By: chadaustin

Differential Revision: D51813855

fbshipit-source-id: aae48e2723fea94f088af02eace831af9e988c7c

commit e6240dc
Author: Kevin Wilfong kevinwilfong@meta.com
Date: Mon Dec 4 11:59:58 2023 -0800

Mark TryExprTest.errorRestoringContext DEBUG_ONLY (#7863)

Summary:
Pull Request resolved: https://github.com/facebookincubator/velox/pull/7863

TryExprTest.errorRestoringContext uses ScopedTestValue which is only supported in debug mode, causing
the test to fail in release mode.

Reviewed By: mbasmanova

Differential Revision: D51814934

fbshipit-source-id: bb3d2fa0bcc8536703e202097e0193a1d1cc4534

commit 6fa5530
Author: Masha Basmanova mbasmanova@fb.com
Date: Mon Dec 4 10:50:57 2023 -0800

Fix find_first for empty arrays (#7852)

Summary:
find_first Presto lambda function used to fail if applied to a subset of rows
 in ArrayVector with only null or empty arrays.

Fixes https://github.com/facebookincubator/velox/issues/7851

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7852

Reviewed By: xiaoxmeng

Differential Revision: D51806646

Pulled By: mbasmanova

fbshipit-source-id: 3a61847b88233e8720652254d65925169c85d7d4

commit 24e58e7
Author: zhli zhli@microsoft.com
Date: Mon Dec 4 07:37:36 2023 -0800

Fix signatures of Spark comparison functions (#7805)

Summary:
Fixes: https://github.com/facebookincubator/velox/issues/7785

Pull Request resolved: https://github.com/facebookincubator/velox/pull/7805

Reviewed By: Yuhta

Differential Revision: D51807439

Pulled By: mbasmanova

fbshipit-source-id: 61a356fe00de025090a5feab9304ca4d973c5cd8

commit 023630d
Author: Open Source Bot generatedunixname499836121@fb.com
Date: Sun Dec 3 09:32:38 2023 -0800

Updating submodules

Summary:
GitHub commits:

https://github.com/facebook/cachelib/commit/e8abfe7809dbd499513f1958c608b5bc4c066290
https://github.com/facebook/fb303/commit/283b7842334cc8b889073375460bca017bf4392e
https://github.com/facebook/fbthrift/commit/c383b5e113b0b74cb768d21bd54d51d8e1a5e964
https://github.com/facebook/folly/commit/f67559532c274fee8a84ab39d5711b422dc963a7
https://github.com/facebook/mcrouter/commit/9483e7926641ef1c8aa6164dd3cbdfe366ef4b4c
https://github.com/facebook/mvfst/commit/ed0b502692f2bc0b12893ea61f747a7cc90416d6
https://github.com/facebook/ocamlrep/commit/82206d449a7f1607d4d42d34b0d1828b57ade418
https://github.com/facebook/proxygen/commit/3ca5204a00a9202d8b1613f00f7cf3650ac679a8
https://github.com/facebook/squangle/commit/96966627ca69f0649c6b0c114fbe578172e5800b
https://github.com/facebook/wangle/commit/58fe85f39907a7d50b1fe5c36a08f04ac66a1160
https://github.com/facebook/watchman/commit/0dd966c1acd578281ad6e3b7774d3826c11d0d4b
https://github.com/facebookexperimental/edencommon/commit/4ac3852a69a4e7aa7f049942d696c2c4830f1d8d
https://github.com/facebookexperimental/rust-shed/commit/8581b27d67dd69cbcec62cbef22db75e67ad77a5
https://github.com/facebookincubator/fizz/commit/5d8e4c8b7adeb3a0eaf3bc84a56a670b7e0c9138
https://github.com/facebookincubator/katran/commit/914bb446a2f233723d4c8f37f8c483df51cb56fc
https://github.com/facebookincubator/velox/commit/dd2703d6c3156b9ab0fd9d3e1e6d8423a58bb8ed

Reviewed By: zpao

fbshipit-source-id: 2519ac20dcafec69569a4dc4a36f859d46a0b04d

How was this patch tested?

CI/CD

Copy link

github-actions bot commented Dec 6, 2023

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

@yma11 yma11 force-pushed the 12-6 branch 2 times, most recently from 24e0e82 to 8c5dd23 Compare December 7, 2023 05:36
Copy link
Contributor

@JkSelf JkSelf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

@JkSelf JkSelf merged commit 10afed2 into apache:main Dec 7, 2023
13 of 15 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_3954_time.csv log/native_master_12_06_2023_d6b1e9138_time.csv difference percentage
q1 32.32 33.13 0.811 102.51%
q2 25.44 25.88 0.439 101.73%
q3 38.46 36.12 -2.345 93.90%
q4 36.26 38.73 2.468 106.81%
q5 70.47 72.19 1.717 102.44%
q6 5.24 7.01 1.777 133.93%
q7 83.95 84.85 0.899 101.07%
q8 87.66 87.08 -0.588 99.33%
q9 127.96 127.07 -0.893 99.30%
q10 43.51 44.57 1.067 102.45%
q11 20.44 20.26 -0.183 99.10%
q12 27.50 21.15 -6.355 76.89%
q13 45.70 48.39 2.696 105.90%
q14 17.27 18.86 1.594 109.23%
q15 29.34 27.18 -2.160 92.64%
q16 15.74 15.92 0.189 101.20%
q17 102.05 102.54 0.492 100.48%
q18 149.44 152.02 2.586 101.73%
q19 12.97 13.90 0.928 107.16%
q20 27.85 27.91 0.054 100.19%
q21 221.79 223.84 2.048 100.92%
q22 13.36 13.24 -0.113 99.15%
total 1234.71 1241.84 7.128 100.58%

@yma11 yma11 deleted the 12-6 branch December 11, 2023 05:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants