Create a benchmark for map subscript. #7026

laithsakka · 2023-10-12T19:56:03Z

Summary:

============================================================================
[...]hmarks/ExpressionBenchmarkBuilder.cpp     relative  time/iter   iters/s
============================================================================
map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>##1               71.48ms     13.99
map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>##2               76.58ms     13.06
map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>##3               85.31ms     11.72
map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>##4              121.56ms      8.23
map_subscript_MAP<INTEGER,INTEGER>##1                      27.19ms     36.78
map_subscript_MAP<INTEGER,INTEGER>##2                      33.10ms     30.21
map_subscript_MAP<INTEGER,INTEGER>##3                      33.47ms     29.88
map_subscript_MAP<INTEGER,INTEGER>##4                      31.70ms     31.55
map_subscript_MAP<VARCHAR,INTEGER>##1                      26.92ms     37.14
map_subscript_MAP<VARCHAR,INTEGER>##2                      36.62ms     27.31
map_subscript_MAP<VARCHAR,INTEGER>##3                      34.19ms     29.24
map_subscript_MAP<VARCHAR,INTEGER>##4                      33.76ms     29.62

Differential Revision: D50237919

netlify · 2023-10-12T19:56:09Z

✅ Deploy Preview for meta-velox canceled.

Name	Link
🔨 Latest commit	`f621618`
🔍 Latest deploy log	https://app.netlify.com/sites/meta-velox/deploys/652873dab554be0008f28dd5

facebook-github-bot · 2023-10-12T19:56:15Z

This pull request was exported from Phabricator. Differential Revision: D50237919

Summary: ``` ============================================================================ [...]hmarks/ExpressionBenchmarkBuilder.cpp relative time/iter iters/s ============================================================================ map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#1 71.48ms 13.99 map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#2 76.58ms 13.06 map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#3 85.31ms 11.72 map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#4 121.56ms 8.23 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#1 27.19ms 36.78 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#2 33.10ms 30.21 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#3 33.47ms 29.88 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#4 31.70ms 31.55 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#1 26.92ms 37.14 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#2 36.62ms 27.31 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#3 34.19ms 29.24 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#4 33.76ms 29.62 ``` Differential Revision: D50237919

facebook-github-bot · 2023-10-12T22:30:38Z

This pull request was exported from Phabricator. Differential Revision: D50237919

Summary: ``` ============================================================================ [...]hmarks/ExpressionBenchmarkBuilder.cpp relative time/iter iters/s ============================================================================ map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#1 71.48ms 13.99 map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#2 76.58ms 13.06 map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#3 85.31ms 11.72 map_subscript_MAP<ARRAY<VARCHAR>,INTEGER>#facebookincubator#4 121.56ms 8.23 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#1 27.19ms 36.78 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#2 33.10ms 30.21 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#3 33.47ms 29.88 map_subscript_MAP<INTEGER,INTEGER>#facebookincubator#4 31.70ms 31.55 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#1 26.92ms 37.14 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#2 36.62ms 27.31 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#3 34.19ms 29.24 map_subscript_MAP<VARCHAR,INTEGER>#facebookincubator#4 33.76ms 29.62 ``` Differential Revision: D50237919

facebook-github-bot · 2023-10-12T22:31:23Z

This pull request was exported from Phabricator. Differential Revision: D50237919

facebook-github-bot · 2023-10-12T22:32:09Z

This pull request was exported from Phabricator. Differential Revision: D50237919

Yuhta · 2023-10-13T14:42:01Z

velox/functions/prestosql/benchmarks/MapSubscriptBenchmark.cpp

+  auto createSet = [&](const TypePtr& mapType) {
+    VectorFuzzer::Options options;
+    options.vectorSize = 1'000;
+    options.containerLength = 20;


Can we make some variability on this? This is the most important parameter in this case IMO

kevinwilfong · 2023-10-13T16:53:26Z

velox/functions/prestosql/benchmarks/MapSubscriptBenchmark.cpp

+    auto makeKeys = [&](int index) {
+      DecodedVector decoded(*columns[index - 1]);
+      auto* map = decoded.base()->as<MapVector>();
+      auto indices = allocateIndices(1000, pool);


everywhere you use 1000 in this lambda, it's because that's the value of options.vectorSize right? could we just use that variable here instead?

kevinwilfong · 2023-10-13T16:56:56Z

velox/functions/prestosql/benchmarks/MapSubscriptBenchmark.cpp

+    VectorFuzzer::Options options;
+    options.vectorSize = 1'000;
+    options.containerLength = 20;
+    options.containerVariableLength = 20;


isn't containerVariableLength a bool? this is probably a typo

kevinwilfong · 2023-10-13T17:01:48Z

velox/functions/prestosql/benchmarks/MapSubscriptBenchmark.cpp

+      auto indices = allocateIndices(1000, pool);
+      auto* mutableIndices = indices->asMutable<vector_size_t>();
+      for (int i = 0; i < 1000; i++) {
+        int keyIndex = folly::Random::rand32() % 20;


This will mostly produce misses right? Is that intended?

The first 20 elements in mapKeys are likely to be in the first few maps (maybe just the first one). And the keys are not very likely to occur in later maps because of the range the random map keys are pulled from.

I may be misreading/misunderstanding the code.

stale · 2024-01-11T18:55:43Z

This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the PR, make sure you've addressed reviewer comments, and rebase on the latest main. Thank you for your contributions!

facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported labels Oct 12, 2023

laithsakka force-pushed the export-D50237919 branch from 2a2eb51 to cd5a7b7 Compare October 12, 2023 22:30

laithsakka force-pushed the export-D50237919 branch from cd5a7b7 to de35346 Compare October 12, 2023 22:31

laithsakka force-pushed the export-D50237919 branch from de35346 to f621618 Compare October 12, 2023 22:31

Yuhta reviewed Oct 13, 2023

View reviewed changes

kevinwilfong reviewed Oct 13, 2023

View reviewed changes

stale bot added the stale label Jan 11, 2024

stale bot closed this Jan 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a benchmark for map subscript. #7026

Create a benchmark for map subscript. #7026

laithsakka commented Oct 12, 2023

netlify bot commented Oct 12, 2023 •

edited

Loading

facebook-github-bot commented Oct 12, 2023

facebook-github-bot commented Oct 12, 2023

facebook-github-bot commented Oct 12, 2023

facebook-github-bot commented Oct 12, 2023

Yuhta Oct 13, 2023

kevinwilfong Oct 13, 2023

kevinwilfong Oct 13, 2023

kevinwilfong Oct 13, 2023

stale bot commented Jan 11, 2024

Create a benchmark for map subscript. #7026

Create a benchmark for map subscript. #7026

Conversation

laithsakka commented Oct 12, 2023

netlify bot commented Oct 12, 2023 • edited Loading

✅ Deploy Preview for meta-velox canceled.

facebook-github-bot commented Oct 12, 2023

facebook-github-bot commented Oct 12, 2023

facebook-github-bot commented Oct 12, 2023

facebook-github-bot commented Oct 12, 2023

Yuhta Oct 13, 2023

Choose a reason for hiding this comment

kevinwilfong Oct 13, 2023

Choose a reason for hiding this comment

kevinwilfong Oct 13, 2023

Choose a reason for hiding this comment

kevinwilfong Oct 13, 2023

Choose a reason for hiding this comment

stale bot commented Jan 11, 2024

netlify bot commented Oct 12, 2023 •

edited

Loading