Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for correlated subqueries & fix all related TPC-H benchmark issues #2885
Add support for correlated subqueries & fix all related TPC-H benchmark issues #2885
Changes from all commits
0432ef2
831812b
fed7706
ed0236e
e4286b8
d86861c
22e4561
e71dd5a
5f004e6
927decb
acf44d3
3a59fe5
6353642
4e49f87
caeafb2
129cf4e
e6a7369
5257dc9
94d7519
4c492d5
869b1f3
9d8ad43
51da916
d9c89c9
eac67cd
a9dc5b0
e464491
27880e5
9d04212
cd6d217
3f5c8fe
ae4273d
ca2af5b
3d5dd4d
b87c1a8
990f8b9
5b8263b
ba1afa0
83565a8
cd64684
aaa952e
06a1379
184f0f8
bbbd2a9
7e8622f
d7ef835
c7b28ad
b951b1a
3ca913e
400ae9c
b22c4ac
5276a6e
47d5e42
653c920
dc8b7ab
9f38ec5
bc82fa5
86a50ea
c515248
5bc2bc8
6136619
7babe7c
c87ac6f
84d7e7a
0525ebf
7f9205d
bea3d88
2687f79
5a1b9c2
1819bc2
5e1fbb1
12175ab
4fbec36
52f9f4f
040a170
4c935de
7d676d1
e813823
6339140
514a932
4e60070
3393bc2
d834176
0099418
198793f
0598af5
9c84053
bc252ac
d7658ca
5a774e4
c500912
0374e98
8594377
396543d
0c570d0
9a64518
fdf656a
097c0dd
972e309
aee3c69
a7aa68a
dff21a5
a18cdc7
14eebd5
8b21621
1a1ef22
d02f660
418e6f7
5016c94
6ba276b
3bc8cae
e897007
b73a252
96b934f
f00f784
b177119
63204dc
11f24ac
ddaeafc
054c07e
953b85a
d885631
63bcf4f
4773ec8
220c404
095a41c
d01c86e
3bd5ab1
c2a25da
ee91f24
fcc7a63
3960ca8
0c941b7
69f5102
e341570
8408614
99ca496
50a952f
517d0bd
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think about using
SessionContext::register_csv
here instead? Is there some reason we need to explicitly parse the CSV file and build in memory tables?https://docs.rs/datafusion/10.0.0/datafusion/execution/context/struct.SessionContext.html#method.register_csv
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I started with the TPC-H
.csv
s that were checked in already, and even added some of my own, then started adding data to existing ones, then updated failed tests with new expected results, then realized I'd had the feeling before where I'm going down the road hell that is shared test data. I see it being particularly bad for aggregates (what's the proper expected result for the sum of all sales with parts from the middle east?)I started to break out
csv
s by folder, but that seemed cumbersome, so finally I thought that this function might be a useful tool to keep the data close to the test itself, like this:https://github.com/spaceandtimelabs/arrow-datafusion/blob/563d87d1c6413e611894619c2bc472b396d75c3d/datafusion/core/tests/sql/subqueries.rs#L171
I don't have strong opinions as to implementation, but I would like to avoid sharing one set of data between all integration tests.