Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update datafusion/tests/sql.rs tests to use assert_batches_eq #1126

Merged

Conversation

matthewmturner
Copy link
Contributor

@matthewmturner matthewmturner commented Oct 15, 2021

Which issue does this PR close?

Related to #743 task 2

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added the datafusion Changes in the datafusion crate label Oct 15, 2021
@houqp
Copy link
Member

houqp commented Oct 15, 2021

optimizers.rs doesn't seem related?

@matthewmturner
Copy link
Contributor Author

matthewmturner commented Oct 15, 2021

optimizers.rs doesn't seem related?

@houqp
Sry, I see now. optimizers.rs wasnt supposed to be added here. It was from another PR. I'm removing.

Comment on lines +1318 to +1324
"+--------------------------+",
"| totimestampmillis(t1.ts) |",
"+--------------------------+",
"| 2009-03-01 00:00:00 |",
"| 2009-03-01 00:01:00 |",
"| 2009-04-01 00:00:00 |",
"+--------------------------+",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb this test seems to be testing for milliseconds but the results dont have that much precision. should the test be updated to get to the appropriate precision? similar question for the microsecond test beneath this one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea of adding some data that actually has millisecond precision sounds like a good idea to me. I think it could also be done as another PR (perhaps file at ticket) as the tests are no worse than before

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @matthewmturner this is looking good / on the right track 👍

Thank you

Comment on lines +1318 to +1324
"+--------------------------+",
"| totimestampmillis(t1.ts) |",
"+--------------------------+",
"| 2009-03-01 00:00:00 |",
"| 2009-03-01 00:01:00 |",
"| 2009-04-01 00:00:00 |",
"+--------------------------+",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea of adding some data that actually has millisecond precision sounds like a good idea to me. I think it could also be done as another PR (perhaps file at ticket) as the tests are no worse than before

@alamb
Copy link
Contributor

alamb commented Oct 25, 2021

This looks like it is getting close... Would you like some help with this one @matthewmturner ?

@matthewmturner
Copy link
Contributor Author

This looks like it is getting close... Would you like some help with this one @matthewmturner ?

@alamb Maybe im missing something but I still see a bunch of assert_eq tests so didnt think i was getting that close yet. Did you have something specific in mind?

@alamb
Copy link
Contributor

alamb commented Oct 25, 2021

@matthewmturner I was thinking that we could polish up the work you have so far and get it committed and then work on the remaining tests as a follow on PR(s).

Rationale for merging sooner rather than later is to minimize potential conflicts (but also so that we are able to keep chipping away at this backlog on master rather than a branch which might go stale / never get entirely completed due to other priorities).

I guess I jealously want to get these changes in so as to take advantage of all your work so far :)

We did one chunk in #760 and this would be a nice chunk

@matthewmturner
Copy link
Contributor Author

@alamb okay makes sense! I will start cleaning it up / getting CI to pass.

@alamb
Copy link
Contributor

alamb commented Oct 26, 2021

❤️

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @matthewmturner -- we are getting closer!

vec!["2020-09-08 12:42:29.190"],
vec!["2020-09-08 11:42:29.190"],
"+---------------------------------+",
"| totimestampmillis(ts_micros.ts) |",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this column name is fascinating -- it seem like it should be to_timestamp_millis(..) but I'll let someone else file a ticket if it is important to them

@alamb alamb merged commit f99e9c8 into apache:master Oct 26, 2021
jgoday pushed a commit to jgoday/arrow-datafusion that referenced this pull request Oct 27, 2021
…#1126)

* First tests updated

* More tests

* More tests

* More tests

* More tests

* Cleanup format

* Cleanup for CI
@houqp houqp added the development-process Related to development process of DataFusion label Nov 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate development-process Related to development process of DataFusion
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants