Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Enable ORC timestamp and decimal predicate push down tests #9075

Closed
2 tasks
thirtiseven opened this issue Aug 18, 2023 · 1 comment · Fixed by #9827
Closed
2 tasks

[FEA] Enable ORC timestamp and decimal predicate push down tests #9075

thirtiseven opened this issue Aug 18, 2023 · 1 comment · Fixed by #9827
Assignees
Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf test Only impacts tests

Comments

@thirtiseven
Copy link
Collaborator

thirtiseven commented Aug 18, 2023

Is your feature request related to a problem? Please describe.
Predicate push down doesn't work for timestamp and decimal types when reading ORC file written by GPU. This is because cuDF ORC writing does not include full Statistics. We should check if PPD works after the metrics are fixed in cuDF.

Describe the solution you'd like

Additional context
#9068

@thirtiseven thirtiseven added feature request New feature or request ? - Needs Triage Need team to review and classify test Only impacts tests labels Aug 18, 2023
@thirtiseven thirtiseven changed the title [FEA] Enable ORC timestamp predicate push down tests [FEA] Enable ORC timestamp and decimal predicate push down tests Aug 22, 2023
@sameerz sameerz added cudf_dependency An issue or PR with this label depends on a new feature in cudf and removed feature request New feature or request ? - Needs Triage Need team to review and classify labels Aug 22, 2023
@thirtiseven
Copy link
Collaborator Author

Test results today:

Run starting. Expected test count is: 9
OrcFilterSuite:
- Support for pushing down filters for boolean types gpu write gpu read
- Support for pushing down filters for boolean types gpu write cpu read
- Support for pushing down filters for boolean types cpu write gpu read
- Support for pushing down filters for decimal types gpu write gpu read
- Support for pushing down filters for decimal types gpu write cpu read
- Support for pushing down filters for decimal types cpu write gpu read
- Support for pushing down filters for timestamp types cpu write gpu read
- Support for pushing down filters for timestamp types gpu write cpu read *** FAILED ***
  0 was less than 10, but 10 was not less than 10 (OrcFilterSuite.scala:37)
- Support for pushing down filters for timestamp types gpu write gpu read *** FAILED ***
  0 was less than 10, but 10 was not less than 10 (OrcFilterSuite.scala:37)
Run completed in 19 seconds, 509 milliseconds.
Total number of tests run: 9
Suites: completed 2, aborted 0
Tests: succeeded 7, failed 2, canceled 0, ignored 0, pending 0
*** 2 TESTS FAILED ***

The decimal tests passed, but the timestamp tests still failed. Will check the generated files and investigate whether cuDF fixed them later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf test Only impacts tests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants