Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move Trunc, Cot, Round, iszero functions to datafusion-functions #10000

Merged
merged 8 commits into from
Apr 8, 2024

Conversation

Omega359
Copy link
Contributor

@Omega359 Omega359 commented Apr 8, 2024

Which issue does this PR close?

Closes #9859

Rationale for this change

Function migration for Trunc, cot, round and iszero functions as part of #9285,

What changes are included in this PR?

Code, tests. math/mod.rs had to be redone to account for trunc taking a Vec argument.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions physical-expr Physical Expressions labels Apr 8, 2024
let mut decimal_places = ColumnarValue::Scalar(ScalarValue::Int64(Some(0)));

if args.len() == 2 {
decimal_places = ColumnarValue::Array(args[1].clone());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole ColumnarArray -> ArrayRef -> ColumnarArray in this code really should be cleaned up in another PR imho.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I agree.

These are some of the oldest functions in DataFusion so were written at a very different time. We have much better patterns now

(floor, num, "nearest integer less than or equal to argument"),
(pi, , "Returns an approximate value of π")
);
/// Return a list of all functions in this package
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to replace the export_functions macro usage here because it doesn't handle Vec args such as is used by trunc function.

@Omega359 Omega359 marked this pull request as ready for review April 8, 2024 17:42
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @Omega359 .

There was some pretty gnarly merge conflicts to handle due to #9983

I did this locally as part of my review and made a PR to your branch Omega359#2 in case you would ilke to use that

Thanks again

let mut decimal_places = ColumnarValue::Scalar(ScalarValue::Int64(Some(0)));

if args.len() == 2 {
decimal_places = ColumnarValue::Array(args[1].clone());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I agree.

These are some of the oldest functions in DataFusion so were written at a very different time. We have much better patterns now

@@ -555,12 +555,12 @@ enum ScalarFunction {
Log = 11;
// 12 was Log10
// 13 was Log2
Round = 14;
// 14 was Round
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow -- this list is looking pretty small now 🙏

@Omega359
Copy link
Contributor Author

Omega359 commented Apr 8, 2024

There was some pretty gnarly merge conflicts to handle due to #9983

I did this locally as part of my review and made a PR to your branch Omega359#2 in case you would ilke to use that

Yeah, I was expecting that depending on which landed first. I'll check out your merge tonight - thanks!

@alamb
Copy link
Contributor

alamb commented Apr 8, 2024

Check config failure seems unrelated: https://github.com/apache/arrow-datafusion/actions/runs/8606494312/job/23585168125?pr=10000

/usr/bin/docker exec 6cacef2be1f3e8731bdd8120b9ceb79bfc495030e312805a9aedd199a82efc2b sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: error while loading shared libraries: /lib/x86_64-linux-gnu/libstdc++.so.6: invalid ELF header

I'll try and rerun it when the rest of the tests pass

@Omega359
Copy link
Contributor Author

Omega359 commented Apr 8, 2024

I'll try and rerun it when the rest of the tests pass

Thanks, all the tests I have are passing locally.

@alamb alamb closed this Apr 8, 2024
@alamb alamb reopened this Apr 8, 2024
@alamb
Copy link
Contributor

alamb commented Apr 8, 2024

Reopened restart checks

@alamb
Copy link
Contributor

alamb commented Apr 8, 2024

Just let me know when you are happy with this PR @Omega359 and I'll merge it in

@Omega359
Copy link
Contributor Author

Omega359 commented Apr 8, 2024

I believe this PR is complete and ready to merge @alamb, thx for the review and handling the merge conflict.

@alamb alamb merged commit 78f8ef1 into apache:main Apr 8, 2024
24 checks passed
@alamb
Copy link
Contributor

alamb commented Apr 8, 2024

Thanks again @Omega359

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions physical-expr Physical Expressions sql SQL Planner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

move Trunc, Cot, Round, iszero functions to datafusion-functions crate
2 participants