Consolidate udf examples (#18142)

[cj-zhukov]

Which issue does this PR close?

• part of #Reduce the number of distinct binaries to reduce CI / compile time #18142.

Rationale for this change

This PR is for consolidating all the udf examples into a single example binary. We are agreed on the pattern and we can apply it to the remaining examples

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[cj-zhukov]

High-Level Overview

This PR consolidates all udf (simple, advanced, async) examples into a single example binary.
Previously, each example had its own file, but now they can be executed via subcommands using:

cargo run --example udf -- [adv_udaf|adv_udf|adv_udwf|async_udf|udaf|udf|udtf|udwf]

[xudong963]

Thank you, I checked the original issue and the PR implements the proposal.

Could you please do some verification to see if the PR can reduce the binary size?

[alamb]

Thanks @cj-zhukov

@xudong963 per testing:

on this branch, the example binary is 174M

cargo run --profile=ci --example udf -- adv_udaf`

$ du -h target/ci/examples/udf
174M	target/ci/examples/udf

On main, the equivalent 6 binaries are:

cargo run --profile=ci --example advanced_udaf 
cargo run --profile=ci --example simple_udaf 
cargo run --profile=ci --example advanced_udf 
cargo run --profile=ci --example  simple_udf 
cargo run --profile=ci --example advanced_udwf 
cargo run --profile=ci --example simple_udwf

And they each take 173M

$ du -s -h target/ci/examples/*
173M	target/ci/examples/advanced_udaf
173M	target/ci/examples/advanced_udf
173M	target/ci/examples/advanced_udwf
173M	target/ci/examples/simple_udaf
173M	target/ci/examples/simple_udf
173M	target/ci/examples/simple_udwf

So by my calculations this PR saves 173*6 - 124 = 914MB !

@cj-zhukov -- one thing I noticed is that by consolidating the examples, we are likely no longer actually execute them as part of CI:

datafusion/ci/scripts/rust_example.sh

Lines 31 to 35 in 32d2618

	example_name=`basename $filename ".rs"`
	# Skip tests that rely on external storage and flight
	if [ ! -d $filename ]; then
	cargo run --profile ci --example $example_name
	fi

Could you make a new PR that also runs them all? Maybe something like add an ExampleKind::All type to the examples and then pass in all as an argument 🤔

[cj-zhukov]

@alamb @xudong963 Thanks a lot for the review and helpful feedback!
I’ll continue with consolidating the remaining examples next.

I’m also planning to open a follow-up PR to update part of the CI in rust_example.sh - this was originally pointed out by @Jefffrey in this comment

[martin-g]

LGTM!

[alamb]

@alamb @xudong963 Thanks a lot for the review and helpful feedback! I’ll continue with consolidating the remaining examples next.

I’m also planning to open a follow-up PR to update part of the CI in rust_example.sh - this was originally pointed out by @Jefffrey in this comment

Got it -- thank you. I filed a ticket to track that work:

• Update rust_example.sh to run the examples in the new format #18512