Hide boilerplate in documentation examples

[alamb]

Is your feature request related to a problem or challenge?

Part of #7013

@ugoa has updated our examples in the following PR to all compile and be tested

• Test all examples from library-user-guide & user-guide docs #14544,

One major problem I have with using # to hide setup code, is that the sphinx won't respect the leading # and still renders the whole block, which leading huge chunk of ugly code blocks.

Describe the solution you'd like

I would like to be able to hide the setup code for the examples, following the rustdoc convention of ignoring lines that begin with #

For example, in our docs here:

datafusion/datafusion/core/src/lib.rs

Lines 59 to 94 in 91c0975

	//! ```rust
	//! # use datafusion::prelude::*;
	//! # use datafusion::error::Result;
	//! # use datafusion::functions_aggregate::expr_fn::min;
	//! # use datafusion::arrow::record_batch::RecordBatch;
	//!
	//! # #[tokio::main]
	//! # async fn main() -> Result<()> {
	//! let ctx = SessionContext::new();
	//!
	//! // create the dataframe
	//! let df = ctx.read_csv("tests/data/example.csv", CsvReadOptions::new()).await?;
	//!
	//! // create a plan
	//! let df = df.filter(col("a").lt_eq(col("b")))?
	//! .aggregate(vec![col("a")], vec![min(col("b"))])?
	//! .limit(0, Some(100))?;
	//!
	//! // execute the plan
	//! let results: Vec<RecordBatch> = df.collect().await?;
	//!
	//! // format the results
	//! let pretty_results = arrow::util::pretty::pretty_format_batches(&results)?
	//! .to_string();
	//!
	//! let expected = vec![
	//! "+---+----------------+",
	//! "| a | min(?table?.b) |",
	//! "+---+----------------+",
	//! "| 1 | 2 |",
	//! "+---+----------------+"
	//! ];
	//!
	//! assert_eq!(pretty_results.trim().lines().collect::<Vec<_>>(), expected);
	//! # Ok(())
	//! # }

Lines that start with # are not rendered in the markdown docs on https://docs.rs/datafusion/latest/datafusion/index.html

Describe alternatives you've considered

I wonder if we can find find some way to add a sphinx plugin or something to avoid rendering the blocks of ugly code.

You can build and test the documentation locally by following the instructions here: https://github.com/apache/datafusion/blob/main/docs/README.md

I normally do

cd docs
./build.sh
open build/html/index.html

Additional context

No response

[alamb]

I think this would be a good first issue as it is super helpful, well specified, and doesn't need deep DataFusion internals expertise

[bhavesh26]

@alamb assign this to me

[alamb]

@alamb assign this to me

Thanks @bhavesh26

Note in the future you can put the comment take and the bot will automatically assign it to you

[ugoa]

Please help review workaround solution in 997504f in #14544

[alamb]

Please help review workaround solution in 997504f in #14544

From my perspective the solution your PR is a real fix @ugoa (not just a workaround)

• Test all examples from library-user-guide & user-guide docs #14544