Skip to content

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Sep 4, 2023

What changes were proposed in this pull request?

In the PR, I propose to document the recent changes related to the format of the to_char/to_varchar functions:

  1. binary formats added by [SPARK-44983][SQL] Convert binary to string by to_char for the formats: hex, base64, utf-8 #42632
  2. datetime formats introduced by [SPARK-44868][SQL] Convert datetime to string by to_char/to_varchar #42534

Why are the changes needed?

To inform users about recent changes.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

By CI.

Was this patch authored or co-authored using generative AI tooling?

No.

@MaxGekk MaxGekk changed the title [WIP][SQL][DOCS] Describe the binary and datetime formats of to_char/to_varchar [SPARK-45070][SQL][DOCS] Describe the binary and datetime formats of to_char/to_varchar Sep 4, 2023
@MaxGekk MaxGekk marked this pull request as ready for review September 4, 2023 13:20
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

* <li>'base64': a base 64 string.</li>
* <li>'hex': a string in the hexadecimal format.</li>
* <li>'utf-8': the input binary is decoded to UTF-8 string.</li>
* </ul>
Copy link
Member

@dongjoon-hyun dongjoon-hyun Sep 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To @MaxGekk , it seems our Linter CI doesn't allow this kind of human-friendly formatting in the comment.

The scalafmt check failed on connector/connect at following occurrences:

Requires formatting: functions.scala

Before submitting your change, please make sure to format your code using the following command:
./build/mvn -Pscala-2.12 scalafmt:format -Dscalafmt.skip=false -Dscalafmt.validateOnly=false -Dscalafmt.changedOnly=false -pl connector/connect/common -pl connector/connect/server -pl connector/connect/client/jvm
Error: Process completed with exit code 1.

The recommended style is something like the following. Please fix the style by run the above command line.

-   *   brackets if the input value is negative.</li> </ul>
-   *   If `e` is a datetime, `format` shall be a valid datetime pattern, see
-   *   <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime Patterns</a>.
-   *   If `e` is a binary, it is converted to a string in one of the formats:
-   *   <ul>
-   *     <li>'base64': a base 64 string.</li>
-   *     <li>'hex': a string in the hexadecimal format.</li>
-   *     <li>'utf-8': the input binary is decoded to UTF-8 string.</li>
-   *   </ul>
+   *   brackets if the input value is negative.</li> </ul> If `e` is a datetime, `format` shall be
+   *   a valid datetime pattern, see <a
+   *   href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime
+   *   Patterns</a>. If `e` is a binary, it is converted to a string in one of the formats: <ul>
+   *   <li>'base64': a base 64 string.</li> <li>'hex': a string in the hexadecimal format.</li>
+   *   <li>'utf-8': the input binary is decoded to UTF-8 string.</li> </ul>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @dongjoon-hyun

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 6, 2023

Merging to master. Thank you, @dongjoon-hyun and @cloud-fan for review.

@MaxGekk MaxGekk closed this in 637f16e Sep 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants