Use ruff format to ensure UNIX line endings in Python files

[weiji14]

Description of proposed changes

Automatically convert line endings in Python files to \n (UNIX style). Requires using beta version of Ruff formatter in v0.1.2, see https://astral.sh/blog/the-ruff-formatter.

We've actually been using dos2unix to do the line ending checks since #736, but not to do the actual formatting, probably because dos2unix is a little trickier to install on Windows (?). Using ruff though is a little easier since we can install it with pip/conda-forge.

Caveats:

• This only runs on Python files that ruff looks at, and not non-Python files like the .rst, .yml, .md files

Partially addresses #736 (comment)

Reminders

• Run make format and make check to make sure the code follows the style guide.
• Add tests for new features or tests that would have caught the bug that you're fixing.
• Add new public functions/methods/classes to doc/api/index.rst.
• Write detailed docstrings for all functions/methods.
• If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
• If adding new functionality, add an example to docstrings or tutorials.
• Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash commands are:

• /format: automatically format and lint the code
• /test-gmt-dev: run full tests on the latest GMT development version

[weiji14]

/format

[weiji14]

See example commit at 22b194c triggered by #2770 (comment) where the CRLF line-endings were converted to LF endings.

[weiji14]

There are some warnings about tab-indentation (W191) from running ruff format:

ruff format --check pygmt doc/conf.py examples
warning: The following rules may cause conflicts when used with the formatter: 'W191'. To avoid unexpected behavior, we recommend disabling these rules, either by removing them from the `select` or `extend-select` configuration, or adding then to the `ignore` configuration.
warning: The following rules may cause conflicts when used with the formatter: 'W191'. To avoid unexpected behavior, we recommend disabling these rules, either by removing them from the `select` or `extend-select` configuration, or adding then to the `ignore` configuration.

See also upstream issue at astral-sh/ruff#8185, these warnings might go away with astral-sh/ruff#8196 🤞

[seisman]

If I understand it correctly, ruff format will replace black, right? If yes, then I think we should split it into two separate RPs:

1. Use ruff format and remove black
2. Use ruff to ensure UNIX line endings (unless UNIX line ending is the default)

[weiji14]

Ah ok, I was under the impression that ruff format ... was a replacement for ruff check --fix, but apparently not, according to https://docs.astral.sh/ruff/formatter and https://docs.astral.sh/ruff/linter, a formatter is different to a linter. It sounds like we'll need both?

Btw, I could only get the UNIX line formatting to work with ruff format, and not ruff check --fix. I'd maybe add ruff format to this PR (and keep ruff check too), and then deactivate black in a separate PR. Which means we'll have two formatters for a while perhaps.

Edit: It seems like there is some thought towards unifying the linter and formatter in one command - astral-sh/ruff#8232. Gonna mark this as draft first.

[seisman]

Edit: It seems like there is some thought towards unifying the linter and formatter in one command - astral-sh/ruff#8232. Gonna mark this as draft first.

Instead of waiting for the upstream change about linter/formatter, maybe we should go ahead and follow the current ruff documentation and use ruff check/ruff format. We can always update our commands if there are any upstream changes in ruff.

[weiji14]

Edit: It seems like there is some thought towards unifying the linter and formatter in one command - astral-sh/ruff#8232. Gonna mark this as draft first.

Instead of waiting for the upstream change about linter/formatter, maybe we should go ahead and follow the current ruff documentation and use ruff check/ruff format. We can always update our commands if there are any upstream changes in ruff.

Ok, done in 6076a50. Now we run the linter ruff check first, followed by the formatter ruff format. Specifically:

• make format does ruff check --fix $(FORMAT_FILES) && ruff format $(FORMAT_FILES)
• make check does ruff check $(FORMAT_FILES) && ruff format --check $(FORMAT_FILES)

[seisman]

See https://docs.astral.sh/ruff/configuration/.

select should be in the [tool.ruff.lint] section and [tool.ruff.per-file-ignores] should be [tool.ruff.lint.per-file-ignores].

I think it also make sense to sort the section by the following order:

[tool.ruff]

[tool.ruff.lint]

[tool.ruff.lint.per-file-ignores]

[tool.ruff.format]

# tool-specific configurations
[tool.ruff.isort]

[tool.ruff.pycodestyle]

We probably also want to require ruff>=0.1.3.

[weiji14]

Hmm, did they change the TOML config settings recently? I see pycodestyle is now under tool.ruff.lint.pycodestyle, and isort is under tool.ruff.lint.isort. Let me fix these up.

[weiji14]

Done in 8c58089.

[weiji14]

If I understand it correctly, ruff format will replace black, right? If yes, then I think we should split it into two separate RPs:

1. Use ruff format and remove black
2. Use ruff to ensure UNIX line endings (unless UNIX line ending is the default)

Oh, and just on this, we could remove black in a follow-up PR. I think ruff format does some sort of line-ending formatting by default, so doing 1 and then 2 doesn't make sense.

[seisman]

Oh, and just on this, we could remove black in a follow-up PR. I think ruff format does some sort of line-ending formatting by default, so doing 1 and then 2 doesn't make sense.

Sounds good to me. I also removed 'CI' from the PR title, because the changes are not directly related to CI.