Allow XDG_CONFIG_HOME on macOS

[0az]

For many tools, especially those designed for use across Linux/Unix/Mac, it makes sense to use XDG as a common standard. This especially applies to command line tools, for which ~/Library/* breaks inherited convention (and thus breaks interop).

I have previously submitted PRs to resolve this in several applications on macOS, but it would be easier to establish good defaults here instead of sending PRs to patch around it downstream. (This also makes this a blocker for personal usage – at least without vendoring and modifying the module, which isn't ideal.)

Appdirs issue: ActiveState/appdirs#78

[Julian]

I want (some version of) this change personally too, but I think we have to be quite careful -- the Apple guidelines are clear, so I'm:

• +1 on "follow XDG_* if set, even on macOS"
• But -1 for any suggestion of following the XDG-recommended behavior (which says what to do when XDG_ is unset) on macOS
• But +0 on "add a way for libraries using platformdirs to opt-in to that behavior even on macOS", which is sometimes useful -- e.g. ~/Library/Application Support has a space in it, so for libraries that install scripts with shebangs and put them there, they literally won't execute (unless they use the newer ~/Library/ApplicationSupport spaceless symlink), and for those, always defaulting to ~/.local/share, a la XDG_DATA_HOME's default seems potentially reasonable

(Not sure you were suggesting anything about default behavior, but yeah just mentioning my own opinion on all 3 decisions).

[0az]

(Not sure you were suggesting anything about default behavior, but yeah just mentioning my own opinion on all 3 decisions).

For the record, I'm +1, +0, and +1 on those three points.

[gaborbernat]

Apple does not follow the XDG spec, so at best what I've seen is people roughly mapping it, see https://stackoverflow.com/questions/3373948/equivalents-of-xdg-config-home-and-xdg-data-home-on-mac-os-x

If anyone has better links/documentation on what exactly would mean following the XDG spec when the XDG env-vars are set feel free to clarify.

[0az]

If XDG* is set (and is a CLI program), then the program should use XDG. Other cases (GUI programs) are less clear, but if it has an application bundle, then it's less likely that XDG is the answer.

[gaborbernat]

How do you identify if the Python app is CLI or not?

[0az]

I think that the default should assume CLI. At the very least, this needs to be specifiable downstream. Cross platform applications using Python most likely see use in XDG compliant systems as well, and they often use configs editable in Vim/etc. So having them in a well known location across platforms is a compelling concern IMHO.

[gaborbernat]

It gets tricky because platformdirs might be a dependency of one of your libraries, so the caller codebase might not have direct access to it, and could not necessarily tell in what env type is used. We could make it a global state I guess, something like

platformdirs.set_cli_type(False)

So then GUIS could call part of their setup that code, even if it's used by a transient dependency.

[0az]

I'm unsure if it makes sense for a dependent library to use platformdirs. I guess it needs to be supported regardless, though. Actually, what if this is done per key?

[gaborbernat]

I'd personally rather have one global flag. Are we aware of other languages/libraries respecting XDG on macOS as precedence?

[0az]

If you are aware of ones that *don't* support XDG, please let me know so that I can PR it in the future. With any luck, I can just PR platformdirs instead of appdirs. :) But in all seriousness, anything ported directly from Linux/Unix that already uses XDG generally adheres to XDG on Mac. Anything that uses ~/.config follows the half of XDG that I care about (and a lot of stuff uses ~/.config these days, including git). And a concrete example: I did recently PR XDG into the Backblaze SDK, which is used by their CLI and in other applications as a library.

[gaborbernat]

Feel free to open a pull request 👍

[0az]

How should backwards compatibility be handled?

[gaborbernat]

What do you suggest?

[0az]

Pushed my proposal for backwards compat to #35, which is mostly complete (just needs more tests, a bit of bikeshedding, and a rebase).

[layday]

* unless they use the newer `~/Library/ApplicationSupport` spaceless symlink

This symlink is created by iTerm2, it's not a new addition to the OS.

[percurnicus]

Would you accept a PR to extend the XDG_* environment variables to Windows as well as Mac? I've ran into cases on Windows where I would like to override the cache directory using an environment variable. My thought on the implementation would be to
have all systems obey the environment variable like Unix currently does and to revert to the current logic if it is not set.

[gaborbernat]

I personally would be alright with this, however, considering its wide-ranging implications I'd like to get input of some of the downstream users; e.g. pip cc @pradyunsg, pytest @asottile @nicoddemus. Plus the opinion of other maintainers: @RonnyPfannschmidt @ofek @Julian.

[Julian]

I felt icky about it when seeing the email notification last night, but I can't place my finger on why yet :)
I am obviously pro doing it on macOS, so I slept on it personally and couldn't think of a reason Windows users should get left out -- I think to me the justification is that a user who sets this envvar, regardless of platform, likely intends to use it. So yeah +0 for now from me for doing this on all OSes.
I would feel less good however about proactively telling users that this is the "recommended" way to get this behavior on all OSes, because indeed these envvars do not have wide adoption on Windows, and even macOS to some extent, so other software will not work this way -- (if we were to advertise a way to mutate these dirs I would instead prefer we have an explicit PLATFORMDIRS_* set of vars for the fully blessed case) But I could be convinced off of that too and just stick with these if others are for it.

[asottile]

I don't think pytest uses platformdirs/appdirs yet though many of its consumers do

so my opinion is less rooted but I'll give it anyway:

• personally (and I know I'm wrong so no need to tell me!) platform-specific directories confuse me and my users so I tend to just use xdg everywhere (the pre-commit cache for instance on every platform will end up in ~/.cache/pre-commit)
• this makes support and triage much easier since it cuts out the "oh you're on windows, uh look in hmm %APPDATA%\Local, or was it Roaming err..."

that said, I think having some way to configure the directories via an environment variable is a desirably feature, and XDG is fairly well specced to enable such a feature. there's an additional question as to how you handle XDG considering a "default" -- and in that case you'd probably want to ignore XDG's recommended defaults and use the platform-specific defaults. the other edge case is when the xdg variable is present but empty -- which seems to imply a bit more "I want xdg" but typically empty should be treated the same as missing (though a popular "standard" (NO_COLOR) gets this wrong as well)

anyway, I'm not a maintainer here -- nor am I currently a direct consumer -- nor am I a maintainer of a direct consumer -- so feel free to ignore my thoughts on this :) -- I'm going to unsubscribe from notifications and if you need me please @-mention directly

[pradyunsg]

I'm a soft -1 for doing any changes related to this on Windows, and -0 for adopting XDG-when-set on MacOS (as long as there's a way to opt out, or if it's made opt-in). I'd lean toward having pip not enable such support on MacOS.

There's absolutely no reason to follow XDG for Windows; and doing so will only complicate the story for files on Windows. For pip, this will mean additional caveats to the configuration file story on Windows.

I'm not sure I'd want pip to support XDG configuration on MacOS either, since we're already technically out of sync with the Apple's specification for where configuration files should be; in more ways than one. Adding XDG as an additional layer of complexity on that would not be something I'd be enthusiastic about, to put it mildly.

[percurnicus]

(if we were to advertise a way to mutate these dirs I would instead prefer we have an explicit PLATFORMDIRS_* set of vars for the fully blessed case) But I could be convinced off of that too and just stick with these if others are for it.

I would prefer this as well. Using XDG on windows feels wrong and that way we don't run into issues causing unintended behavior for other applications that are using those environment variables.

I'm a soft -1 for doing any changes related to this on Windows

Does that also include adding PLATFORMDIRS_* env vars?

[RonnyPfannschmidt]

Are there any well know / used standards for this on windows?

IMHO the platform specific override mechanism should be used

[Julian]

CC @zooba in case he cares to opine.

[julie777]

I'm a soft -1 for doing any changes related to this on Windows, and -0 for adopting XDG-when-set on MacOS (as long as there's a way to opt out, or if it's made opt-in). I'd lean toward having pip not enable such support on MacOS.

There's absolutely no reason to follow XDG for Windows; and doing so will only complicate the story for files on Windows. For pip, this will mean additional caveats to the configuration file story on Windows.

I'm not sure I'd want pip to support XDG configuration on MacOS either, since we're already technically out of sync with the Apple's specification for where configuration files should be; in more ways than one. Adding XDG as an additional layer of complexity on that would not be something I'd be enthusiastic about, to put it mildly.

I have to disagree with the logic. There is absolutely a reason to support XDG on both MacOS and Windows. It gives developers options. Regular "I don't know anything about computers" users won't care. But, I certainly do.

I share my HOME directory tree across platforms. During development I hate to have to go digging around in AppData to find things. Being able to control that would be very helpful. You may think that Windows has standard directories for everything, but many application don't use them. For years Python only installed in C:\Python.

And along came WSL. Now I can run Linux and Windows at the same time and share files. Being able to have a project directory seen by both platforms is great for development. With XDG dirs I can have python tools install into $HOME/.local/bin or $HOME/.local/bin_win.

Please remember that adding support for XDG dirs doesn't affect anyone that doesn't use them (for MacOS and Windows). On Linux if not set they should default appropriately.

[gaborbernat]

Please remember that adding support for XDG dirs doesn't affect anyone that doesn't use them (for MacOS and Windows).

We hear your points, but please respect the decision of the maintainer team on this topic 👍 Unless the current maintainers come back that they've changed their minds there's nothing else to do.

[julie777]

I understand.

[gaborbernat]

Closing per above.

[gshpychka]

@gaborbernat is the maintainer team also against implementing it on MacOS? If not - might be worth keeping this open?

[gaborbernat]

I think we were 🤔

[GalaxySnail]

What about a opt-in environment variable PLATFORMDIRS_USE_XDG? Poeple who don't have it will not be affected. It will be a quite simple patch:

--- __init__.py.bak     2023-03-25 16:36:36.753498600 +0800
+++ __init__.py 2023-03-25 16:38:20.665189000 +0800
@@ -19,6 +19,9 @@


 def _set_platform_dir_class() -> type[PlatformDirsABC]:
+    if os.getenv("PLATFORMDIRS_USE_XDG"):
+        from platformdirs.unix import Unix as Result
+
     if sys.platform == "win32":
         from platformdirs.windows import Windows as Result
     elif sys.platform == "darwin":

[ink-splatters]

Got surprised I could not find a way to specify PDM config dir on macOS, in order to change it from

~/Library/Application Support/pdm

to much more appropriate

~/.config/pdm

which is effectively XDG_CONFIG_HOME, as per XDG Base Directory Specification

Digging into PDM code led me eventually to this thread, as well as I found your referring some macOS Guidelines:

https://github.com/platformdirs/platformdirs/blob/b866d404c74d3467ab7a16e2be974612b14145b7/src/platformdirs/macos.py#L10-L11

Having config stored at

~/Library/Application Support

is anything but Unix thing. It's also not exactly macOS native thing, as all plist configs should go to

~/Library/Preferences

but I guess it's fine for some cross platform, especially, GUI apps, to use the former.

Because macOS is certified Unix (inspired by BSD at POSIX level), it's not unnatural to expect basic Unix conformance, which is luckily the case (notwithstanding its infamously being known to deviate from all kinds of standards).
It also implies one should be able to live a decent life, without need to know about Library and other macOS-specific stuff (given they enjoy Unix-only workflows).

The concern I can immediately come up with (there might be more) is related to dotfiles management, e.g. using one of tools:

• Home Manager
• stow
• dotdrop
• dotbot
• chezmoi

Besides:

there is unspoken contract for them to reside at

$HOME/.config

or

$HOME/.local/share

which rules out their placement to Application Support. For this to work,
it's expected for CLI apps to have their config dirs configurable, if they opt for Application Support relative path.

With all respect to the maintainers: it's disappointing that the issue was closed, and, from what I see, PRs are not too much welcome. Please kindly correct me if I'm wrong, @gaborbernat.

[gaborbernat]

Because macOS is certified Unix

I do not believe this to be the case to the extent you wish to be.

[justuswilhelm]

I do not believe this to be the case to the extent you wish to be.

I suppose it was meant that macOS is POSIX certified? But at that level we are just splitting hairs and getting nowhere further to solving a major pain point that users are complaining about here.

The commenters in this issue have given plenty of use cases for supporting XDG_CONFIG_HOME on macOS.

Being technically correct about this or that is not the same as "doing the right thing". To me, open source is about respecting user freedom.

[RonnyPfannschmidt]

Sometimes freedom from incorrect implementation is more important than freedom to do broken things out of preference