Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a More Correct (TM) directory for local data #148

Merged
merged 0 commits into from
Aug 12, 2014

Conversation

emberian
Copy link
Member

@emberian emberian commented Jul 8, 2014

No description provided.

@emberian
Copy link
Member Author

emberian commented Jul 8, 2014

This conforms with the XDG base directory specification. It's possible we want to do something else on OS X, which is more idiomatic?

@tomjakubowski
Copy link
Contributor

(pulled from an orphaned comment)

The equivalent of $XDG_DATA_HOME on OS X is ~/Library. Not many CLI applications conform to this, though, tending to just dump stuff in dotfiles/directories directly under ~.

@o11c
Copy link
Contributor

o11c commented Jul 8, 2014

For Windows it would be CSIDL_LOCAL_APPDATA

@alexcrichton
Copy link
Member

Can you elaborate on the reasons for this change? Other package managers like npm and bundler have files in my home directory, so there's certainly precedent for this.

@emberian
Copy link
Member Author

emberian commented Jul 9, 2014

On Linux at least, storing data directly in the home directory and not in an XDG location is considered poor form. There is certainly precedent for it, but I'd argue that every application that does it is wrong, not just according to the spec, but using the basic namespacing argument.

@o11c
Copy link
Contributor

o11c commented Jul 9, 2014

@alexcrichton The existence of badly-behaved programs is no justification to do the same. It is incorrect for programs to assume that they are important enough for a new top-level directory in HOME. That would be like every program installing itself into /rust, /cargo, etc.

XDG specifies the semantics of (by default) ~/.local/share, ~/.config/, and ~/.cache, which is quite a useful improvement over figuring out "how important is this junk".

Other subdirectories such as ~/.local/lib/ and ~/.local/bin/ are not part of XDG yet (probably just because no one has submitted a patch), but it is generally assumed that ~/.local/ is a sort of parallel to the FHS /usr/local/ hierarchy; in particular, it is often expected that ./configure --prefix=$HOME/.local will DTRT, though note that ~/.config is sort of similar to /etc/ and ~/.local/share is writable (taking on some of the characteristics of /var/)

The only actual standard that I'm aware of is PEP 370, which may be useful to determine the best locations on non-Linux platforms.

Given that there are some standards, it is better to follow them than to make something up from scratch, and additionally work on finishing up. (Particularly, some of the open questions from the last Fedora flamewar on the subject would be resolved by making XDG_BIN_HOME a thing, so that it can be overridden instead of being hard-coded)

@o11c
Copy link
Contributor

o11c commented Jul 9, 2014

Related: rust-lang/rust#12725 and somewhat rust-lang/rust#5219 and rust-lang/rust#6425

@emberian
Copy link
Member Author

emberian commented Jul 9, 2014

Updated to handle Windows and OSX correctly, with links to relevant documentation.

@emberian
Copy link
Member Author

emberian commented Jul 9, 2014

The OSX travis builder seems quite broken -- it can't find rustc!

@jhasse
Copy link

jhasse commented Jul 12, 2014

I think it should be XDG_CONFIG_HOME instead of XDG_DATA_HOME. If I understood it correctly XDG_DATA_HOME represents /usr/local/share which contains read-only data a program might need to run. As cargo checks out git repositories at runtime I would classify those files as "config" data suited for ~/.config/cargo.

@o11c
Copy link
Contributor

o11c commented Jul 13, 2014

XDG_DATA_HOME represents /usr/local/share which contains read-only data

Close, but wrong.

From the spec: There is a single base directory relative to which user-specific data files should be written. This directory is defined by the environment variable $XDG_DATA_HOME.

(Note that in order to implement cargo install it will be necessary to exceed the spec by writing to ~/.local/lib and ~/.local/bin, but since those are a de facto standard, there shouldn't be a problem).

I've quickly written out a rust-xdg library, but I'm thinking about 1. renaming it to reflect its hopefully-cross-platform nature soon, 2. changing the API so that programs can only access one application-specific subdirectory instead of just anything.

@o11c
Copy link
Contributor

o11c commented Jul 13, 2014

@cmr Note that you are required to ignore the environment variable if it's not an absolute path, please don't join it to home.

@errordeveloper
Copy link

Looks like this is some sort global cache of git repositories? Having a global one is totally an optimisation, it should be per-project for the start, which is the safest way.

I'd vote on closing this, it's making this feature more complicated, and it wasn't is not quite right already. Instead there could be just a simple environment variable (or a config option or flag) that user can set, but it would default to the current working directory.

I also think this PR was a big time bikeshed, sorry @cmr.

@emberian
Copy link
Member Author

@errordeveloper it is not any less safe to have a per-project "cache", and this is already how cargo works. It does not affect the feature in any way besides storing everything in a more correct location than ~/.cargo

@errordeveloper
Copy link

Are those by any chance bare clones which are then used as reference for per-project clones? (That would be my optimisation)

@jhasse
Copy link

jhasse commented Jul 14, 2014

@o11c How does the spec say that I'm wrong? It doesn't say what is meant by "user-specific data files". Therefore I assume the same rules as for /usr/share and /usr/local/share (since the only thing different is the prefix, /usr, /usr/local and ~/.local).

The rules for /usr/share are read-only data files: http://www.pathname.com/fhs/pub/fhs-2.3.html

@o11c
Copy link
Contributor

o11c commented Jul 15, 2014

@jhasse because despitet the similarities FHS does not specify ~/.local/, XDG does. And XDG says that ~/.local/share is for writable data. (Usually it's read-mostly, but write if you're going to read it later).

That said, since this information is nonessential and could be recovered, there's also a good argument for XDG_CACHE_HOME.

@jhasse
Copy link

jhasse commented Jul 15, 2014

Where does XDG say that local/share is for writable (during runtime) data?

@lucy
Copy link

lucy commented Jul 15, 2014

The XDG basedir spec doesn't say anything about being analogous to FHS, and XDG_DATA_HOME is already widely used for runtime writable data.

@jhasse
Copy link

jhasse commented Jul 15, 2014

Okay I give in. But IMHO the spec should be clearer about this and also it's kind of confusing (to me at least) that this has a different role as /usr/share.

@SimonSapin
Copy link
Contributor

Since git repositories and checkouts are "non-essential" (they can be downloaded again and recreated), I thought they’d go into $XDG_CACHE_HOME rather than $XDG_DATA_HOME.

@alexchandel
Copy link

👍 👍 👍 Why hasn't this landed yet? There's no reason whatsoever for Cargo to be dumping garbage into my home directory. This isn't 1970, Cargo shouldn't imitate the few hardcoded disappointments that do (.ssh, .bashrc, etc). If ~/.config/git is enough for git, then ~/.cache/cargo is enough for Cargo.

@alexchandel
Copy link

Moreover for the OS X case, ~/Library/cargo is NOT an appropriate place for cargo's files.

Either ~/.cache/cargo or ~/Library/io.crates.cargo is appropriate for cache files, and either ~/.local/share/cargo or ~/Library/Application Data/io.crates.cargo is appropriate for persistent data files. However, the ~/Library locations should be reserved for Cocoa .apps, so Cargo's best option on OS X is ~/.cache/cargo.

@tomjakubowski
Copy link
Contributor

However, the ~/Library locations should be reserved for Cocoa .apps, so Cargo's best option on OS X is ~/.cache/cargo.

@alexchandel do you have a source for that? I don't have my Mac handy but IIRC homebrew and some other CLI applications use ~/Library.

@alexcrichton
Copy link
Member

If we would like to move forward with this, I would be in favor of a less radical approach than one taken here. It is easier to convey "look in ~/.cargo for information" than "look up your local XDG spec to see where you're supposed to look", and I also believe that generally most users don't necessarily mind if the spec is "violated".

Would the following be palatable?

  • Configuration is queried for in both the .cargo/config location
  • Configuration is also queried (merged) fro the xdg location
  • Add a configuration key for ~/.cargo/git (to move to an "appropriate location").
  • Add configuration keys for other such locations cargo requires so they can be moved.

This would allow those that would like to respect the XDG spec to have their config file in the right location with configuration to keep it all in the right location while also allowing other users to have an easily discoverable location for this information.

@SimonSapin
Copy link
Contributor

Speaking only for myself: I’m mildly annoyed that Cargo puts temporary files at a non-standard location, but not enough to bother writing a configuration file and maintaining it when Cargo evolves.

Also, I don’t understand why having a configuration key for where to put things is preferable to just looking up an environment variable.

Users don’t need to look at the XDG spec if they haven’t overridden the environment variables themselves. It’s not that complicated: Configuration is in ~/.config/<program name>/, ~/.cache can safely be nuked (or just excluded in backups).

And it’s really not radical. Lots of programs already use this convention: ls -1 ~/.config|wc -l is 52 on my system.

@alexcrichton
Copy link
Member

I'm under the impression that if I'm anywhere but linux then the configuration is not in ~/.config, so I don't believe that it's "just that simple."

I would also be ok with environment variables, but can you point to any other package managers that use this sort of scheme? All package managers I know of store data in ~/.<name>.

@SimonSapin
Copy link
Contributor

I don’t know what the conventions are on other systems, I’m only stating what they are on Linux.

I don’t see why package managers would be special, compared to other programs, regarding where to have configuration files and cache files.

@tomjakubowski
Copy link
Contributor

~/.cargo/config should also be configurable at $XDG_CONFIG_HOME/cargo/config, as Git does with ~/.gitconfig.

@Earnestly
Copy link

Along with git, other low-level projects such as JACK and util-linux also follow the XDG Base Directory Specification.

Other Operating Systems also have equivalents such as Application Data and Local Application Data which has existed since Windows 98. OSX provides HOME/Library (barring /Library and /System/Library which are OSX-only).

This problem has been so prevalent on places like Windows and OSX that developer guides where written such as "Don't Pollute User Space" from Apple (Atwood blog about it) to people such as Raymond Chen and his Stop the madness: Subdirectories of My Documents article.

Unlike Windows and OSX, Linux has not had a simple guide for developers to use which gives control back to the users to define the hierarchical structure in their home directory until the XDG Base Directory Specification.

Before this we (Linux users) relied on ad-hoc environment variables that some programs exposed such as GNUPGHOME or SLRNHOME with some people going so far as you create LD_PRELOAD hacks such as libetc and even FUSE filesystems such as rewritefs in a vein attempt to combat the ever increasing complexity of HOME.

Not to mention that "hiding" dotfiles was a originally a bug which has since become a regretable feature (in my opinion).

Does cargo really think HOME/.cargo is acceptable these days?

@errordeveloper
Copy link

@tomjakubowski why don't you just symlink it for now... like in in the old days ;) It will take people here a while to decide, as it seems.

@mcpherrinm
Copy link

Cargo checking out git repos into .cargo is a pretty painful: I have small & slow NFS homedirs on several machines I use, and not respecting $XDG_CACHE_DIR and friends means I have to manually ln -s .cargo $XDG_CACHE_DIR/cargo

While this is not the most typical configuration, I really hope we can conform to the XDG spec, at least on Linux. My ~/.cargo is currently 1.2GB, which is more than double the size of my homedir.

@o11c
Copy link
Contributor

o11c commented Nov 12, 2014

Can someone please make a PR on https://github.com/o11c/rust-xdg to use the proper directories on Windows?

I'd also like to add support for Documents/Videos/Pictures/Music eventually, and also give it a nicer API, but it's certainly functional already.

@Earnestly
Copy link

@o11c That is quite a nice implementation, I'm glad you actually check for absolute paths as well! (Many seem to miss that.)

@o11c
Copy link
Contributor

o11c commented Nov 13, 2014

@Earnestly it's amazing how much easier it is to follow the spec when you sit down and read the spec.

@gyscos
Copy link

gyscos commented May 1, 2015

What's the current status regarding XDG-directories support?

Issue #1148 is fixed, and if I set $CARGO_HOME to $HOME/.config/cargo, it is correctly used without creating a .cargo inside. But when $CARGO_HOME is not set, we don't seem to use the XDG directories by default; we don't even seem to look at it at all.

@SimonSapin
Copy link
Contributor

What's the current status regarding XDG-directories support?

The status is that it’s not supported, as you found out. This PR being shown as "merged" is a bug.

@Earnestly
Copy link

Is there a new bug report about getting this supported? Snowflake environment
variables are nice and all but when you have 40+, not so much.

@SimonSapin
Copy link
Contributor

@Earnestly I could not find an open issue. Please feel free to file one.

soc added a commit to soc/cargo that referenced this pull request Mar 15, 2018
This change stops cargo from violating the operating system rules
regarding the placement of config, cache, ... directories on Linux,
macOS and Windows.

Existing directories and overrides are retained.

The precedence is as follows:

1) use the `CARGO_HOME` environment variable if it exists (legacy)
2) use `CARGO_CACHE_DIR`, `CARGO_CONFIG_DIR` etc. env vars if they exist
3) use the ~/.cargo directory if it exists (legacy)
4) follow operating system standards

A new cargo command, `dirs`,  is added, which can provide path
information to other command line tools.

Fixes:
  rust-lang#1734
  rust-lang#1976
  rust-lang/rust#12725

Addresses:
  rust-lang/rfcs#1615
  rust-lang#148,
  rust-lang#3981
soc added a commit to soc/cargo that referenced this pull request Apr 28, 2018
This change stops cargo from violating the operating system rules
regarding the placement of config, cache, ... directories on Linux,
macOS and Windows.

Existing directories and overrides are retained.

The precedence is as follows:

1) use the `CARGO_HOME` environment variable if it exists (legacy)
2) use `CARGO_CACHE_DIR`, `CARGO_CONFIG_DIR` etc. env vars if they exist
3) use the ~/.cargo directory if it exists (legacy)
4) follow operating system standards

A new cargo command, `dirs`,  is added, which can provide path
information to other command line tools.

Fixes:
  rust-lang#1734
  rust-lang#1976
  rust-lang/rust#12725

Addresses:
  rust-lang/rfcs#1615
  rust-lang#148,
  rust-lang#3981
ehuss pushed a commit to ehuss/cargo that referenced this pull request Nov 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.