Be a good citizen on Windows and use appropriate directories.

[MicahZoltu]

TL;DR: On Windows, node-gyp should install into %LOCALAPPDATA%/<vendor>/<product>.

In Linux, the accepted location to store :allthethings: is the home directory (~). On Windows, this is not appropriate and almost nothing should go into the root of the USERPROFILE directory or HOME directory. At the moment, node-gyp installs into ~ on Windows 10. It should instead install to somewhere more appropriate such as:

• %APPDATA%/<vendor>/<product> for small user config of a particular product. Note, this is synced over a network/internet so nothing in here should be particularly large.
• %LOCALAPPDATA%/<vendor>/<product> for large user config/extensions/per-user app installs/per-user caching. This folder is not synced over a network/internet so it can be big. Also, it is often redirected to a larger slower drive.
• %PROGRAMDATA%/<vendor>/<product> for local (not synced over network) multi-user configuration data. This data is more permissive in that any user can read/write to it unlike %PROGRAMFILES% which is read only for everyone but admins. Care should be taken with regards to security any executable written here.
• %PROGRAMFILES%/<vendor>/<product> for multi-user application data. Only admins can write here, but everyone can read. Good place to put installed software.

For the case of node_gyp, I assume that %LOCALAPPDATA%/<vendor>/<product> or %PROGRAMFILES%/<vendor>/<product> is the appropriate location, depending on whether there is an expectation of admin rights at install time.

Note: None of the above folders are %USERPROFILE% because the only thing that should use that folder are save dialogs. 😄

[bnoordhuis]

Any change in the defaults is probably automatically semver-major. That means that even if node-gyp makes the switch, it won't end up on user's systems for a long time.

So far no one has complained either so I'd be inclined to leave well enough alone. Changes to the defaults are disruptive to users. Unless there are strongly compelling reasons it's probably not worth doing.

[MicahZoltu]

So far no one has complained either so I'd be inclined to leave well enough alone

This is me complaining. 😄 %USERPROFILE% is the starting point for many user interactions on Windows. When applications like node-gyp (and others) throw a bunch of garbage (things end-users don't ever need to look at/see) into this folder it means that my starting point for all of these user interactions are in the midst of a bunch of garbage I don't want to see. If I want to play some music I have to sift through piles of "trash" that various applications have decided to dump into %USERPROFILE% to find my Music folder. Same if I want to view some pictures or open a document. Normally my strategy for dealing with bad-acting applications like this is to go find/use an alternative. At the moment, however I'm working on a project that requires Node (and node-gyp) so that isn't a tenable solution for me this time, hence the Issue submission and PR.

Changes to the defaults are disruptive to users. Unless there are strongly compelling reasons it's probably not worth doing.

I believe the way I acquired node-gyp was by doing an npm install of some project (though it is possible I got it some other way). If the path were to change, wouldn't it just be re-downloaded to the new location (with cruft left in the old location)? Perhaps there are multiple ways to acquire node-gyp that aren't as auto-magic and won't self-heal?

If there is a desire to maintain backward compatibility, I am willing to adjust my naive PR to check first to see if .node-gyp exists in %USERPROFILE% and if it does set that to the dev-dir and if not then set the correct path to dev-dir. Then on the next major version bump the fallback code can be removed.

If there is a desire to remove the cruft I can add some code that will cleanup the old location if present and use the new location for everything else.

[rvagg]

I don't mind this change, it bothers me too to see config files in %USERPROFILE%. It's definitely semver-major though.

To Ben's point though, it's disruptive because there is a strong assumption in the ecosystem that this is where node-gyp stores and looks for its files. One example is at https://github.com/mafintosh/node-gyp-install which was mainly useful before we had a way of configuring the download source by building it into the binary. Another example would be someone preparing a container or VM image and prepopulating it with required files so it doesn't need to grab things from the internet to do a compile.

So, one way to deal with this might be to first check if the required installVersion file exists in the old directory as well as the new (probably prioritise the new). This can be done partly in lib/install.js (look for devDir there where it's got the version number appended to it) but you'll also need to store the old devDir as built by bin/node-gyp.js that you've edited (i.e. execute the old logic in there as well as the new but store differently).

If node-gyp continues to look for files in the old directory then we'd take care of a large class of the concerns around compatibility.

[MicahZoltu]

To make sure I follow your suggestion fully @rvagg, is the idea to do something like this?

function fileDoesNotExistsInOldLocation() {
  // TODO: check for existence of <homeDir>/.node-gyp/installVersion
}

if (prog.devDir) {
  prog.devDir = prog.devDir.replace(/^~/, homeDir)
} else if (process.platform === 'win32' && process.env.LOCALAPPDATA && fileDoesNotExistsInOldLocation()) {
  prog.devDir = path.resolve(process.env.LOCALAPPDATA, 'nodejs', 'node-gyp');
} else if (homeDir) {
  prog.devDir = path.resolve(homeDir, '.node-gyp')
} else {
  throw ...
}

Plus some async stuff for reading the file off disk of course.

[snewell92]

tldr; naw, keep throwing stuff in ~/.node-gyp

Interesting, I never even really thought about it and I've grown up with windows my whole life. Looking at it now, there's a .gitconfig file, a .rediscli_historyfile, a .VirtualBox folder and VirtualBoxVMs folder and .gimp-2.8 folder, simple build tool has a folder in there, and I actually expect .ssh folder to be in my home directory (as I use multiple ssh keys for CLI git). And none of those examples are from the node community! (There's plenty of those too, cordova, ionic, .vscode, .npmrc, etc.)

I don't really feel intruded upon. I put my music in ~/Music and my pictures in ~/Pictures. I don't think this is a big deal for most windows users, especially programmers.

In fact, I recall learning about %AppData and the local/roaming stuff because of Minecraft I believe, and it felt really odd to have to delve into the depths of hidden folders to fix an issue, patch stuff, or mod things.

I'm not a fan of hard to find configurations. I like ~ on windows/mac/ubuntu, that way I can find stuff as I move between OS's at work or home (mostly Ubuntu <-> Windows).

There's also a file called NTUSER.DAT in my home directory - pretty sure that's a windows-specific file!

[MicahZoltu]

If I understand your argument correctly @snewell92, it is basically (PREFACE: the following paraphrasing is in jest):

edgy humor in the fold

These other 12 people pee in the pool so the pool is already filthy. I have grown accustomed to the scent and taste of urine and come to like it, so Node should pee in the pool as well.

 

While this argument (the non-troll version) does have some merit, I think Node should strive to not contribute to the problem. The reason I call it a problem is because storing large data in %USERPROFILE% can break roaming profiles (not a problem if you only use a single computer, a big problem if you use multiple computers). Also, the guidelines from Microsoft are quite clear on the matter and from a UX point of view, %USERPROFILE% is supposed to contain data that the user put there, not data that an application put there and the user will likely never need to access directly.

Personally, I tend to avoid using any application that dumps stuff in my %USERPROFILE% and I have managed to keep it mostly clean. Node is the one tool that I can't seem to stop using no matter how hard I try to avoid it, and if any alternative option comes up I'll jump ship in a heartbeat partly because of things like this.

[refack]

Just an FYI for people who like their %USERPROFILE% clean; you can set either an env var npm_config_devdir or npm config set devdir to direct node-gyp to use a better path.
Ref: https://github.com/nodejs/node-gyp#configuration

IMHO changing the default is something to think about, mostly since the stuff in .node-gyp is cruft (that is, on it's own, of very little use to the user), contrary to thing like .gitconfig or .ssh/. "Worse" than that, it is just cache, it does not contain any user data. IMHO the default should be %TMP%/.node-gyp-cache.

Anyone wants to submit a PR? We are in semver-major season (node-gyp@4.0.0 is comming soon)
Hint: start digging at https://github.com/nodejs/node-gyp/blob/master/bin/node-gyp.js#L27)

---

[personal opinions]

Personally I have a love/hate relationship with my %USERPROFILE%, I want to keep it clean, but I think I've been dragging it with me for ~15 years (copying and culling every few years, but still 🧀) so it sure looks like a security blanket.

P.S. @MicahZoltu I put your joke in a fold (not so harsh needs to be deleted, but just too colorful to keep as is)

:cough: VSCode :cough:

[MicahZoltu]

Ooh, I didn't know you could do that in GitHub issues. Thanks!

[snewell92]

@MicahZoltu hehe, peeing in the pool xD

I have some genuine questions for you Micah, but they aren't necessarily on topic for this issue thread - I've put them in a fold - I can remove them and post them in a more appropriate place if need be. (um... email?.... gitter chat? irc?)

Question

storing large data in %USERPROFILE% can break roaming profiles

How could it break roaming profiles if it is large? And how large? Like 20, 10, 5Gb? Is this because windows can't sync such a large directory? Windows leverages Microsoft's skydrive tech to do that synchronization, which should not be limited by content/size/encoding, right?

I still don't share your opinion that apps shouldn't place things in the home directory. If an app places files in there, those are usually files the app intends for me to interact with (configuration/ssh/cache - in this case I can manually download bindings for node-gyp), but really tho, if all these dot files do mess with the roaming profile feature I'll get rid of them, it makes wiping/restoring/upgrading windows so much nicer if I can rely on a set of files moving over.

@refack, you asked if anyone wanted to open a PR, but @MicahZoltu opened PR #1125. Are you wanting something different to be opened?

[refack]

@refack, you asked if anyone wanted to open a PR, but @MicahZoltu opened PR #1125. Are you wanting something different to be opened?

Sorry, was not aware. Looking at it.

[jasnell]

Having these files dropped into the right location would be a good thing, semver-major or not. I know that it complicates things on the installation side, but changing the default location but giving the user the option to use the old locations would be beneficial.

[refack]

Having these files dropped into the right location would be a good thing, semver-major or not. I know that it complicates things on the installation side, but changing the default location but giving the user the option to use the old locations would be beneficial.

Since it's a cache, I'm not convinced on the semverity. Only case that would break is a pre-set system that went offline, or if files were manually changed. All other cases will just download again.
Is manually changing the files in .node-gyp/ an interesting use-case?

[bnoordhuis]

People do it to work around corporate proxies and such, so yes.

[gibfahn]

Since it's a cache, I'm not convinced on the semverity.

I suspect changing this would break enough things that it'd need to be semver-major.

If we're changing it for Windows, it'd be good to respect the XDG spec, I guess by defaulting to $XDG_CACHE_HOME/node-gyp on Unix (previously discussed in #175).

[MicahZoltu]

Re: %TEMP%, applications should assume this gets wiped on a regular basis, and applications should be deleting their own stuff out of this directory after they are done with the files. This is meant for very short lived things, not for caching. Caching should be done in %LOCALAPPDATA% as size is generally not constrained there and management of object lifetime in that directory is entirely left up to the application. A good application should delete their vendor folder in %LOCALAPPDATA% on uninstall.

How could it break roaming profiles if it is large?

(feel free to ping me on Gitter @MicahZoltu if you want to have a more live discussion)
Roaming profiles are synced over the network when running on a domain. A common setup for this is to have a roaming profile quota of something like 50MB and then every time you use a new machine (e.g., in a school lab environment) your roaming profile is pulled in over the network at login and then synced back to the network at logout. Often times the local machine is wiped to some well defined clean slate between uses, though this isn't always the case. Files in %APPDATA% should be measured in bytes or kilobytes. Generally config files and such are good here, but not much beyond that.

[soc]

FWIW, I wrote directories a while ago to provide the correct config, cache, data paths across Linux, BSDs, Windows and macOS. Having such a library for NodeJS would address this issue as well as #175 (and follow rules on macOS, too).

Feel free to look at/use the code at https://github.com/soc/directories-rs or https://github.com/soc/directories-jvm.

[jasonkarns]

there's a .gitconfig file

Just an FYI that git doesn't pollute ~ anymore; it adheres to the XDG specification for new installs. (on *nix it would default to ${XDG_CONFIG_HOME:-~/.config}/git/config.) It only uses the old location if the file already exists. I can't speak to what XDG-compliant apps would do on windows. I'd be curious to see what a fresh install on windows does.

(Since it seemed this discussion has been using git as an appeal to authority for where to place files.)