fix: Use cmdshim instead of symlink on win32 in base-fetcher

[yoadsn]

Summary
This is an implementation of the suggestion by @arcanis to fix #6604. In short, on win32 platforms, the symlink requires elevated privileges. This fix would use the cmdshim to work around the need for the symlink specifically on the base-fetcher module.

Test plan
Same test suit was used since platform emulation is not included today.

[arcanis]

Perfect, thanks!

[arcanis]

I fixed the cache versioning, but it seems there are issues with two tests on Windows:

Summary of all failing tests
FAIL __tests__\commands\check.js
  ● --integrity should ignore comments and whitespaces in yarn.lock
    Error: ENOENT: no such file or directory, stat 'C:\Users\appveyor\AppData\Local\Temp\1\yarn-bundled-dep-check-1541114868149-0.6443591651832778\.yarn-cache\v4\npm-nyc-10.3.2-35563cb271637e2dc922a1145d8981e948a9f5c3\node_modules\nyc\bin\nyc.js' 
    Console output:
     [1/4] Resolving packages...
    [2/4] Fetching packages...
      185 |     }
      186 |   } catch (err) {
    > 187 |     throw new Error(`${err && err.stack} \nConsole output:\n ${out}`);
      188 |   } finally {
      189 |     reporter.close();
      190 |     await fs.unlink(cwd);
      
      
      at __tests__/commands/_helpers.js:187:11
          at Generator.throw (<anonymous>)
      at step (node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
      at node_modules/babel-runtime/helpers/asyncToGenerator.js:30:13
  ● should ignore bundled dependencies
    Error: ENOENT: no such file or directory, stat 'C:\Users\appveyor\AppData\Local\Temp\1\yarn-bundled-dep-check-1541114868149-0.6443591651832778\.yarn-cache\v4\npm-nyc-10.3.2-35563cb271637e2dc922a1145d8981e948a9f5c3\node_modules\nyc\bin\nyc.js' 
    Console output:
     [1/4] Resolving packages...
    [2/4] Fetching packages...
      185 |     }
      186 |   } catch (err) {
    > 187 |     throw new Error(`${err && err.stack} \nConsole output:\n ${out}`);
      188 |   } finally {
      189 |     reporter.close();
      190 |     await fs.unlink(cwd);
      
      
      at __tests__/commands/_helpers.js:187:11
          at Generator.throw (<anonymous>)
      at step (node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
      at node_modules/babel-runtime/helpers/asyncToGenerator.js:30:13
Test Suites: 1 failed, 87 passed, 88 total
Tests:       2 failed, 3 skipped, 1195 passed, 1200 total

[yoadsn]

Able to repro on a windows machine.
@arcanis Obviously, the check command is not happy about the shim usage. I'm not proficient enough
with the code and the check logic to determine if it's the way check is implemented or the checks should be different on win32. Maybe its the test suit itself who should have different expectations on win32.
Regardless, since the shim does not do what a symlink does - there might be other places where the tests do not cover well and yarn would still fail due to the missing real symlinks.
What would you suggest? Any other approach or should we keep going this path? Getting elevated privileges on Azure sandbox seems out of the question.

[arcanis]

Getting elevated privileges on Azure sandbox seems out of the question.

Yep, totally!

What would you suggest? Any other approach or should we keep going this path?

I'll try to repro it when I get to a Windows machine (this evening, hopefully). I'm not sure what yarn check does that is so special (we're actually considering removing it Yarn 2).

[arcanis]

Ok, I think I've found the problem. It's just that the test fixture in those tests is only partial, meaning that the destination of the binary link doesn't actually exist. The new code that creates it expects it to do (probably because of lockMutex, I'd say), and thus crashes. I'm not sure why it crashes during the fetch step (I'd have expected it to happen during the linking), though.

Will fix that next week.

[yoadsn]

So again, since I read this code-base for the first time - In the failing fixture's runInstall, which creates an Install and calls init() on it would cause a bunch of steps to be queued and executed. The first queued step is the resolve, the second queued step would be a fetch which would in turn call the fetch() on the fetcher. This package-fetcher would use the patched code in base-fetcher which expects the target bin file to exist but is not since the fixture does not run something which usually would have been. Strange.
The linker should be the one calling linkBin which would place the file where it should be. Since fetch runs before that (from all I can figure out) - What would you expect to run before that to make the test complete? Did I understand correctly?

Is it safe to assume the original code did not fail since symlink does not check/depend on the existence of the src (first arg, usually called target though) but the patched code relies on it to exist?
In that case - My thought it that the fact that cmdShim does expect the file to exist is the culprit and in fact, if the package's bin has not bin linked before, this is a valid scenario (src does not exist).
When patching cmdShim to use cmdShim.ifExists the tests seem to pass. So it is semantically close in that regard to the original symlink.

WDYT?

[buildsize]

File name	Previous Size	New Size	Change
yarn-[version].noarch.rpm	1.09 MB	1.09 MB	319 bytes (0%)
yarn-[version].js	4.27 MB	4.27 MB	454 bytes (0%)
yarn-legacy-[version].js	4.44 MB	4.45 MB	450 bytes (0%)
yarn-v[version].tar.gz	1.1 MB	1.1 MB	1.63 KB (0%)
yarn_[version]all.deb	805.17 KB	805.04 KB	-136 bytes (0%)