The test was disabled for a while. We got a VSO item tracking it separately (though that issue is partially fixed now). Opening a new issue here.
The issue doesn't repro on a direct run from command line. I suspect the problem is either path or timing related. Assigning myself for now.