How to tell when FileSystemWatcher actually starts listening for events

[vaclavHala]

In our tests it seems when we create a file watcher

const w = vscode.workspace.createFileSystemWatcher(...)

it actually does not start seeing file events for some time after creation.
I went through the docs and code briefly and it seems to really behave that way since the watcher is returned immediately, but in the background there is the machinery of dedicated file watcher process to which the request needs to be delivered by some RPC which is bound to create some delay even if everything else (other vscode internals, parcel, ...) starts the watch instantaneously.

While this is not a huge deal when a real user interacts with the extension (since people are inherently slow and are able to retry any action) it makes it very hard to test any code that depends on file watching in automated tests.

For example we need to do this

const unwrapper = new ProjectUnwrapper() // some code creating file watcher in constructor
await sleep(5000) // give the backing file watchers enought time to actually start watching
await vscode.workspace.fs.rename(j('foo.fizz'), j('bar.fizz'))
expect(unwrapper).reactedToFileEventSomehow()

Which works most of the time but breaks down when the CI is under load because the 5sec sleep is no longer enough.

We also tried watching the system log (by tailing the log file on disk and using this to unblock barriers in the test) with trace logging enabled as described here https://github.com/microsoft/vscode/wiki/File-Watcher-Issues but sometimes that did not work either + some ways of creating the watcher do not seem to produce any message into that log at all (e.g. when given just single string arg to the createFileSystemWatcher).

Is there some better way to go about this?

I think it'd be very useful to have some means provided by vscode API to know when one can reliably expect file events to be emitted e.g.

export interface FileSystemWatcher extends Disposable {
  onWatcherStateChanged: Event<FileWatcherState>;
}

export enum FileWatcherState {
  Preparing, // no events will arrive yet, watch is being set up
  Watching , // fully operational
  Degraded,  // e.g. when the dreaded "No more file descriptors" problem appears, 
             // some events might still appear but can't be relied on to be complete
}

[theReynald]

Here's what Copilot suggest which seems reasonable to me but let me know your thoughts.

Your analysis of the delay in the FileSystemWatcher becoming fully operational due to the underlying asynchronous setup process is accurate. This indeed poses challenges for automated testing, especially in CI environments where resources can be constrained and unpredictable.

Currently, the Visual Studio Code API does not provide a direct way to determine when a FileSystemWatcher is fully operational. Your approach of introducing a delay (await sleep(5000)) is a common workaround but, as you've discovered, it's not reliable under all conditions.

Proposed Solution
While the Visual Studio Code API does not currently offer an event or method to directly track the state of a FileSystemWatcher, you can propose this feature to the VS Code development team. The proposed FileSystemWatcher interface with an onWatcherStateChanged event is a solid idea that could significantly improve the reliability of extensions that rely on file watching, especially in automated testing scenarios.

Workaround Strategies
Until such a feature might be implemented, here are a few strategies that could help mitigate the issue:

1. Polling for Changes: Instead of relying on a fixed sleep duration, implement a polling mechanism that checks for an expected change or a sentinel file's presence. This is more reliable than sleeping for a fixed period but can introduce its own overhead.

2. Optimize CI Environment: Ensure that your CI environment is optimized for performance to reduce variability in execution times. This includes using faster machines, ensuring sufficient resources are available, and minimizing the load from other processes.

3. Mocking or Intercepting: For unit tests, consider mocking the FileSystemWatcher or the functionality that relies on it. This allows you to bypass the actual file watching mechanism and directly test the logic that responds to file changes.

4. Contribution to VS Code: As a more long-term solution, consider contributing to the VS Code project itself. Implementing the onWatcherStateChanged event as part of the FileSystemWatcher interface would be a valuable addition to the API. This would involve engaging with the VS Code development community, proposing the feature, and potentially contributing code.

5. Feedback to VS Code Team: If contributing code is not feasible, you can still propose this feature through the VS Code GitHub issues page. Detailed proposals that outline the use case, benefits, and potential implementation can gain traction and lead to official support.

Example Polling Workaround
Here's a simplified example of a polling mechanism:

async function waitForFileChange(expectedFilePath, timeout = 10000) {
  let elapsedTime = 0;
  const pollInterval = 500;

  while (elapsedTime < timeout) {
    if (await checkFileExists(expectedFilePath)) {
      return true; // File change detected
    }
    await sleep(pollInterval);
    elapsedTime += pollInterval;
  }
  throw new Error('Timeout waiting for file change');
}

async function checkFileExists(filePath) {
  try {
    await vscode.workspace.fs.stat(vscode.Uri.file(filePath));
    return true;
  } catch (error) {
    return false;
  }
}

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

This example checks for the existence of a file, but you could adapt it to check for specific file changes or the presence of sentinel files that indicate readiness.

While these workarounds can help, the ideal solution would be for the VS Code API to provide a more reliable way to determine the operational state of FileSystemWatcher instances.

[olguzzar]

Hi @vaclavHala, are you still having this issue?

[vaclavHala]

Hello @olguzzar , yes we are still facing this problem. I created a feature request here but since this is a feature only really useful for extension authors who try to test the way their extensions integrate with file watching there seems to be little interest.

At the moment we are considering mocking the whole file watcher in tests and emitting expected file events manually, since the vscode provided facility can't be relied on in automated testing for reasons outlined above. While this approach makes the tests reliable, it lowers confidence in the features in the extension being tested working as expected "in the real world" when combined with the real file watching. Also it makes writing the tests harder in general as the extension author has to effectively replicate the expected watch behavior in every test. I hope the file watching issue will be addressed in vscode eventually.

[olguzzar]

Thanks for your response - I appreciate you filing the feature request. I'll leave this discussion open here in case others have thoughts on a workaround, and so anyone who comes across this can upvote your feature request to gain interest.

[akim-muto]

I also ran into the problem of changes being delayed when opening files using vscode.workspace.openTextDocument via FileSystemWatcher's onDidChange.

I will share the repository for those who are interested in verification.
https://github.com/akim-muto/vsc-opentxt-tmp