avoid race condition on worker disposal

[morgilad]

Disposing of Kusto Worker on setLanguageSettings Call

Every time setLanguageSettings is called, the old Kusto worker is disposed of, and a new one is created. This means that if the old worker is in the middle of handling requests, those requests will be terminated before completion.

What This Causes

We register the semantic token provider after worker.setSchema has completed. Since setLanguageSettings is called when entering the query page editor, the worker gets disposed of during that call, leaving the setSchema promise unresolved. As a result, the semantic tokens provider is not registered.

The Workaround Solution

The solution is a bit of a workaround—apologies for that. I wait for 5 seconds before disposing of the old worker to allow it to finish processing previous requests. Before that, I reset the workerManager private member of workerDetails, so new requests will create a new worker and be applied to the new instance.

Is Disposing the Old Worker Necessary?

I did consider whether disposing of the old worker when setLanguageSettings is called is truly necessary. I believe it isn’t. However, Noam and I decided not to remove this behavior currently because it might impact unknown scenarios that we cannot easily test. Therefore, this change is not suitable when a Quick Fix Engineering (QFE) is required.

[noam-shapira]

nit: remove the prefix _

[morgilad]

Merging this in order to start publishing the new version as it takes a few hours.
If Ill get important comments I will postpone updating the version in the main project.