Recompiling & Refreshing of all Jobs to be made asynchronous

[sravankorumilli]

As per RFC let's make the refresh api asynchronous.

Scenarios

1. On A successful refresh request user should be provided with a request_id for deployment
2. Optimus CLI to automatically poll with the respective deploy request id.
3. If a deploy request exists for a given project in the queue then same request_id to be returned to the user instead of creating a new one.

optimus job refresh

[arinda-arif]

1. Refresh CLI should be able to:
   • Request a refresh which does the dependency resolution refresh and returns a deployment request ID
   • Poll for response until succeed or until a certain timeout.
2. Refresh request will still have stream response
   • Dependency resolution logs
   • Deployment request ID
3. There will be deployment workers that will run priority resolution, compilation, deployment asynchronously.
   • This will be managed by the deployment manager.
   • A project can only be deployed one at a time.
4. Deployment manager will check if the previous event with the same request is available or not.
   • If a similar request is not there in the queue, will create a new deployment request and return the deployment request ID.
   • If the similar request is already there in the queue, will not push the event. Deployment manager will return the existing deployment request ID.
5. Deployment requests should be persisted.
6. Should be able to update deployment status if it has been finished.
7. Will need a new RPC to get deployment status. Or we can consider to have a single RPC to check long running status that can be used by backup, replay, refresh, and deployments.