Remote: Add RemoteExecutionService as a layer between spawn execution…

… and remote execution.

The initiative for this change is to let it easier to make changes to RemoteCache and other places.

RemoteCache provides a set of methods to access the remote cache in a protocol independent manner and is used in the following places:
    - In RemoteSpawnCache (for remote cache only mode), we lookup remote cache before local execution. If no cache found, we execute the spawn locally and then upload ouputs to remote cache.
    - In RemoteSpawnRunner (for remote execution mode), we lookup remote cache before requesting remote execution. If no cache found, we upload inputs to remote cache, submit execution request and then download outputs from remote cache.
    - In RemoteRepositoryRemoteExecutor (for executing repository command remotely), we lookup remote cache and upload inputs.
    - In ExecutionServer (for remote worker), we download inputs from remote cache, execute the action and upload outputs to remote cache.

Among these methods, only a few need spawn specific types as members, most of them are just helper functions for easily calling RemoteCacheClient. It's hard to use RemoteCache in other places since we need spawn specific types to construct it. Besides that, some state are not available at the time when we construct RemoteCache e.g. execRoot which only make sense when executing actions. It's impossible to add state as members that are only available during spawn execution to RemoteCache. We move these methods that depend spawn specific types to RemoteExecutionService so RemoteCache is decoupled from spawn execution.

RemoteExecutionService is designed to provide a set of primitive operations for remote cache and execution of spawn actions. RemoteSpwanCache/RemoteSpawnRunner are the intended call sites and do the orchestration.

This change only restructure the code in the call sides. Following changes will update RemoteCache.

PiperOrigin-RevId: 370028737

src/main/java/com/google/devtools/build/lib/remote/RemoteActionContextProvider.java

@@ -48,6 +48,7 @@ final class RemoteActionContextProvider implements ExecutorLifecycleListener {
  private final DigestUtil digestUtil;
  @Nullable private final Path logDir;
  private ImmutableSet<ActionInput> filesToDownload = ImmutableSet.of();
+ private RemoteExecutionService remoteExecutionService;
 
  private RemoteActionContextProvider(
  CommandEnvironment env,
@@ -100,6 +101,24 @@ RemotePathResolver createRemotePathResolver() {
  return remotePathResolver;
  }
 
+ RemoteExecutionService getRemoteExecutionService() {
+ if (remoteExecutionService == null) {
+ remoteExecutionService =
+ new RemoteExecutionService(
+ env.getExecRoot(),
+ createRemotePathResolver(),
+ env.getBuildRequestId(),
+ env.getCommandId().toString(),
+ digestUtil,
+ checkNotNull(env.getOptions().getOptions(RemoteOptions.class)),
+ cache,
+ executor,
+ filesToDownload);
+ }
+
+ return remoteExecutionService;
+ }
+
  /**
  * Registers a remote spawn strategy if this instance was created with an executor, otherwise does
  * nothing.
@@ -121,15 +140,9 @@ public void registerRemoteSpawnStrategyIfApplicable(
  env.getOptions().getOptions(ExecutionOptions.class),
  verboseFailures,
  env.getReporter(),
- env.getBuildRequestId(),
- env.getCommandId().toString(),
- (RemoteExecutionCache) cache,
- executor,
  retryScheduler,
- digestUtil,
  logDir,
- filesToDownload,
- createRemotePathResolver());
+ getRemoteExecutionService());
  registryBuilder.registerStrategy(
  new RemoteSpawnStrategy(env.getExecRoot(), spawnRunner, verboseFailures), "remote");
  }
@@ -145,13 +158,8 @@ public void registerSpawnCache(ModuleActionContextRegistry.Builder registryBuild
  env.getExecRoot(),
  checkNotNull(env.getOptions().getOptions(RemoteOptions.class)),
  checkNotNull(env.getOptions().getOptions(ExecutionOptions.class)).verboseFailures,
- cache,
- env.getBuildRequestId(),
- env.getCommandId().toString(),
  env.getReporter(),
- digestUtil,
- filesToDownload,
- createRemotePathResolver());
+ getRemoteExecutionService());
  registryBuilder.register(SpawnCache.class, spawnCache, "remote-cache");
  }