Add scheduler client that pulls from bucket cache

[cgardens]

What

• Part 1 of 2 of this issue: Airbyte Cloud: "Cache" connector containers #5266
• In response to the get spec call being too slow in Cloud, this PR adds a new scheduler client that will pull the spec from a GCS bucket instead of us needing to run the get spec command on a given connector. Retrieving this spec should be much faster than having to run those connectors.
• In a separate PR, I need to add the logic where we write to that spec (as part of the build for each connector).
• See the spec in the issue for more information on how we are thinking about this.

[davinchia]

Quick work!

I'm assuming you tested the GCS storage calls separately.

[davinchia]

This seems more suited for the top-level build.gradle. Can we skip this for now and specify the version in the dependency itself? We can do a sweep later on to consolidate all this at one go.

[cgardens]

agreed.

[davinchia]

Suggested change

	if (cachedSpecOptional.isPresent()) {
	final long now = Instant.now().toEpochMilli();
	final SynchronousJobMetadata mockMetadata = new SynchronousJobMetadata(
	UUID.randomUUID(),
	ConfigType.GET_SPEC,
	null,
	now,
	now,
	true,
	Path.of(""));
	return new SynchronousResponse<>(cachedSpecOptional.get(), mockMetadata);
	} else {
	return client.createGetSpecJob(dockerImage);
	}
	if (cachedSpecOptional.isPresent()) {
	final long now = Instant.now().toEpochMilli();
	final SynchronousJobMetadata mockMetadata = new SynchronousJobMetadata(
	UUID.randomUUID(),
	ConfigType.GET_SPEC,
	null,
	now,
	now,
	true,
	Path.of(""));
	return new SynchronousResponse<>(cachedSpecOptional.get(), mockMetadata);
	}
	return client.createGetSpecJob(dockerImage);

[cgardens]

I'm assuming you tested the GCS storage calls separately.

Do you mean the reads or the writes?

The writes I'm going to do in a separate PR now. The nice thing about this client is that even if the bucket doesn't exist it is safe to merge. It'll just fall back on actually running the spec.

The read is tested in the unit tests. It's a bit awkward because it is more akin to an integration test.