[Python 3.11 Update] Primitive progressive rollout implementation #4039
Conversation
vitorguidi
changed the title
vitorguidi
force-pushed
the
feature/proto-progressive-rollout
branch
from
5bdbcf7 to
29587f7
Compare
| @@ -92,7 +92,8 @@ def _deploy_app_prod(project, | |||
| yaml_paths, | |||
| package_zip_paths, | |||
| deploy_appengine=True, | |||
| test_deployment=False): | |||
| test_deployment=False, | |||
There was a problem hiding this comment.
Should we delete the test_deployment feature?
There was a problem hiding this comment.
What is this feature? I have never used it
There was a problem hiding this comment.
Oh it is the test-targets thing. We could delete it if this proves successful, we leave the test cluster alive and perform our releases on it with this approach (might be a good idea to create a windows one for that purpose too)
There was a problem hiding this comment.
Now that I think of it, I would rather keep this feature. At some point in the future I would like to pin releases (git commit) in our deployments, so we can have granular control over what code is executed in each cluster.
For now, this is still useful.
|
Do we also need to change the windows startup script? And Android and Mac? |
vitorguidi
force-pushed
the
feature/proto-progressive-rollout
branch
from
53593fb to
8d7fc81
Compare
I would rather not touch Android and Mac. It is too much friction to implement prog rollout in an environment we do not control. As far as windows goes, since we do have our own fleet, a change to the windows startup is on the way. Will be in the PR soon |
…on interpreter versions at runtime from user data env
…t do not implement prog rollout
vitorguidi
force-pushed
the
feature/proto-progressive-rollout
branch
from
9e70992 to
3ebc508
Compare
vitorguidi
changed the title
| TESTS_LAST_UPDATE_KEY = 'tests_last_update' | ||
| TESTS_UPDATE_INTERVAL_DAYS = 1 | ||
|
|
||
| MANIFEST_FILENAME = 'clusterfuzz-source.manifest.3' | ||
| REMOTE_MANIFEST_FILENAME = 'clusterfuzz-source.manifest' |
There was a problem hiding this comment.
It feels like there's a lot of repetition everywhere with this logic to compute the suffixes. Can we instead just create helpers for this that can be used everywhere?
e.g. something along the lines of
def get_source_zip_suffix():
...
def get_source_zip_name():
...We can remove some of these constants if necessary and just make them dynamically computed based on reading from env.
There was a problem hiding this comment.
Moved this to clusterfuzz._internal.base.utils so butler package/deploy and update_task can share this code
| @@ -13,6 +13,13 @@ | |||
| # See the License for the specific language governing permissions and | |||
| # limitations under the License. | |||
|
|
|||
| DEPLOYMENT_ZIP="linux-3.zip" | |||
There was a problem hiding this comment.
I think we might need to rebuild the google images for this to be used.
There was a problem hiding this comment.
The plan is to use progressive rollout to validate on internal, we should not need to perform this operation for now.
vitorguidi
changed the title
Motivation
There is no available mechanism by which two clusterfuzz versions can coexist in production. This PR assumes that, during the GCE VM creation, an user data script will:
If these assumptions hold, we can assume that clusterfuzz will start with the correct package ({platform}-3.zip or {platfor}-3-candidate.zip), and pull the corresponding file in all future update task runs.
This also requires change to butler deploy/package/remote, in order to take into account the two possible releases.
This PR belongs to this effort.