[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #6865

Allenz5 · 2025-06-09T20:37:43Z

Description of changes

Extend AllocationStrategy to include capacity-optimized-prioritized and prioritized options
Add EnableSingleAvailabilityZone as a new configuration parameter
Extend instances_validators.py::InstancesAllocationStrategyValidator to verify that the new AllocationStrategy works correctly with the specified CapacityType
Add and register a new validator to verify that EnableSingleAvailabilityZone works with capacity-optimized-prioritized and prioritized Allocation Strategy
Move AllocationStrategy Enum from config_cluster.py to common.py
Modify slurm.full_config.snapshot.yaml to include the new EnableSingleAvailabilityZone configuration parameter

Tests

Extend test_instances_validators.py::test_instances_allocation_strategy_validator to include new test cases
Extend test_all_validators.py::test_slurm_validators_are_called_with_correct_argument to verify that EnableSingleAvailabilityZoneValidator is properly registered
Add new test_networking_validators.py::test_enable_single_availability_zone_validator test module

References

Checklist

Make sure you are pointing to the right branch.
If you're creating a patch for a branch other than develop add the branch name as prefix in the PR title (e.g. [release-3.6]).
Check all commits' messages are clear, describing what and why vs how.
Make sure to have added unit tests or integration tests to cover the new/modified code.
Check if documentation is impacted by this change.

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…d to AllocationStrategy configuration; Add new configuration of enable_single_availability_zone; Signed-off-by: Hanxuan Zhang <hanxuanz@amazon.com>

… develop

…gy validator

… enable_single_availability_zone

…nfig.cluster_config to pcluster.config.common

…ingle_availability_zone

…fig.py to common.py

…ster_config.py to common.py" This reverts commit 13d8cfa.

…dator in cluster_config.py and add registration tests

…ility_zone from False to None

… to slurm.full_config.snapshot.yaml

cli/src/pcluster/config/cluster_config.py

himani2411 · 2025-06-09T21:45:41Z

cli/src/pcluster/config/cluster_config.py

@@ -3179,6 +3191,11 @@ def _register_validators(self, context: ValidatorContext = None):  # noqa: C901
                    ]
                    for validator in flexible_instance_types_validators:
                        self._register_validator(validator, **validator_args)
+                    self._register_validator(


Why add the EnableSingleAvailabilityZoneValidator in a loop for each compute resource when we define the Parameters to be queue Specific and do not take any values from those said Compute resources?

himani2411 · 2025-06-09T21:47:57Z

cli/src/pcluster/schemas/cluster_schema.py

@@ -752,6 +751,7 @@ class SlurmQueueNetworkingSchema(QueueNetworkingSchema):
        PlacementGroupSchema, metadata={"update_policy": UpdatePolicy.MANAGED_PLACEMENT_GROUP}
    )
    proxy = fields.Nested(QueueProxySchema, metadata={"update_policy": UpdatePolicy.QUEUE_UPDATE_STRATEGY})
+    enable_single_availability_zone = fields.Bool(metadata={"update_policy": UpdatePolicy.QUEUE_UPDATE_STRATEGY})


Did we test the use of keeping this UpdatePolicy.QUEUE_UPDATE_STRATEGY for this paramter?

himani2411 · 2025-06-09T21:50:24Z

cli/src/pcluster/validators/instances_validators.py

+            self._add_failure(
+                f"Compute Resource {compute_resource_name} is using a SPOT CapacityType but the "
+                f"Allocation Strategy specified is {allocation_strategy.value}. SPOT CapacityType cannot use "
+                f"'{common.AllocationStrategy.PRIORITIZED.value}' allocation strategy.",


Perfect! However, lets follow the messaging to specify which AllocationStrategy SPOT Capacity type supports as the last line. Similar to ONDemand

himani2411 · 2025-06-09T21:53:27Z

...ster/templates/test_cluster_stack/test_cluster_config_limits/slurm.full_config.snapshot.yaml

@@ -219,6 +219,7 @@ Scheduling:
    Networking:
      AdditionalSecurityGroups: null
      AssignPublicIp: null
+      EnableSingleAvailabilityZone: null


Can we keep one of them as True?

himani2411 · 2025-06-09T22:02:07Z

cli/tests/pcluster/validators/test_all_validators.py

@@ -224,6 +224,9 @@ def test_slurm_validators_are_called_with_correct_argument(test_datadir, mocker)
    single_instance_type_subnet_validator = mocker.patch(
        networking_validators + ".SingleInstanceTypeSubnetValidator._validate", return_value=[]
    )
+    enable_single_availability_zone_validator = mocker.patch(


I think lets test both for true and false (values) for enable_single_availability_zone and different AllocationStrategy

himani2411 · 2025-06-10T17:24:54Z

I would suggest to add a title to the PR too

… develop

…eValidator registration

…tionStrategyValidator

…tor registration test

Allenz5 and others added 13 commits June 3, 2025 14:56

[Subnet Prioritization] Add prioritized|capacity-optimized-prioritize…

bcb3346

…d to AllocationStrategy configuration; Add new configuration of enable_single_availability_zone; Signed-off-by: Hanxuan Zhang <hanxuanz@amazon.com>

Merge branch 'develop' of github.com:Allenz5/aws-parallelcluster into…

bd77a73

… develop

Merge branch 'aws:develop' into develop

96d1e01

[Subnet Prioritization] Add test cases for instance allocation strate…

1360560

…gy validator

[Subnet Prioritization] Update the default value and update policy of…

8cf10aa

… enable_single_availability_zone

[Subnet Prioritization] Move AllocationStrategy Enum from pcluster.co…

7f303dd

…nfig.cluster_config to pcluster.config.common

[Subnet Prioritization] Add validator and validator test for enable_s…

26ddfec

…ingle_availability_zone

[Subnet Prioritization] Move AllocationStrategy Enum from cluster_con…

13d8cfa

…fig.py to common.py

Revert "[Subnet Prioritization] Move AllocationStrategy Enum from clu…

f8de4a5

…ster_config.py to common.py" This reverts commit 13d8cfa.

[Subnet Prioritization] Register enable_single_availability_zone_vali…

c7889cd

…dator in cluster_config.py and add registration tests

[Subnet Prioritization] Change default value of enable_single_availab…

b6506cf

…ility_zone from False to None

[Subnet Prioritization] Add enable_single_availability_zone parameter…

a3b7672

… to slurm.full_config.snapshot.yaml

Merge branch 'aws:develop' into develop

33d3fd1

Allenz5 requested review from a team as code owners June 9, 2025 20:37

github-advanced-security bot found potential problems Jun 9, 2025

View reviewed changes

cli/src/pcluster/config/cluster_config.py Fixed Show resolved Hide resolved

himani2411 reviewed Jun 9, 2025

View reviewed changes

Allenz5 changed the title ~~[Subnet Prioritization]~~ [Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy Jun 11, 2025

Allenz5 added 7 commits June 11, 2025 15:55

[Subnet Prioritization] Fix format issues

406d5e8

Merge branch 'develop' of github.com:Allenz5/aws-parallelcluster into…

dee33ca

… develop

[Subnet Prioritization] Update CHANGELOG.md

f2a0c7e

[Subnet Prioritization] Remove duplicated AllocationStrategy Enum

2e94485

[Subnet Prioritization] Remove duplicated EnableSingleAvailabilityZon…

d2c2ba6

…eValidator registration

[Subnet Prioritization] Update the failure message of InstancesAlloca…

db7a9c5

…tionStrategyValidator

[Subnet Prioritization] Update enable_single_availability_zone_valida…

99773c4

…tor registration test

Allenz5 added 3 commits June 12, 2025 14:59

[Subnet Prioritization] Update format

ce299af

[Subnet Prioritization] Fix EnableSingleAvailabilityZoneValidator

aa81737

[Subnet Prioritization] Fix format issue

24129ff

Allenz5 force-pushed the develop branch from 66049cf to 24129ff Compare June 18, 2025 18:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #6865

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #6865

Uh oh!

Allenz5 commented Jun 9, 2025

Uh oh!

Uh oh!

himani2411 Jun 9, 2025

Uh oh!

himani2411 Jun 9, 2025

Uh oh!

himani2411 Jun 9, 2025

Uh oh!

himani2411 Jun 9, 2025

Uh oh!

himani2411 Jun 9, 2025

Uh oh!

himani2411 commented Jun 10, 2025

Uh oh!

Uh oh!

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #6865

Are you sure you want to change the base?

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #6865

Uh oh!

Conversation

Allenz5 commented Jun 9, 2025

Description of changes

Tests

References

Checklist

Uh oh!

Uh oh!

himani2411 Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

himani2411 Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

himani2411 Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

himani2411 Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

himani2411 Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

himani2411 commented Jun 10, 2025

Uh oh!

Uh oh!