Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove custom custom_llm_gpu resource specification for text-embeddings template #89

Merged
merged 5 commits into from
Feb 26, 2024
Merged

Remove custom custom_llm_gpu resource specification for text-embeddings template #89

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
45dceff
add custom resource config
Feb 24, 2024
e8ef6bf
update custom_llm_gpu
Feb 24, 2024
2fd67b3
use default gpu instead of custom resource config
Feb 25, 2024
daca950
fix gcp config
Feb 25, 2024
a2a45dc
remove gpu specification
Feb 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions configs/text-embeddings/aws.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,5 @@ aws:
TagSpecifications:
- ResourceType: instance
Tags:
- Key: as-feature-enable-multi-az-serve
Value: "true"
- Key: as-feature-multi-zone
Value: "true"
- Key: as-feature-cross-group-min-count-custom_llm_gpu
Value: "2"
- Key: as-feature-cross-group-max-count-custom_llm_gpu
Value: "16"
13 changes: 0 additions & 13 deletions configs/text-embeddings/gce.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,6 @@ worker_node_types:
max_workers: 16
use_spot: true
fallback_to_ondemand: true
gcp_advanced_configurations_json:
instance_properties:
labels:
as-feature-multi-zone: 'true'
as-feature-enable-multi-az-serve: 'true'

# 8 vCPU, 1 NVIDIA Tesla V100 16GB GPU, 30 GiB memory
- name: gpu-worker-v100
Expand Down Expand Up @@ -72,11 +67,6 @@ gcp_advanced_configurations_json:
"accelerator_type:A100-40G": 1
min_workers: 0
max_workers: 8
gcp_advanced_configurations_json:
instance_properties:
labels:
as-feature-multi-zone: 'true'
as-feature-enable-multi-az-serve: 'true'

# 12 vCPU, 1 NVIDIA A100 80GB GPU, 170 GiB memory
- name: gpu-worker-a100-80g
Expand All @@ -94,6 +84,3 @@ gcp_advanced_configurations_json:
instance_properties:
labels:
as-feature-multi-zone: 'true'
as-feature-enable-multi-az-serve: 'true'
as-feature-cross-group-min-count-custom_llm_gpu: '2'
as-feature-cross-group-max-count-custom_llm_gpu: '16'
Loading