Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Sagemaker pipeline schedules if specified #3271

Merged
merged 71 commits into from
Jan 23, 2025
Merged
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
5570c0c
Create Sagemaker pipeline schedules if specified
htahir1 Dec 20, 2024
6f7bf28
Add property to check if orchestrator is schedulable
htahir1 Dec 20, 2024
0212388
Add EventBridge rule for SageMaker pipeline execution
htahir1 Dec 20, 2024
2125312
Update IAM policy and trust relationship for EventBridge
htahir1 Dec 20, 2024
966f712
Refactor schedule metadata generation for Sagemaker orchestrator
htahir1 Dec 20, 2024
dd7110c
Add scheduling support for SageMaker orchestrator
htahir1 Dec 20, 2024
fcf934e
Remove trust relationship logic in Sagemaker orchestrator
htahir1 Dec 20, 2024
358b3e8
Handle unsupported schedule in custom orchestrator
htahir1 Dec 20, 2024
7ce08b6
Refactor yield statement to use 'yield from' syntax
htahir1 Dec 20, 2024
72bdae1
Ensure IAM permissions for scheduled SageMaker pipelines
htahir1 Dec 20, 2024
abf4610
Update authentication instructions for SageMaker orchestrator
htahir1 Dec 20, 2024
67705c2
Refactor Sagemaker orchestrator metadata handling
htahir1 Dec 20, 2024
d190f31
Add unit tests for SageMaker orchestrator metadata
htahir1 Dec 20, 2024
f1cabc7
Add exception handling for pipeline preparation errors
htahir1 Dec 20, 2024
726b47a
Add timezone information to first execution message
htahir1 Dec 20, 2024
80c8e8e
Add timezone support to AWS SageMaker orchestrator
htahir1 Dec 20, 2024
da8fd35
Update error handling in SagemakerOrchestrator
htahir1 Dec 20, 2024
ab0c06d
Update error handling messages for AWS in Sagemaker orchestrator
htahir1 Dec 20, 2024
3c12d45
Refactor error handling in SagemakerOrchestrator
htahir1 Dec 20, 2024
51a499a
Handle insufficient permissions creating EventBridge rules
htahir1 Dec 20, 2024
7d65a20
Update error message for EventBridge creation failure
htahir1 Dec 20, 2024
e2ecaf3
Remove logging in SagemakerOrchestrator class
htahir1 Dec 20, 2024
a5fd82b
Refactor orchestrator metadata computation logic
htahir1 Dec 20, 2024
d229460
Merge branch 'develop' into feature/add-sagemaker-schedule
htahir1 Dec 22, 2024
a005e5a
Update handling of scheduled pipeline updates in SageMaker.md
htahir1 Dec 22, 2024
645903a
Merge branch 'feature/add-sagemaker-schedule' of github.com:zenml-io/…
htahir1 Dec 22, 2024
8811f3f
Add optional IAM permissions for policy updates
htahir1 Dec 23, 2024
44fa6c5
Remove redundant code for getting SageMaker session
htahir1 Dec 23, 2024
3146b57
Add pipeline scheduler role and handling for scheduling errors
htahir1 Jan 9, 2025
b52f7ed
Merge remote-tracking branch 'origin/develop' into feature/add-sagema…
htahir1 Jan 9, 2025
e52109b
Refactor Sagemaker orchestrator test methods
htahir1 Jan 9, 2025
0272c5e
Update SageMaker orchestrator for scheduled pipelines
htahir1 Jan 9, 2025
6826613
Auto-update of LLM Finetuning template
actions-user Jan 9, 2025
dcfb03e
Auto-update of Starter template
actions-user Jan 9, 2025
87ef96b
Add comment about rounding up to 1 minute for SageMaker
htahir1 Jan 9, 2025
eb96d95
Auto-update of E2E template
actions-user Jan 9, 2025
9542ed8
Auto-update of NLP template
actions-user Jan 9, 2025
467f25b
Validate and format cron expression for SageMaker
htahir1 Jan 9, 2025
48f93ff
Merge branch 'feature/add-sagemaker-schedule' of github.com:zenml-io/…
htahir1 Jan 9, 2025
b995970
Update start time calculation for SageMaker Orchestrator
htahir1 Jan 9, 2025
a0b7756
Add metadata tracking in Sagemaker orchestrator
htahir1 Jan 13, 2025
921292e
Merge branch 'develop' into feature/add-sagemaker-schedule
bcdurak Jan 14, 2025
67bfe9f
merged develop
bcdurak Jan 15, 2025
56b0796
checkpoint
bcdurak Jan 16, 2025
1037230
docs updates
bcdurak Jan 16, 2025
98896d4
second checkpoint
bcdurak Jan 16, 2025
7b07154
merged develop
bcdurak Jan 16, 2025
ab8990d
removing the old dependency limitations
bcdurak Jan 16, 2025
737fb6e
schedule metadata
bcdurak Jan 19, 2025
7f33927
update on the docs
bcdurak Jan 19, 2025
689df66
fixing the endpoint
bcdurak Jan 19, 2025
2d7dc6c
formatting
bcdurak Jan 19, 2025
c5dfd58
improving the error messages
bcdurak Jan 19, 2025
4feb1a2
merged develop
bcdurak Jan 19, 2025
2f88d09
docstrings, linting and formatting
bcdurak Jan 19, 2025
4326836
adding triggers to mocked_libs
bcdurak Jan 19, 2025
770f104
Update docs/book/component-guide/orchestrators/vertex.md
bcdurak Jan 19, 2025
23f37cb
Update docs/book/component-guide/orchestrators/sagemaker.md
bcdurak Jan 19, 2025
f787e51
fixing the failing integration test
bcdurak Jan 20, 2025
c04c92e
merged develop
bcdurak Jan 20, 2025
22a466a
Merge branch 'develop' into feature/add-sagemaker-schedule
bcdurak Jan 20, 2025
8efe7b5
merge develop
bcdurak Jan 21, 2025
d7853fa
formatting
bcdurak Jan 21, 2025
46f8ad7
Update src/zenml/models/v2/core/schedule.py
bcdurak Jan 21, 2025
f02f952
final fixes
bcdurak Jan 21, 2025
969d172
formatting
bcdurak Jan 21, 2025
fda6569
merged develop;
bcdurak Jan 21, 2025
b357940
fixing the integraiton tests
bcdurak Jan 21, 2025
2d3384d
new dependencies
bcdurak Jan 22, 2025
7a5b683
merge develop and resolve conflicts
bcdurak Jan 22, 2025
2ccf917
formatting
bcdurak Jan 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
adding triggers to mocked_libs
bcdurak committed Jan 19, 2025
commit 432683655a264381c4a1fc87857e304aedf00a77
1 change: 1 addition & 0 deletions docs/mocked_libs.json
Original file line number Diff line number Diff line change
@@ -188,6 +188,7 @@
"sagemaker.workflow.execution_variables",
"sagemaker.workflow.pipeline",
"sagemaker.workflow.steps",
"sagemaker.workflow.triggers",
"scipy",
"scipy.sparse",
"sklearn",