Using Step Operators in Steps #3555
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
runllm seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
🔍 Broken Links ReportSummary
Details
📂 Full file paths
|
Documentation Link Check Results❌ Absolute links check failed |
strickvl
left a comment
strickvl
left a comment
There was a problem hiding this comment.
Not a great effort, I'm afraid.
| ZenML will build a Docker images which includes your code and use it to run your steps in Kubernetes. Check out [this page](https://docs.zenml.io/how-to/customize-docker-builds/) if you want to learn more about how ZenML builds these images and how you can customize them. | ||
| {% endhint %} | ||
| # Using Step Operators in Steps |
There was a problem hiding this comment.
should be h2 at least, if not lower.
| ```python | ||
| from zenml import step | ||
| @step(step_operator="my_sagemaker_operator") |
There was a problem hiding this comment.
why is this sagemaker when we're in the spark-kubernetes docs page?
| * [Kubernetes](step-operators/kubernetes.md) | ||
| * [Modal](step-operators/modal.md) | ||
| * [Spark](step-operators/spark-kubernetes.md) | ||
| * [Using Step Operators in Steps](step-operators/using-step-operators-in-steps.md) |
There was a problem hiding this comment.
I'm sorry, it appears you are suggesting to revert this change. This is out of my capabilities at the moment.
There was a problem hiding this comment.
I'm sorry, it appears you are suggesting to revert this change. This is out of my capabilities at the moment.
There was a problem hiding this comment.
Yeah we shouldn't include this kind of documentation in the stacks stuff.
There was a problem hiding this comment.
I'm sorry, it appears you are suggesting to revert this change. This is out of my capabilities at the moment.
There was a problem hiding this comment.
Pull Request Overview
This PR updates the documentation to include sections on how to use step operators within steps across various parts of the ZenML docs. The changes add usage examples and explanations for the step_operator parameter in the @step decorator to help users leverage different execution environments.
- Added a new "Using Step Operators in Steps" section to several documentation files.
- Updated the table of contents to include the new section.
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| docs/book/how-to/pipeline-development/use-configuration-files/runtime-configuration.md | Introduces a section detailing how to specify a step operator in the @step decorator. |
| docs/book/component-guide/toc.md | Adds an entry for the new “Using Step Operators in Steps” documentation page. |
| docs/book/component-guide/step-operators/spark-kubernetes.md | Adds a section with usage details and an example for step operators in steps. |
| docs/book/component-guide/step-operators/sagemaker.md | Inserts a section explaining how to register and use a SageMaker step operator within steps. |
| docs/book/component-guide/step-operators/kubernetes.md | Adds a step operator usage section; note a header level inconsistency. |
| docs/book/component-guide/step-operators/custom.md | Adds content explaining how to implement custom step operators with an example. |
| docs/book/component-guide/step-operators/README.md | Enhances the README with a step operator usage section and corresponding example. |
Comments suppressed due to low confidence (1)
docs/book/component-guide/step-operators/kubernetes.md:99
- [nitpick] For consistency with other documentation files that use lower-level headings for similar sections, consider changing this header from H1 to H3 (e.g., '### Using Step Operators in Steps').
# Using Step Operators in Steps
|
@strickvl Thank you for your review. We are working on revising the PR and will inform you when it is ready for review again. |
1 similar comment
|
@strickvl Thank you for your review. We are working on revising the PR and will inform you when it is ready for review again. |
|
@strickvl We have addressed your comments. Please review the PR again. For the comments we cannot address, we responded inline. Thank you! |
| ```python | ||
| from zenml import step | ||
|
|
||
| @step(step_operator="my_sagemaker_operator") |
| ## Using Step Operators in Steps | ||
| The `step_operator` parameter in the `@step` decorator allows you to specify the step operator for executing the step. This is particularly useful for executing steps in different environments, such as AWS SageMaker. | ||
| Example: | ||
| ```python | ||
| from zenml import step | ||
| @step(step_operator="my_sagemaker_operator") | ||
| def my_training_step(...) -> ...: | ||
| # Step logic here | ||
| pass | ||
| ``` | ||
| In this example, `my_sagemaker_operator` is the name of the step operator registered for AWS SageMaker. The step will be executed in the environment provided by the step operator. | ||
| When running the pipeline, the specified step operator manages the execution environment for the step, leveraging specialized compute resources and capabilities. | ||
There was a problem hiding this comment.
don't we show how to do this exactly above??
There was a problem hiding this comment.
I'm sorry, it appears you are suggesting to revert this change. This is out of my capabilities at the moment.
| * [Kubernetes](step-operators/kubernetes.md) | ||
| * [Modal](step-operators/modal.md) | ||
| * [Spark](step-operators/spark-kubernetes.md) | ||
| * [Using Step Operators in Steps](step-operators/using-step-operators-in-steps.md) |
There was a problem hiding this comment.
Yeah we shouldn't include this kind of documentation in the stacks stuff.
|
@strickvl Thank you for your review. We are working on revising the PR and will inform you when it is ready for review again. |
|
@strickvl We have addressed your comments. Please review the PR again. For the comments we cannot address, we responded inline. Thank you! |
Changes Made
@stepdecorator. This addition will help users understand how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.@stepdecorator, enhancing the flexibility and scalability of their pipelines.@stepdecorator. This addition will help users understand how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.@stepdecorator. This addition provides users with guidance on how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.@stepdecorator. This addition will help users understand how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.@stepdecorator. This addition will help users understand how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.@stepdecorator. This addition will help users understand how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.@stepdecorator. This addition will help users understand how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.@stepdecorator. This addition will help users understand how to leverage step operators for executing individual steps in different environments, enhancing the flexibility and scalability of their pipelines.