Skip to content

Commit

Permalink
Merge branch 'main' into support-placeholders-for-transform-step
Browse files Browse the repository at this point in the history
  • Loading branch information
ca-nguyen authored Oct 15, 2021
2 parents d575518 + 5f73cf7 commit f16041b
Show file tree
Hide file tree
Showing 8 changed files with 409 additions and 66 deletions.
51 changes: 51 additions & 0 deletions .github/ISSUE_TEMPLATE/bug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
name: "\U0001F41B Bug Report"
about: Report a bug
title: "short issue description"
labels: bug, needs-triage
---

<!--
description of the bug:
-->




### Reproduction Steps

<!--
minimal amount of code that causes the bug (if possible) or a reference.
The code sample should be an SSCCE. See http://sscce.org/ for details.
In short, provide a code sample that we can copy/paste, run and reproduce.
-->

### What did you expect to happen?

<!--
What were you trying to achieve by performing the steps above?
-->

### What actually happened?

<!--
What is the unexpected behavior you were seeing? If you got an error, paste it here.
-->


### Environment

- **AWS Step Functions Data Science Python SDK version :**
- **Python Version:** <!-- Version of Python (run the command `python3 --version`) -->

### Other

<!-- e.g. detailed explanation, stack-traces, related issues, suggestions on how to fix, links for us to have context, eg. associated pull-request, stackoverflow, slack, etc -->




---

This is :bug: Bug Report
28 changes: 28 additions & 0 deletions .github/ISSUE_TEMPLATE/doc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
name: "📕 Documentation Issue"
about: Issue in the reference documentation
title: "short issue description"
labels: feature-request, documentation, needs-triage
---

<!--
- want to help? submit a pull request! docs can be found here: https://github.com/aws/aws-step-functions-data-science-sdk-python/tree/main/doc
-->

<!--
link to reference doc page:
-->



<!--
describe your issue:
-->





---

This is a 📕 documentation issue
46 changes: 46 additions & 0 deletions .github/ISSUE_TEMPLATE/feature-request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
name: "\U0001F680 Feature Request"
about: Request a new feature
title: "short issue description"
labels: feature-request, needs-triage
---

<!-- short description of the feature you are proposing: -->





### Use Case

<!-- why do you need this feature? -->





### Proposed Solution

<!-- Please include prototype/workaround/sketch/reference implementation: -->





### Other

<!--
e.g. detailed explanation, stacktraces, related issues, suggestions on how to fix,
links for us to have context, eg. associated pull-request, stackoverflow, slack, etc
-->





* [ ] :wave: I may be able to implement this feature request
* [ ] :warning: This feature might incur a breaking change

---

This is a :rocket: Feature Request
43 changes: 43 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
### Description

Please include a summary of the change being made.

Fixes #(issue)

### Why is the change necessary?

What capability does it enable? What problem does it solve?

### Solution

Please include an overview of the solution. Discuss trade-offs made, caveats, alternatives, etc.

### Testing

How was this change tested?

----

### Pull Request Checklist

Please check all boxes (including N/A items)

#### Testing

- [ ] Unit tests added
- [ ] Integration test added
- [ ] Manual testing - why was it necessary? could it be automated?

#### Documentation

- [ ] __docs__: All relevant [docs](https://github.com/aws/aws-step-functions-data-science-sdk-python/tree/main/doc) updated
- [ ] __docstrings__: All public APIs documented

### Title and description

- [ ] __Change type__: Title is prefixed with change type: and follows [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/)
- [ ] __References__: Indicate issues fixed via: `Fixes #xxx`

----

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.
48 changes: 37 additions & 11 deletions src/stepfunctions/steps/states.py
Original file line number Diff line number Diff line change
Expand Up @@ -254,27 +254,29 @@ def accept(self, visitor):

def add_retry(self, retry):
"""
Add a Retry block to the tail end of the list of retriers for the state.
Add a retrier or a list of retriers to the tail end of the list of retriers for the state.
See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-retrying-after-an-error>`_ for more details.
Args:
retry (Retry): Retry block to add.
retry (Retry or list(Retry)): A retrier or list of retriers to add.
"""
if Field.Retry in self.allowed_fields():
self.retries.append(retry)
self.retries.extend(retry) if isinstance(retry, list) else self.retries.append(retry)
else:
raise ValueError("{state_type} state does not support retry field. ".format(state_type=type(self).__name__))
raise ValueError(f"{type(self).__name__} state does not support retry field. ")

def add_catch(self, catch):
"""
Add a Catch block to the tail end of the list of catchers for the state.
Add a catcher or a list of catchers to the tail end of the list of catchers for the state.
See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-fallback-states>`_ for more details.
Args:
catch (Catch): Catch block to add.
catch (Catch or list(Catch): catcher or list of catchers to add.
"""
if Field.Catch in self.allowed_fields():
self.catches.append(catch)
self.catches.extend(catch) if isinstance(catch, list) else self.catches.append(catch)
else:
raise ValueError("{state_type} state does not support catch field. ".format(state_type=type(self).__name__))
raise ValueError(f"{type(self).__name__} state does not support catch field. ")

def to_dict(self):
result = super(State, self).to_dict()
Expand Down Expand Up @@ -487,10 +489,12 @@ class Parallel(State):
A Parallel state causes the interpreter to execute each branch as concurrently as possible, and wait until each branch terminates (reaches a terminal state) before processing the next state in the Chain.
"""

def __init__(self, state_id, **kwargs):
def __init__(self, state_id, retry=None, catch=None, **kwargs):
"""
Args:
state_id (str): State name whose length **must be** less than or equal to 128 unicode characters. State names **must be** unique within the scope of the whole state machine.
retry (Retry or list(Retry), optional): A retrier or list of retriers that define the state's retry policy. See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-retrying-after-an-error>`_ for more details.
catch (Catch or list(Catch), optional): A catcher or list of catchers that define a fallback state. See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-fallback-states>`_ for more details.
comment (str, optional): Human-readable comment or description. (default: None)
input_path (str, optional): Path applied to the state’s raw input to select some or all of it; that selection is used by the state. (default: '$')
parameters (dict, optional): The value of this field becomes the effective input for the state.
Expand All @@ -500,6 +504,12 @@ def __init__(self, state_id, **kwargs):
super(Parallel, self).__init__(state_id, 'Parallel', **kwargs)
self.branches = []

if retry:
self.add_retry(retry)

if catch:
self.add_catch(catch)

def allowed_fields(self):
return [
Field.Comment,
Expand Down Expand Up @@ -536,11 +546,13 @@ class Map(State):
A Map state can accept an input with a list of items, execute a state or chain for each item in the list, and return a list, with all corresponding results of each execution, as its output.
"""

def __init__(self, state_id, **kwargs):
def __init__(self, state_id, retry=None, catch=None, **kwargs):
"""
Args:
state_id (str): State name whose length **must be** less than or equal to 128 unicode characters. State names **must be** unique within the scope of the whole state machine.
iterator (State or Chain): State or chain to execute for each of the items in `items_path`.
retry (Retry or list(Retry), optional): A retrier or list of retriers that define the state's retry policy. See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-retrying-after-an-error>`_ for more details.
catch (Catch or list(Catch), optional): A catcher or list of catchers that define a fallback state. See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-fallback-states>`_ for more details.
items_path (str, optional): Path in the input for items to iterate over. (default: '$')
max_concurrency (int, optional): Maximum number of iterations to have running at any given point in time. (default: 0)
comment (str, optional): Human-readable comment or description. (default: None)
Expand All @@ -551,6 +563,12 @@ def __init__(self, state_id, **kwargs):
"""
super(Map, self).__init__(state_id, 'Map', **kwargs)

if retry:
self.add_retry(retry)

if catch:
self.add_catch(catch)

def attach_iterator(self, iterator):
"""
Attach `State` or `Chain` as iterator to the Map state, that will execute for each of the items in `items_path`. If an iterator was attached previously with the Map state, it will be replaced.
Expand Down Expand Up @@ -586,10 +604,12 @@ class Task(State):
Task State causes the interpreter to execute the work identified by the state’s `resource` field.
"""

def __init__(self, state_id, **kwargs):
def __init__(self, state_id, retry=None, catch=None, **kwargs):
"""
Args:
state_id (str): State name whose length **must be** less than or equal to 128 unicode characters. State names **must be** unique within the scope of the whole state machine.
retry (Retry or list(Retry), optional): A retrier or list of retriers that define the state's retry policy. See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-retrying-after-an-error>`_ for more details.
catch (Catch or list(Catch), optional): A catcher or list of catchers that define a fallback state. See `Error handling in Step Functions <https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-fallback-states>`_ for more details.
resource (str): A URI that uniquely identifies the specific task to execute. The States language does not constrain the URI scheme nor any other part of the URI.
timeout_seconds (int, optional): Positive integer specifying timeout for the state in seconds. If the state runs longer than the specified timeout, then the interpreter fails the state with a `States.Timeout` Error Name. (default: 60)
timeout_seconds_path (str, optional): Path specifying the state's timeout value in seconds from the state input. When resolved, the path must select a field whose value is a positive integer.
Expand All @@ -608,6 +628,12 @@ def __init__(self, state_id, **kwargs):
if self.heartbeat_seconds is not None and self.heartbeat_seconds_path is not None:
raise ValueError("Only one of 'heartbeat_seconds' or 'heartbeat_seconds_path' can be provided.")

if retry:
self.add_retry(retry)

if catch:
self.add_catch(catch)

def allowed_fields(self):
return [
Field.Comment,
Expand Down
Loading

0 comments on commit f16041b

Please sign in to comment.