Skip to content

Commit

Permalink
fix(stepfunctions-tasks): fix bedrock input/output path in step-funct… (
Browse files Browse the repository at this point in the history
#31305)

### Issue # (if applicable)

Closes #31302.

### Reason for this change

PR#30298 introduced a breaking change in the stepfunctions-tasks which was encoding the input/output path from the `TaskStateBaseProps` to be used to input a S3 URI, which is not always the case and customer might be using the JSON string as it is.

### Description of changes

To keep the functionality from the original ask of issue #29229 ,added another props to specify S3 URI as an input to the stepfunctions-tasks.

Introduced a feature flag to keep the existing behaviour for the customers and not introduce a breaking change.

### Description of how you validated changes

Updated integ test and deployed in account

Integ Test Results:
`aws stepfunctions start-execution --state-machine-arn <deployed state machine arn> `: should return execution arn

A5-zEnXPmmPWTJx
{
    "executionArn": "arn:aws:states:us-east-1:XXXX:execution:StateMachine2E01A3A5-zEnXPmmPWTJx:516f2e60-9507-46cb-95e6-4d9453429b08",
    "startDate": "2024-09-04T15:43:33.200000-07:00"
}

`aws stepfunctions describe-execution --execution-arn <exection-arn generated before> `: should return status as SUCCEEDED



### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
shikha372 authored and GavinZZ committed Sep 10, 2024
1 parent 224f509 commit 73506ae
Show file tree
Hide file tree
Showing 16 changed files with 473 additions and 35 deletions.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@
{
"Ref": "AWS::Region"
},
"::foundation-model/amazon.titan-text-express-v1\",\"Body\":{\"inputText.$\":\"States.Format('Alphabetize this list of first names:\\n{}', $.names)\",\"textGenerationConfig\":{\"maxTokenCount\":100,\"temperature\":1}}}},\"Prompt3\":{\"End\":true,\"Type\":\"Task\",\"InputPath\":\"$.names\",\"OutputPath\":\"$.names\",\"Resource\":\"arn:",
"::foundation-model/amazon.titan-text-express-v1\",\"Body\":{\"inputText.$\":\"States.Format('Alphabetize this list of first names:/n{}', $.names)\",\"textGenerationConfig\":{\"maxTokenCount\":100,\"temperature\":1}}}},\"Prompt3\":{\"Next\":\"Prompt4\",\"Type\":\"Task\",\"OutputPath\":\"$.Body.results[0].outputText\",\"Resource\":\"arn:",
{
"Ref": "AWS::Partition"
},
Expand All @@ -115,7 +115,19 @@
{
"Ref": "AWS::Region"
},
"::foundation-model/amazon.titan-text-express-v1\",\"Input\":{\"S3Uri.$\":\"$.names\"},\"Output\":{\"S3Uri.$\":\"$.names\"}}}},\"TimeoutSeconds\":30}"
"::foundation-model/amazon.titan-text-express-v1\",\"Body\":{\"inputText.$\":\"States.Format('Echo list of first names: {}', $.names)\",\"textGenerationConfig\":{\"maxTokenCount\":100,\"temperature\":1}}}},\"Prompt4\":{\"End\":true,\"Type\":\"Task\",\"Resource\":\"arn:",
{
"Ref": "AWS::Partition"
},
":states:::bedrock:invokeModel\",\"Parameters\":{\"ModelId\":\"arn:",
{
"Ref": "AWS::Partition"
},
":bedrock:",
{
"Ref": "AWS::Region"
},
"::foundation-model/amazon.titan-text-express-v1\",\"Input\":{\"S3Uri\":\"$.names\"},\"Output\":{\"S3Uri\":\"$.names\"}}}},\"TimeoutSeconds\":30}"
]
]
},
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ const prompt2 = new BedrockInvokeModel(stack, 'Prompt2', {
body: sfn.TaskInput.fromObject(
{
inputText: sfn.JsonPath.format(
'Alphabetize this list of first names:\n{}',
'Alphabetize this list of first names:/n{}',
sfn.JsonPath.stringAt('$.names'),
),
textGenerationConfig: {
Expand All @@ -52,13 +52,33 @@ const prompt2 = new BedrockInvokeModel(stack, 'Prompt2', {
resultPath: '$',
});

/** Test for Bedrock Output Path */
const prompt3 = new BedrockInvokeModel(stack, 'Prompt3', {
model,
inputPath: sfn.JsonPath.stringAt('$.names'),
outputPath: sfn.JsonPath.stringAt('$.names'),
body: sfn.TaskInput.fromObject(
{
inputText: sfn.JsonPath.format(
'Echo list of first names: {}',
sfn.JsonPath.stringAt('$.names'),
),
textGenerationConfig: {
maxTokenCount: 100,
temperature: 1,
},
},
),
outputPath: '$.Body.results[0].outputText',
});

/** Test for Bedrock s3 URI Path */
//State Machine Execution will fail for the following input as it expects a valid s3 URI from previous prompt
const prompt4 = new BedrockInvokeModel(stack, 'Prompt4', {
model,
input: { s3InputUri: '$.names' },
output: { s3OutputUri: '$.names' },
});

const chain = sfn.Chain.start(prompt1).next(prompt2).next(prompt3);
const chain = sfn.Chain.start(prompt1).next(prompt2).next(prompt3).next(prompt4);

new sfn.StateMachine(stack, 'StateMachine', {
definitionBody: sfn.DefinitionBody.fromChainable(chain),
Expand Down
40 changes: 39 additions & 1 deletion packages/@aws-cdk/cx-api/FEATURE_FLAGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ Flags come in three types:
| [@aws-cdk/pipelines:reduceAssetRoleTrustScope](#aws-cdkpipelinesreduceassetroletrustscope) | Remove the root account principal from PipelineAssetsFileRole trust policy | 2.141.0 | (default) |
| [@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm](#aws-cdkaws-ecsremovedefaultdeploymentalarm) | When enabled, remove default deployment alarm settings | 2.143.0 | (default) |
| [@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault](#aws-cdkcustom-resourceslogapiresponsedatapropertytruedefault) | When enabled, the custom resource used for `AwsCustomResource` will configure the `logApiResponseData` property as true by default | 2.145.0 | (fix) |
| [@aws-cdk/aws-s3:keepNotificationInImportedBucket](#aws-cdkaws-s3keepnotificationinimportedbucket) | When enabled, Adding notifications to a bucket in the current stack will not remove notification from imported stack. | 2.155.0 | (fix) |
| [@aws-cdk/aws-stepfunctions-tasks:useNewS3UriParametersForBedrockInvokeModelTask](#aws-cdkaws-stepfunctions-tasksusenews3uriparametersforbedrockinvokemodeltask) | When enabled, use new props for S3 URI field in task definition of state machine for bedrock invoke model. | V2NEXT | (fix) |

<!-- END table -->

Expand Down Expand Up @@ -131,7 +133,9 @@ The following json shows the current recommended set of flags, as `cdk init` wou
"@aws-cdk/aws-eks:nodegroupNameAttribute": true,
"@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
"@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
"@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false
"@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
"@aws-cdk/aws-s3:keepNotificationInImportedBucket": false,
"@aws-cdk/aws-stepfunctions-tasks:useNewS3UriParametersForBedrockInvokeModelTask": true
}
}
```
Expand Down Expand Up @@ -1338,4 +1342,38 @@ property from the event object.
| 2.145.0 | `false` | `false` |


### @aws-cdk/aws-s3:keepNotificationInImportedBucket

*When enabled, Adding notifications to a bucket in the current stack will not remove notification from imported stack.* (fix)

Currently, adding notifications to a bucket where it was created by ourselves will override notification added where it is imported.

When this feature flag is enabled, adding notifications to a bucket in the current stack will only update notification defined in this stack.
Other notifications that are not managed by this stack will be kept.


| Since | Default | Recommended |
| ----- | ----- | ----- |
| (not in v1) | | |
| 2.155.0 | `false` | `false` |


### @aws-cdk/aws-stepfunctions-tasks:useNewS3UriParametersForBedrockInvokeModelTask

*When enabled, use new props for S3 URI field in task definition of state machine for bedrock invoke model.* (fix)

Currently, 'inputPath' and 'outputPath' from the TaskStateBase Props is being used under BedrockInvokeModelProps to define S3URI under 'input' and 'output' fields
of State Machine Task definition.

When this feature flag is enabled, specify newly introduced props 's3InputUri' and
's3OutputUri' to populate S3 uri under input and output fields in state machine task definition for Bedrock invoke model.


| Since | Default | Recommended |
| ----- | ----- | ----- |
| (not in v1) | | |
| V2NEXT | `true` | `true` |

**Compatibility with old behavior:** Disable the feature flag to use input and output path fields for s3 URI

<!-- END details -->
37 changes: 37 additions & 0 deletions packages/aws-cdk-lib/aws-stepfunctions-tasks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,11 +398,48 @@ const task = new tasks.BedrockInvokeModel(this, 'Prompt Model', {
names: sfn.JsonPath.stringAt('$.Body.results[0].outputText'),
},
});

```
### Using Input Path for S3 URI

Provide S3 URI as an input or output path to invoke a model

To specify the S3 URI as JSON path to your input or output fields, use props `s3InputUri` and `s3OutputUri` under BedrockInvokeModelProps and set
feature flag `@aws-cdk/aws-stepfunctions-tasks:useNewS3UriParametersForBedrockInvokeModelTask` to true.


If this flag is not enabled, the code will populate the S3Uri using `InputPath` and `OutputPath` fields, which is not recommended.

```ts

import * as bedrock from 'aws-cdk-lib/aws-bedrock';

const model = bedrock.FoundationModel.fromFoundationModelId(
this,
'Model',
bedrock.FoundationModelIdentifier.AMAZON_TITAN_TEXT_G1_EXPRESS_V1,
);

const task = new tasks.BedrockInvokeModel(this, 'Prompt Model', {
model,
input : { s3InputUri: sfn.JsonPath.stringAt('$.prompt') },
output: { s3OutputUri: sfn.JsonPath.stringAt('$.prompt') },
});

```

### Using Input Path

Provide S3 URI as an input or output path to invoke a model

Currently, input and output Path provided in the BedrockInvokeModelProps input is defined as S3URI field under task definition of state machine.
To modify the existing behaviour, set `@aws-cdk/aws-stepfunctions-tasks:useNewS3UriParametersForBedrockInvokeModelTask` to true.

If this feature flag is enabled, S3URI fields will be generated from other Props(`s3InputUri` and `s3OutputUri`), and the given inputPath, OutputPath will be rendered as
it is in the JSON task definition.

If the feature flag is set to `false`, the behavior will be to populate the S3Uri using the `InputPath` and `OutputPath` fields, which is not recommended.

```ts

import * as bedrock from 'aws-cdk-lib/aws-bedrock';
Expand Down
Loading

0 comments on commit 73506ae

Please sign in to comment.