Skip to content

v1.5.0
Compare
Choose a tag to compare
@dlpzx dlpzx released this 25 Apr 12:15
· 552 commits to main since this release
v1.5.0
219553f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

What's Changed

New features:

Check each PR for a complete description of the feature.

  • Support OpenSearch Serverless by @kukushking in #292
  • Include Pivot Role as part of environment stack (avoiding manual pivot role creation) by @dlpzx in #355
  • Configurable restricted VPC for tooling resources by @dlpzx in #337
  • Better handling of missing default VPCs and added VPC creation for SageMaker domains by @dlpzx in #427

Bug-Fixes:

  • Fix dev Docker images base by @AmrSaber in #387
  • Fix get AWS credentials from environment tab by @dlpzx in #391
  • Added waiting conditions for slow creation of access points in sharing folders @dlpzx in #392
  • Fix shared dbs worksheet list (duplicates) by @noah-paige in #402
  • Fix sharing update of storage location by @dlpzx in #404
  • Backwards compatibility V1.5 fixes and documentation #431

⚠️ ⚠️ ⚠️ Important ⚠️ ⚠️ ⚠️

Breaking changes

Both the environment and the dataset stacks have been updated in this release. We need to update environment stacks BEFORE creating new datasets or updating existing ones in the environment. There are 3 ways of updating your stacks:

  1. Automatically (daily task) - There is an schedule ECS task that updates stacks daily. it has been modified to update environments and then datasets. Until the task is executed environments and datasets won't reflect the latest status of the code and creation of new datasets will fail.
  2. Automatically (add CICD stage) - We have introduced an optional CICD stage that triggers the ECS stack-updater task from the CICD pipeline. You need to set enable_update_dataall_stacks_in_cicd_pipeline to true in the cdk.json file to enable this stage (check #355 for more details). In this case the only downtime will be the time in which the CICD pipeline is running.
  3. Manually - In data.all console, go to the environment window > Stack tab > click on Update. Once it has completed, go to the required dataset window > Stack tab > click on Update.

Migrating to OpenSearch serverless

If you have deployed data.all with Amazon OpenSearch and would like to migrate to Amazon OpenSearch Serverless,
you would need to migrate the indexes to your new cluster. Although data.all currently does not provide an automated
migration tool, it is possible to do so manually using the following approaches:

Migrating from manual pivot roles to automatically created pivot roles

If you already have environments which use a manually created Pivot Role and want to upgrade to automatically create the pivot Roles as part of the environment stack, you just need to add the enable_pivot_role_auto_create parameter to cdk.json and set it to true. While the CICD pipeline is upgrading you will experience downtimes because backend, frontend and environment and dataset stacks are updated in different CodeBuild stages. For this upgrade we recommend you to enable_update_dataall_stacks_in_cicd_pipeline to update the environment and dataset stacks, otherwise you can either wait for the daily task or manually update all stacks as explained above.

Special thanks to @kukushking, @nikpodsh, @noah-paige , @AmrSaber for their contributions!
Full Changelog: v1.4.3...v1.5.0

Contributors

kukushking, AmrSaber, and 3 other contributors
Assets 2
Loading