Update modularization work with the latest main #661
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
…iadne 0.13 -> 0.17, fastapi 0.78 -> 0.92 (data-dot-all#379) ### Feature or Bugfix - Bugfix ### Detail - Upgrade starlette version: vulnerability found in starlette <0.25 (https://security.snyk.io/vuln/SNYK-PYTHON-STARLETTE-3319937). It does not affect data.all as we do not use `python-multipart` but nevertheless it is better to be in a non-vulnerable version. - Upgrade sqlalchemy version: the vulnerability is not stopping the CICD pipeline, but by upgrading we are able to use the latest version of alembic and we can revert the pinning of the version which happened in data-dot-all#354 - Upgrade ariadne to version 0.17.0: needed to support starlette 0.25.0 Higher version of ariadne==0.18.0 removes `PLAYGROUND_HTML` constant that we use in testing (Check [docs](https://ariadnegraphql.org/docs/0.17/constants-reference)) - Upgrade fastapi version to 0.92.0: needed to support starlette 0.25.0 (Version that supports this particular version of starlette, [docs](https://fastapi.tiangolo.com/release-notes/#0920)) ### Relates - data-dot-all#378 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - Added dependency to dataset S3 Bucket for the dataset crawler ### Relates - data-dot-all#384 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…d dashboards (data-dot-all#380) feat: generate url with dynamically domain name for quicksight embeded dashboards ### Feature or Bugfix - Feature ### Detail Previously one had to configure statically data.all domain name inside QuickSight to allow dashboards to be embeddable inside data.all. With new api it is possible to dynamically set in inside request. User no longer has to configure QuickSight before using it. ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bug-fix - Bug-fix ### Detail The latest version of dev docker images for FE and BE no-longer has `amazon-linux-extras`, this update changes the based of the docker image to use tag `2` (which is consistent with the rest of the images) instead of `latest` (which is a bad practice anyway -- see 2.4 [here](https://sysdig.com/blog/dockerfile-best-practices)) By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - Get credentials access token was missing groupUri input variable, as a result all users appeared as Unauthorized ### Relates - data-dot-all#389 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…s and errors out (data-dot-all#392) ### Feature or Bugfix - Bugfix ### Detail - The creation of S3 access points is asynchronous and can take more than 5 seconds to complete. When the share managers tries attaching the policy to the access points it fails in certain cases. This PR replaces the waiting time of 5 seconds for a while loop that checks that the access points has been created and if not it waits for 30s ### Relates - data-dot-all#388 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…ot-all#403) ### Feature or Bugfix - Bugfix ### Detail In PR data-dot-all#380 domains that are not custom domains are not taken into account. Reverting changes back and will continue that feature on the side. ### Relates - data-dot-all#400 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - Fix Worksheet View to only show shares to a environment-team specific to the team's IAM role (not consumption role) ### Relates - [data-dot-all#401 ] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail When we import a dataset stack, if the S3 location was already registered data.all does not create a storage location. The issue is that for datasets where data.all needs to create a storage location: 1. the first time that it creates the stack it detects that there is no storage location and it creates the corresponding CFN resource 2. the first time that it UPDATES the stack it detects the storage location from 1. and it deletes the CFN resource 3. the next time that it UPDATES the stack it does not detect any storage location (it was deleted in 2.) and it creates the CFN resource again. To fix this behavior, in V1.5 we will use Lambda custom resource to check the storage location and avoid CFN resources. But for previous versions, this PR includes: - in the method that checks the existence of an storage location, we filter by the roleArn of the location. If the roleArn is the `dataallPivotRole` then we assume that it was created by the dataset, which means that `existing_storage_location = False` I tested locally but with actual stacks being created. No additional policies are needed ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - V1.5.0 Features. Check each PR for a complete description of the feature. ### Detail - data-dot-all#292 - data-dot-all#355 - data-dot-all#337 - data-dot-all#427 - data-dot-all#431 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: kukushking <kukushkin.anton@gmail.com> Co-authored-by: Dariusz Osiennik <osiend@amazon.com> Co-authored-by: Noah Paige <69586985+noah-paige@users.noreply.github.com> Co-authored-by: Dennis Goldner <107395339+degoldner@users.noreply.github.com>
Bumps [flask](https://github.com/pallets/flask) from 2.0.3 to 2.3.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pallets/flask/releases">flask's releases</a>.</em></p> <blockquote> <h2>2.3.2</h2> <p>This is a security fix release for the 2.3.x release branch.</p> <ul> <li>Security advisory: <a href="https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq">https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq</a>, CVE-2023-30861</li> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/29?closed=1">https://github.com/pallets/flask/milestone/29?closed=1</a></li> </ul> <h2>2.3.1</h2> <p>This is a fix release for the 2.3.x release branch.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/28?closed=1">https://github.com/pallets/flask/milestone/28?closed=1</a></li> </ul> <h2>2.3.0</h2> <p>This is a feature release, which includes new features, removes previously deprecated code, and adds new deprecations. The 2.3.x branch is now the supported fix branch, the 2.2.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all dependencies and control upgrades. Test with warnings treated as errors to be able to adapt to deprecation warnings early.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/24?closed=1">https://github.com/pallets/flask/milestone/24?closed=1</a></li> </ul> <h2>2.2.4</h2> <p>This is a fix release for the 2.2.x release branch.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/27?closed=1">https://github.com/pallets/flask/milestone/27?closed=1</a></li> </ul> <h2>2.2.3</h2> <p>This is a fix release for the 2.2.x release branch.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/26?closed=1">https://github.com/pallets/flask/milestone/26?closed=1</a></li> </ul> <h2>2.2.2</h2> <p>This is a fix release for the <a href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a> feature release.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/25?closed=1">https://github.com/pallets/flask/milestone/25?closed=1</a></li> </ul> <h2>2.2.1</h2> <p>This is a fix release for the <a href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a> feature release.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/23?closed=1">https://github.com/pallets/flask/milestone/23?closed=1</a></li> </ul> <h2>2.2.0</h2> <p>This is a feature release, which includes new features and removes previously deprecated code. The 2.2.x branch is now the supported bug fix branch, the 2.1.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all dependencies and control upgrades.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/19?closed=1">https://github.com/pallets/flask/milestone/19?closed=1</a></li> </ul> <h2>2.1.3</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pallets/flask/blob/main/CHANGES.rst">flask's changelog</a>.</em></p> <blockquote> <h2>Version 2.3.2</h2> <p>Released 2023-05-01</p> <ul> <li>Set <code>Vary: Cookie</code> header when the session is accessed, modified, or refreshed.</li> <li>Update Werkzeug requirement to >=2.3.3 to apply recent bug fixes.</li> </ul> <h2>Version 2.3.1</h2> <p>Released 2023-04-25</p> <ul> <li>Restore deprecated <code>from flask import Markup</code>. :issue:<code>5084</code></li> </ul> <h2>Version 2.3.0</h2> <p>Released 2023-04-25</p> <ul> <li> <p>Drop support for Python 3.7. :pr:<code>5072</code></p> </li> <li> <p>Update minimum requirements to the latest versions: Werkzeug>=2.3.0, Jinja2>3.1.2, itsdangerous>=2.1.2, click>=8.1.3.</p> </li> <li> <p>Remove previously deprecated code. :pr:<code>4995</code></p> <ul> <li>The <code>push</code> and <code>pop</code> methods of the deprecated <code>_app_ctx_stack</code> and <code>_request_ctx_stack</code> objects are removed. <code>top</code> still exists to give extensions more time to update, but it will be removed.</li> <li>The <code>FLASK_ENV</code> environment variable, <code>ENV</code> config key, and <code>app.env</code> property are removed.</li> <li>The <code>session_cookie_name</code>, <code>send_file_max_age_default</code>, <code>use_x_sendfile</code>, <code>propagate_exceptions</code>, and <code>templates_auto_reload</code> properties on <code>app</code> are removed.</li> <li>The <code>JSON_AS_ASCII</code>, <code>JSON_SORT_KEYS</code>, <code>JSONIFY_MIMETYPE</code>, and <code>JSONIFY_PRETTYPRINT_REGULAR</code> config keys are removed.</li> <li>The <code>app.before_first_request</code> and <code>bp.before_app_first_request</code> decorators are removed.</li> <li><code>json_encoder</code> and <code>json_decoder</code> attributes on app and blueprint, and the corresponding <code>json.JSONEncoder</code> and <code>JSONDecoder</code> classes, are removed.</li> <li>The <code>json.htmlsafe_dumps</code> and <code>htmlsafe_dump</code> functions are removed.</li> <li>Calling setup methods on blueprints after registration is an error instead of a warning. :pr:<code>4997</code></li> </ul> </li> <li> <p>Importing <code>escape</code> and <code>Markup</code> from <code>flask</code> is deprecated. Import them directly from <code>markupsafe</code> instead. :pr:<code>4996</code></p> </li> <li> <p>The <code>app.got_first_request</code> property is deprecated. :pr:<code>4997</code></p> </li> <li> <p>The <code>locked_cached_property</code> decorator is deprecated. Use a lock inside the decorated function if locking is needed. :issue:<code>4993</code></p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pallets/flask/commit/f3b8f570545200c87465d18386f3fc9f2258307a"><code>f3b8f57</code></a> release version 2.3.2</li> <li><a href="https://github.com/pallets/flask/commit/c990bba94ab9bc81adf2d33e83c9a9628a2098f2"><code>c990bba</code></a> update min test env</li> <li><a href="https://github.com/pallets/flask/commit/adedb2a64ea7703369bc89021710b439ee79f8dc"><code>adedb2a</code></a> Merge pull request <a href="https://redirect.github.com/pallets/flask/issues/5101">#5101</a> from pallets/update-werkzeug</li> <li><a href="https://github.com/pallets/flask/commit/e1aedecdc689cc9a79131851dbdabf6c3bc49c9e"><code>e1aedec</code></a> update werkzeug</li> <li><a href="https://github.com/pallets/flask/commit/37badc3ce8b0665e3454547839196a676729309f"><code>37badc3</code></a> update changelog</li> <li><a href="https://github.com/pallets/flask/commit/70f906c51ce49c485f1d355703e9cc3386b1cc2b"><code>70f906c</code></a> Merge pull request from GHSA-m2qf-hxjv-5gpq</li> <li><a href="https://github.com/pallets/flask/commit/8705dd39c4fa563ea0fe0bf84c85da8fcc98b88d"><code>8705dd3</code></a> set <code>Vary: Cookie</code> header consistently for session</li> <li><a href="https://github.com/pallets/flask/commit/9532cba45d2339e90ebf04f178b1e4f2064e7328"><code>9532cba</code></a> fix mypy finding</li> <li><a href="https://github.com/pallets/flask/commit/0bc7356ce1ae11e633426902aba76d525f4523da"><code>0bc7356</code></a> start version 2.3.2</li> <li><a href="https://github.com/pallets/flask/commit/f07fb2b607c1eaa724ca9bfe43e2dc20d97d34de"><code>f07fb2b</code></a> Merge pull request <a href="https://redirect.github.com/pallets/flask/issues/5086">#5086</a> from pallets/release-2.3.1</li> <li>Additional commits viewable in <a href="https://github.com/pallets/flask/compare/2.0.3...2.3.2">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/awslabs/aws-dataall/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…-all#438) Bumps [flask](https://github.com/pallets/flask) from 2.0.3 to 2.3.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pallets/flask/releases">flask's releases</a>.</em></p> <blockquote> <h2>2.3.2</h2> <p>This is a security fix release for the 2.3.x release branch.</p> <ul> <li>Security advisory: <a href="https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq">https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq</a>, CVE-2023-30861</li> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/29?closed=1">https://github.com/pallets/flask/milestone/29?closed=1</a></li> </ul> <h2>2.3.1</h2> <p>This is a fix release for the 2.3.x release branch.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/28?closed=1">https://github.com/pallets/flask/milestone/28?closed=1</a></li> </ul> <h2>2.3.0</h2> <p>This is a feature release, which includes new features, removes previously deprecated code, and adds new deprecations. The 2.3.x branch is now the supported fix branch, the 2.2.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all dependencies and control upgrades. Test with warnings treated as errors to be able to adapt to deprecation warnings early.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/24?closed=1">https://github.com/pallets/flask/milestone/24?closed=1</a></li> </ul> <h2>2.2.4</h2> <p>This is a fix release for the 2.2.x release branch.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/27?closed=1">https://github.com/pallets/flask/milestone/27?closed=1</a></li> </ul> <h2>2.2.3</h2> <p>This is a fix release for the 2.2.x release branch.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/26?closed=1">https://github.com/pallets/flask/milestone/26?closed=1</a></li> </ul> <h2>2.2.2</h2> <p>This is a fix release for the <a href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a> feature release.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/25?closed=1">https://github.com/pallets/flask/milestone/25?closed=1</a></li> </ul> <h2>2.2.1</h2> <p>This is a fix release for the <a href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a> feature release.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/23?closed=1">https://github.com/pallets/flask/milestone/23?closed=1</a></li> </ul> <h2>2.2.0</h2> <p>This is a feature release, which includes new features and removes previously deprecated code. The 2.2.x branch is now the supported bug fix branch, the 2.1.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all dependencies and control upgrades.</p> <ul> <li>Changes: <a href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0</a></li> <li>Milestone: <a href="https://github.com/pallets/flask/milestone/19?closed=1">https://github.com/pallets/flask/milestone/19?closed=1</a></li> </ul> <h2>2.1.3</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pallets/flask/blob/main/CHANGES.rst">flask's changelog</a>.</em></p> <blockquote> <h2>Version 2.3.2</h2> <p>Released 2023-05-01</p> <ul> <li>Set <code>Vary: Cookie</code> header when the session is accessed, modified, or refreshed.</li> <li>Update Werkzeug requirement to >=2.3.3 to apply recent bug fixes.</li> </ul> <h2>Version 2.3.1</h2> <p>Released 2023-04-25</p> <ul> <li>Restore deprecated <code>from flask import Markup</code>. :issue:<code>5084</code></li> </ul> <h2>Version 2.3.0</h2> <p>Released 2023-04-25</p> <ul> <li> <p>Drop support for Python 3.7. :pr:<code>5072</code></p> </li> <li> <p>Update minimum requirements to the latest versions: Werkzeug>=2.3.0, Jinja2>3.1.2, itsdangerous>=2.1.2, click>=8.1.3.</p> </li> <li> <p>Remove previously deprecated code. :pr:<code>4995</code></p> <ul> <li>The <code>push</code> and <code>pop</code> methods of the deprecated <code>_app_ctx_stack</code> and <code>_request_ctx_stack</code> objects are removed. <code>top</code> still exists to give extensions more time to update, but it will be removed.</li> <li>The <code>FLASK_ENV</code> environment variable, <code>ENV</code> config key, and <code>app.env</code> property are removed.</li> <li>The <code>session_cookie_name</code>, <code>send_file_max_age_default</code>, <code>use_x_sendfile</code>, <code>propagate_exceptions</code>, and <code>templates_auto_reload</code> properties on <code>app</code> are removed.</li> <li>The <code>JSON_AS_ASCII</code>, <code>JSON_SORT_KEYS</code>, <code>JSONIFY_MIMETYPE</code>, and <code>JSONIFY_PRETTYPRINT_REGULAR</code> config keys are removed.</li> <li>The <code>app.before_first_request</code> and <code>bp.before_app_first_request</code> decorators are removed.</li> <li><code>json_encoder</code> and <code>json_decoder</code> attributes on app and blueprint, and the corresponding <code>json.JSONEncoder</code> and <code>JSONDecoder</code> classes, are removed.</li> <li>The <code>json.htmlsafe_dumps</code> and <code>htmlsafe_dump</code> functions are removed.</li> <li>Calling setup methods on blueprints after registration is an error instead of a warning. :pr:<code>4997</code></li> </ul> </li> <li> <p>Importing <code>escape</code> and <code>Markup</code> from <code>flask</code> is deprecated. Import them directly from <code>markupsafe</code> instead. :pr:<code>4996</code></p> </li> <li> <p>The <code>app.got_first_request</code> property is deprecated. :pr:<code>4997</code></p> </li> <li> <p>The <code>locked_cached_property</code> decorator is deprecated. Use a lock inside the decorated function if locking is needed. :issue:<code>4993</code></p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pallets/flask/commit/f3b8f570545200c87465d18386f3fc9f2258307a"><code>f3b8f57</code></a> release version 2.3.2</li> <li><a href="https://github.com/pallets/flask/commit/c990bba94ab9bc81adf2d33e83c9a9628a2098f2"><code>c990bba</code></a> update min test env</li> <li><a href="https://github.com/pallets/flask/commit/adedb2a64ea7703369bc89021710b439ee79f8dc"><code>adedb2a</code></a> Merge pull request <a href="https://redirect.github.com/pallets/flask/issues/5101">#5101</a> from pallets/update-werkzeug</li> <li><a href="https://github.com/pallets/flask/commit/e1aedecdc689cc9a79131851dbdabf6c3bc49c9e"><code>e1aedec</code></a> update werkzeug</li> <li><a href="https://github.com/pallets/flask/commit/37badc3ce8b0665e3454547839196a676729309f"><code>37badc3</code></a> update changelog</li> <li><a href="https://github.com/pallets/flask/commit/70f906c51ce49c485f1d355703e9cc3386b1cc2b"><code>70f906c</code></a> Merge pull request from GHSA-m2qf-hxjv-5gpq</li> <li><a href="https://github.com/pallets/flask/commit/8705dd39c4fa563ea0fe0bf84c85da8fcc98b88d"><code>8705dd3</code></a> set <code>Vary: Cookie</code> header consistently for session</li> <li><a href="https://github.com/pallets/flask/commit/9532cba45d2339e90ebf04f178b1e4f2064e7328"><code>9532cba</code></a> fix mypy finding</li> <li><a href="https://github.com/pallets/flask/commit/0bc7356ce1ae11e633426902aba76d525f4523da"><code>0bc7356</code></a> start version 2.3.2</li> <li><a href="https://github.com/pallets/flask/commit/f07fb2b607c1eaa724ca9bfe43e2dc20d97d34de"><code>f07fb2b</code></a> Merge pull request <a href="https://redirect.github.com/pallets/flask/issues/5086">#5086</a> from pallets/release-2.3.1</li> <li>Additional commits viewable in <a href="https://github.com/pallets/flask/compare/2.0.3...2.3.2">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/awslabs/aws-dataall/network/alerts). </details> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Nikita Podshivalov <nikpodsh@amazon.com>
… ACL access (data-dot-all#437) ### Feature or Bugfix - Bugfix ### Detail Solved bug 433, starting from April 2023 S3 default configurations changed, the default for s3 is set to disable ACL. Which is giving an issue for cloudfront logging on s3. The solution was to change the ownership of the object to object writer (enabling ACL for object writer as stated in cloudfront documentation). ### Relates [- <URL or Ticket>](data-dot-all#433) By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. Co-authored-by: akaitoua-sa <126820454+akaitoua-sa@users.noreply.github.com>
data-dot-all#446) ### Feature or Bugfix - Bugfix - Refactoring ### Detail Added `AWS_REGION` to the environment variables of the Docker containers for local development. Set both`AWS_DEFAULT_REGION` and `AWS_REGION` to their values set on the terminal where `docker-compose up` is run. If these values are not set, `eu-west-1` is used as default Another PR with better instructions to the github pages documentation (deploy locally) will follow. ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
data-dot-all#456) Bumps pymdown-extensions from 8.1.1 to 10.0.
…l#460) ### Feature or Bugfix - Bugfix ### Detail - Solve vulnerabilities found in starlette 0.25.0 ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - BugFix ### Detail - In line (https://github.com/awslabs/aws-dataall/blob/13a2fc082694600a0dacaa7e88d0d61ec950d753/deploy/configs/cognito_urls_config.py#L61) It checks for example.com where instead the right callback to check is ```https://example.com``` and that's why it doesn't get replaced during the configuration phase. ### Relates - data-dot-all#454 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - Add "aoss:APIAccessAll" to lambda/ECS task IAM roles required since May 10th (see message below). Fixes 403 errors from APIs. ``` [Action required] Amazon OpenSearch Serverless requires mandatory IAM permission for access to resources Starting May 10th, 2023, OpenSearch Serverless is mandating two new IAM permissions for collection resources. The two IAM permissions are "aoss:APIAccessAll" for Data Plane API access, and "aoss:DashboardsAccessAll" for Dashboards access from the browser. You are required to add these two IAM permissions for your OpenSearch Serverless "aoss:APIAccessAll" for Data Plane API access, and "aoss:DashboardsAccessAll" for Dashboards access. You must complete this action by May 9th, 2023. Failure to add the two new IAM permissions will result in 403 errors starting on May 10th, 2023 For a sample data-plane policy [here](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/security-iam-serverless.html#security_iam_id-based-policy-examples-data-plane.html) If you have any questions or concerns, please contact [AWS Support](https://aws.amazon.com/support) ``` By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…esql11'. (data-dot-all#466) ### Feature or Bugfix - Bugfix ### Detail - Update Aurora default Parameter Group to 'default.aurora-postgresql11'. Fixes an issue where the Aurora nested stack deploy in the data.all backend deploy would fail and/or block indefinitely due to 'default.aurora-postgresql10' mismatch with version 11 of the Aurora database engine. ### Relates - data-dot-all#465 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. Authored-by: rbernota <rbernota@yahooinc.com>
Bumps [requests](https://github.com/psf/requests) from 2.27.1 to 2.31.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/releases">requests's releases</a>.</em></p> <blockquote> <h2>v2.31.0</h2> <h2>2.31.0 (2023-05-22)</h2> <p><strong>Security</strong></p> <ul> <li> <p>Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of <code>Proxy-Authorization</code> headers to destination servers when following HTTPS redirects.</p> <p>When proxies are defined with user info (<a href="https://user:pass@proxy:8080">https://user:pass@proxy:8080</a>), Requests will construct a <code>Proxy-Authorization</code> header that is attached to the request to authenticate with the proxy.</p> <p>In cases where Requests receives a redirect response, it previously reattached the <code>Proxy-Authorization</code> header incorrectly, resulting in the value being sent through the tunneled connection to the destination server. Users who rely on defining their proxy credentials in the URL are <em>strongly</em> encouraged to upgrade to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy credentials once the change has been fully deployed.</p> <p>Users who do not use a proxy or do not supply their proxy credentials through the user information portion of their proxy URL are not subject to this vulnerability.</p> <p>Full details can be read in our <a href="https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q">Github Security Advisory</a> and <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-32681">CVE-2023-32681</a>.</p> </li> </ul> <h2>v2.30.0</h2> <h2>2.30.0 (2023-05-03)</h2> <p><strong>Dependencies</strong></p> <ul> <li> <p>⚠️ Added support for urllib3 2.0.⚠️ </p> <p>This may contain minor breaking changes so we advise careful testing and reviewing <a href="https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html">https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html</a> prior to upgrading.</p> <p>Users who wish to stay on urllib3 1.x can pin to <code>urllib3<2</code>.</p> </li> </ul> <h2>v2.29.0</h2> <h2>2.29.0 (2023-04-26)</h2> <p><strong>Improvements</strong></p> <ul> <li>Requests now defers chunked requests to the urllib3 implementation to improve standardization. (<a href="https://redirect.github.com/psf/requests/issues/6226">#6226</a>)</li> <li>Requests relaxes header component requirements to support bytes/str subclasses. (<a href="https://redirect.github.com/psf/requests/issues/6356">#6356</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/blob/main/HISTORY.md">requests's changelog</a>.</em></p> <blockquote> <h2>2.31.0 (2023-05-22)</h2> <p><strong>Security</strong></p> <ul> <li> <p>Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of <code>Proxy-Authorization</code> headers to destination servers when following HTTPS redirects.</p> <p>When proxies are defined with user info (<a href="https://user:pass@proxy:8080">https://user:pass@proxy:8080</a>), Requests will construct a <code>Proxy-Authorization</code> header that is attached to the request to authenticate with the proxy.</p> <p>In cases where Requests receives a redirect response, it previously reattached the <code>Proxy-Authorization</code> header incorrectly, resulting in the value being sent through the tunneled connection to the destination server. Users who rely on defining their proxy credentials in the URL are <em>strongly</em> encouraged to upgrade to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy credentials once the change has been fully deployed.</p> <p>Users who do not use a proxy or do not supply their proxy credentials through the user information portion of their proxy URL are not subject to this vulnerability.</p> <p>Full details can be read in our <a href="https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q">Github Security Advisory</a> and <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-32681">CVE-2023-32681</a>.</p> </li> </ul> <h2>2.30.0 (2023-05-03)</h2> <p><strong>Dependencies</strong></p> <ul> <li> <p>⚠️ Added support for urllib3 2.0.⚠️ </p> <p>This may contain minor breaking changes so we advise careful testing and reviewing <a href="https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html">https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html</a> prior to upgrading.</p> <p>Users who wish to stay on urllib3 1.x can pin to <code>urllib3<2</code>.</p> </li> </ul> <h2>2.29.0 (2023-04-26)</h2> <p><strong>Improvements</strong></p> <ul> <li>Requests now defers chunked requests to the urllib3 implementation to improve standardization. (<a href="https://redirect.github.com/psf/requests/issues/6226">#6226</a>)</li> <li>Requests relaxes header component requirements to support bytes/str subclasses. (<a href="https://redirect.github.com/psf/requests/issues/6356">#6356</a>)</li> </ul> <h2>2.28.2 (2023-01-12)</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/psf/requests/commit/147c8511ddbfa5e8f71bbf5c18ede0c4ceb3bba4"><code>147c851</code></a> v2.31.0</li> <li><a href="https://github.com/psf/requests/commit/74ea7cf7a6a27a4eeb2ae24e162bcc942a6706d5"><code>74ea7cf</code></a> Merge pull request from GHSA-j8r2-6x86-q33q</li> <li><a href="https://github.com/psf/requests/commit/302225334678490ec66b3614a9dddb8a02c5f4fe"><code>3022253</code></a> test on pypy 3.8 and pypy 3.9 on windows and macos (<a href="https://redirect.github.com/psf/requests/issues/6424">#6424</a>)</li> <li><a href="https://github.com/psf/requests/commit/b639e66c816514e40604d46f0088fbceec1a5149"><code>b639e66</code></a> test on py3.12 (<a href="https://redirect.github.com/psf/requests/issues/6448">#6448</a>)</li> <li><a href="https://github.com/psf/requests/commit/d3d504436ef0c2ac7ec8af13738b04dcc8c694be"><code>d3d5044</code></a> Fixed a small typo (<a href="https://redirect.github.com/psf/requests/issues/6452">#6452</a>)</li> <li><a href="https://github.com/psf/requests/commit/2ad18e0e10e7d7ecd5384c378f25ec8821a10a29"><code>2ad18e0</code></a> v2.30.0</li> <li><a href="https://github.com/psf/requests/commit/f2629e9e3c7ce3c3c8c025bcd8db551101cbc773"><code>f2629e9</code></a> Remove strict parameter (<a href="https://redirect.github.com/psf/requests/issues/6434">#6434</a>)</li> <li><a href="https://github.com/psf/requests/commit/87d63de8739263bbe17034fba2285c79780da7e8"><code>87d63de</code></a> v2.29.0</li> <li><a href="https://github.com/psf/requests/commit/51716c4ef390136b0d4b800ec7665dd5503e64fc"><code>51716c4</code></a> enable the warnings plugin (<a href="https://redirect.github.com/psf/requests/issues/6416">#6416</a>)</li> <li><a href="https://github.com/psf/requests/commit/a7da1ab3498b10ec3a3582244c94b2845f8a8e71"><code>a7da1ab</code></a> try on ubuntu 22.04 (<a href="https://redirect.github.com/psf/requests/issues/6418">#6418</a>)</li> <li>Additional commits viewable in <a href="https://github.com/psf/requests/compare/v2.27.1...v2.31.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/awslabs/aws-dataall/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…a-dot-all#470) [//]: # (dependabot-start)⚠️ **Dependabot is rebasing this PR**⚠️ Rebasing might not happen immediately, so don't worry if this takes some time. Note: if you make any changes to this PR yourself, they will take precedence over the rebase. --- [//]: # (dependabot-end) Bumps [requests](https://github.com/psf/requests) from 2.27.1 to 2.31.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/releases">requests's releases</a>.</em></p> <blockquote> <h2>v2.31.0</h2> <h2>2.31.0 (2023-05-22)</h2> <p><strong>Security</strong></p> <ul> <li> <p>Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of <code>Proxy-Authorization</code> headers to destination servers when following HTTPS redirects.</p> <p>When proxies are defined with user info (<a href="https://user:pass@proxy:8080">https://user:pass@proxy:8080</a>), Requests will construct a <code>Proxy-Authorization</code> header that is attached to the request to authenticate with the proxy.</p> <p>In cases where Requests receives a redirect response, it previously reattached the <code>Proxy-Authorization</code> header incorrectly, resulting in the value being sent through the tunneled connection to the destination server. Users who rely on defining their proxy credentials in the URL are <em>strongly</em> encouraged to upgrade to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy credentials once the change has been fully deployed.</p> <p>Users who do not use a proxy or do not supply their proxy credentials through the user information portion of their proxy URL are not subject to this vulnerability.</p> <p>Full details can be read in our <a href="https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q">Github Security Advisory</a> and <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-32681">CVE-2023-32681</a>.</p> </li> </ul> <h2>v2.30.0</h2> <h2>2.30.0 (2023-05-03)</h2> <p><strong>Dependencies</strong></p> <ul> <li> <p>⚠️ Added support for urllib3 2.0.⚠️ </p> <p>This may contain minor breaking changes so we advise careful testing and reviewing <a href="https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html">https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html</a> prior to upgrading.</p> <p>Users who wish to stay on urllib3 1.x can pin to <code>urllib3<2</code>.</p> </li> </ul> <h2>v2.29.0</h2> <h2>2.29.0 (2023-04-26)</h2> <p><strong>Improvements</strong></p> <ul> <li>Requests now defers chunked requests to the urllib3 implementation to improve standardization. (<a href="https://redirect.github.com/psf/requests/issues/6226">#6226</a>)</li> <li>Requests relaxes header component requirements to support bytes/str subclasses. (<a href="https://redirect.github.com/psf/requests/issues/6356">#6356</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/blob/main/HISTORY.md">requests's changelog</a>.</em></p> <blockquote> <h2>2.31.0 (2023-05-22)</h2> <p><strong>Security</strong></p> <ul> <li> <p>Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of <code>Proxy-Authorization</code> headers to destination servers when following HTTPS redirects.</p> <p>When proxies are defined with user info (<a href="https://user:pass@proxy:8080">https://user:pass@proxy:8080</a>), Requests will construct a <code>Proxy-Authorization</code> header that is attached to the request to authenticate with the proxy.</p> <p>In cases where Requests receives a redirect response, it previously reattached the <code>Proxy-Authorization</code> header incorrectly, resulting in the value being sent through the tunneled connection to the destination server. Users who rely on defining their proxy credentials in the URL are <em>strongly</em> encouraged to upgrade to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy credentials once the change has been fully deployed.</p> <p>Users who do not use a proxy or do not supply their proxy credentials through the user information portion of their proxy URL are not subject to this vulnerability.</p> <p>Full details can be read in our <a href="https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q">Github Security Advisory</a> and <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-32681">CVE-2023-32681</a>.</p> </li> </ul> <h2>2.30.0 (2023-05-03)</h2> <p><strong>Dependencies</strong></p> <ul> <li> <p>⚠️ Added support for urllib3 2.0.⚠️ </p> <p>This may contain minor breaking changes so we advise careful testing and reviewing <a href="https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html">https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html</a> prior to upgrading.</p> <p>Users who wish to stay on urllib3 1.x can pin to <code>urllib3<2</code>.</p> </li> </ul> <h2>2.29.0 (2023-04-26)</h2> <p><strong>Improvements</strong></p> <ul> <li>Requests now defers chunked requests to the urllib3 implementation to improve standardization. (<a href="https://redirect.github.com/psf/requests/issues/6226">#6226</a>)</li> <li>Requests relaxes header component requirements to support bytes/str subclasses. (<a href="https://redirect.github.com/psf/requests/issues/6356">#6356</a>)</li> </ul> <h2>2.28.2 (2023-01-12)</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/psf/requests/commit/147c8511ddbfa5e8f71bbf5c18ede0c4ceb3bba4"><code>147c851</code></a> v2.31.0</li> <li><a href="https://github.com/psf/requests/commit/74ea7cf7a6a27a4eeb2ae24e162bcc942a6706d5"><code>74ea7cf</code></a> Merge pull request from GHSA-j8r2-6x86-q33q</li> <li><a href="https://github.com/psf/requests/commit/302225334678490ec66b3614a9dddb8a02c5f4fe"><code>3022253</code></a> test on pypy 3.8 and pypy 3.9 on windows and macos (<a href="https://redirect.github.com/psf/requests/issues/6424">#6424</a>)</li> <li><a href="https://github.com/psf/requests/commit/b639e66c816514e40604d46f0088fbceec1a5149"><code>b639e66</code></a> test on py3.12 (<a href="https://redirect.github.com/psf/requests/issues/6448">#6448</a>)</li> <li><a href="https://github.com/psf/requests/commit/d3d504436ef0c2ac7ec8af13738b04dcc8c694be"><code>d3d5044</code></a> Fixed a small typo (<a href="https://redirect.github.com/psf/requests/issues/6452">#6452</a>)</li> <li><a href="https://github.com/psf/requests/commit/2ad18e0e10e7d7ecd5384c378f25ec8821a10a29"><code>2ad18e0</code></a> v2.30.0</li> <li><a href="https://github.com/psf/requests/commit/f2629e9e3c7ce3c3c8c025bcd8db551101cbc773"><code>f2629e9</code></a> Remove strict parameter (<a href="https://redirect.github.com/psf/requests/issues/6434">#6434</a>)</li> <li><a href="https://github.com/psf/requests/commit/87d63de8739263bbe17034fba2285c79780da7e8"><code>87d63de</code></a> v2.29.0</li> <li><a href="https://github.com/psf/requests/commit/51716c4ef390136b0d4b800ec7665dd5503e64fc"><code>51716c4</code></a> enable the warnings plugin (<a href="https://redirect.github.com/psf/requests/issues/6416">#6416</a>)</li> <li><a href="https://github.com/psf/requests/commit/a7da1ab3498b10ec3a3582244c94b2845f8a8e71"><code>a7da1ab</code></a> try on ubuntu 22.04 (<a href="https://redirect.github.com/psf/requests/issues/6418">#6418</a>)</li> <li>Additional commits viewable in <a href="https://github.com/psf/requests/compare/v2.27.1...v2.31.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/awslabs/aws-dataall/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ot-all#472) ### Feature or Bugfix - Bugfix ### Detail Remove the GitHub template development strategy from the possible types of data.all pipelines. The initial idea was to use the parameter `--template` from the [AWS DDK CLI](https://awslabs.github.io/aws-ddk/release/stable/api/cli/aws_ddk.html#ddk-init) which has been deprecated after its last major release (1.0.0). Using templates would enable customers to use any cookiecutter template directly in data.all. However, from the way that it was implemented it exposed a **vulnerability** in which customers could enter code instead of a template and perform cmd code injections in data.all ECS deployment task. Given that this is a high-risk issue + AWS DDK 1.0.0 does not use CLI + `templates` are not critical for any known customer we will remove it for the moment to ensure security. In the future we will revisit other ways of providing templates and accelerating data pipeline building in a secure manner. ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail Update Aurora engine version to 11.16. Fixes an issue where the Aurora nested stack deployment in the data.all backend which goes to deployment account would fail as AuroraPostgresEngineVersion.VER_10_18 is not compatible with parameter group default.aurora-postgresql11 ### Relates - data-dot-all#466 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…ns (data-dot-all#476) ### Feature or Bugfix - BugFix ### Detail - cdkproxy was using an outdated version of aws-cdk-lib which uses NODEJS_12_X for the AWS Custom Resources Lambda Functions, which are now not anymore supported in the AWS Accounts and causes failure of the creation of CloudFormation stacks in the case when you create a new DataSet Stack - The version change also triggered a minor type enforcement for the AccountPrincipal AccountId to be explicitly ```string``` ### Relates - data-dot-all#475 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail Custom resources created by the [cloudfront-authorization-at-edge](https://github.com/aws-samples/cloudfront-authorization-at-edge/blob/master/example-serverless-app-reuse/README.md) application used in data.all use node12 for the version of the application previously used (2.0.4). By upgrading to the latest version (2.1.5) the Lambda custom resources used also use node14 at runtime. After upgrading the semantic version, I performed the following tests: - [X] upgrade a pre-existing deployment (Lambdas node12) and check that the runtime has been updated to node14. See screenshot below. - [X] open userguide (where auth at edge is used) in pre-existing deployment - [X] execute GraphQL APIs in pre-existing deployment - [X] execute ES APIs in pre-existing deployment  - [X] deploy data.all from scratch and check that the Lambdas deployed use node14 - [X] open userguide (where auth at edge is used) in new deployment - [X] execute GraphQL APIs in new deployment - [X] execute ES APIs in new deployment  ### Relates - data-dot-all#479 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail The constant to define the dataallPivotRole missed an "a" and as a consequence the storage location for the Dataset was not registered ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…ta-dot-all#484) ### Feature or Bugfix - Bugfix - Refactoring ### Detail - The AWS Cloud Development Kit (CDK) Team recently identified an issue with the CDK Pipelines construct library that may result in unintended permissions being granted to authenticated users within your account. As of April 4, 2023, we have fixed the issue in version 1.200.0 [1] for CDK v1, and version 2.77.0 [2] for CDK v2. We strongly recommend you upgrade to one of these versions as soon as possible. Please refer to the Managing Dependencies documentation [3] in the CDK Developer Guide for instructions on how to perform the upgrade. Starting with versions 1.158.0 and 2.26.0, released May 30, 2022, the library creates a role that allows every identity in the same account with sts:AssumeRole permissions on Resource: * to assume it. This may result in granting privileges to authenticated users in your account allowing them to take pipeline actions beyond what was intended. ### Relates - N.A By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…-dot-all#485) ### Feature or Bugfix - Bugfix ### Detail - Added check and exception if there are open share requests on a consumption role or on a group that we are removing from an environment ### Relates - data-dot-all#450 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - fix how dynamic SQL with varying table names is generated ### Relates - <URL or Ticket> By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - Resolve nth-check in sub-dependencies to version 2.0.1 ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix <!-- please choose --> - Feature ### Detail Update import dataset documenation ### Relates - <URL or Ticket> By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
…-all#523) Limiting read-only access for ECS tasks deployed by data.all, reasoning detailed in: data-dot-all#426 Out of the 7 ECS tasks that gets deployed, only CDKProxy performs multiple write operations to the root filesystem. The workaround is to mount [bind volumes](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bind-mounts.html) to the proper paths in the filesystem: - **/dataall:** required for cdk deploy write operations (cdk.out, cdk.context.json) and further file write operations invoked through dataa.all business logic like archiving objects for the Glue profiling job - **/tmp:** required since by upon importing aws_cdk libraries a write operation happens to the /tmp folder Since the [currently used CDK class](https://docs.aws.amazon.com/cdk/api/v1/python/aws_cdk.aws_ecs/FargateTaskDefinition.html) for the Fargate task definition doesn't allow the definition of mount points, I had to replace the it with the [CFN-style class.](https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_ecs/CfnTaskDefinition.html) **[Testing]** I've created 2 environments and a dataset, and performed the sharing of the dataset between the 2 environments. I've verified, that: - the newly created CDKProxy task definition has the same attributes as the old one (with the further addition of the ReadOnlyRootFileSystem=True flag and the 2 new bind volumes) - the other 6 task definitions have ReadOnlyRootFileSystem=True enabled - all 7 tasks were executed without failure with the new setting - the security alert in security hub got archived By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Bumps [tough-cookie](https://github.com/salesforce/tough-cookie) from 4.1.2 to 4.1.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/salesforce/tough-cookie/releases">tough-cookie's releases</a>.</em></p> <blockquote> <h2>4.1.3</h2> <p>Security fix for Prototype Pollution discovery in <a href="https://redirect.github.com/salesforce/tough-cookie/issues/282">#282</a>. This is a minor release, although output from the <code>inspect</code> utility is affected by this change, we felt this change was important enough to be pushed into the next patch.</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/salesforce/tough-cookie/commit/4ff4d29f6cefd279a412b8d62a21142ebd410b36"><code>4ff4d29</code></a> 4.1.3 release preparation, update the package and lib/version to 4.1.3. (<a href="https://redirect.github.com/salesforce/tough-cookie/issues/284">#284</a>)</li> <li><a href="https://github.com/salesforce/tough-cookie/commit/12d474791bb856004e858fdb1c47b7608d09cf6e"><code>12d4747</code></a> Prevent prototype pollution in cookie memstore (<a href="https://redirect.github.com/salesforce/tough-cookie/issues/283">#283</a>)</li> <li><a href="https://github.com/salesforce/tough-cookie/commit/f06b72d1d447f33dfa6222c0a3c0c5e063558248"><code>f06b72d</code></a> Fix documentation for store.findCookies, missing allowSpecialUseDomain proper...</li> <li>See full diff in <a href="https://github.com/salesforce/tough-cookie/compare/v4.1.2...v4.1.3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/awslabs/aws-dataall/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Feature or Bugfix - Bugfix ### Detail - Fix query to check dataset shares before deleting a dataset The `existingSharedItems` property of `ShareObject` was not resolving correctly to a `True` or `False` value but rather to `None`. The impact is: 1. When checking a dataset's shares before deleting the dataset - we would not raise an exception for datasets that did in fact have existing shared items. 2. Additionally, when deleting a dataset with associated shared objects created - we would not successfully delete the associated share object and share object items and not clean up rows in the DB appropriately. This caused issues when viewing the Share Tab as the dataset does not exist but is still referenced in the outdated ShareObject and throws an error. ### Relates - data-dot-all#544 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix Refactoring ### Detail Optimized docker image size of `backend/docker/ecs/` from `3.6GB` to `1.9GB` By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - BugFix ### Detail For the release of v1.6 one permission is needed in CDK Synth CodeBuild stage. Since this permission will be added in the next CodeBuild stage of the deployment pipeline (UpdatePipeline and SelfMutate) the pipeline execution will fail when upgrading to V1.6. This PR adds the permission so if customers want to avoid manually adding the permission they can upgrade to v1.5.X and then to V1.6.0 in a 2 steps upgrade ### Relates Release v1.6.0 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Bumps [semver](https://github.com/npm/node-semver) from 5.7.1 to 5.7.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/npm/node-semver/releases">semver's releases</a>.</em></p> <blockquote> <h2>v5.7.2</h2> <h2><a href="https://github.com/npm/node-semver/compare/v5.7.1...v5.7.2">5.7.2</a> (2023-07-10)</h2> <h3>Bug Fixes</h3> <ul> <li><a href="https://github.com/npm/node-semver/commit/2f8fd41487acf380194579ecb6f8b1bbfe116be0"><code>2f8fd41</code></a> <a href="https://redirect.github.com/npm/node-semver/pull/585">#585</a> better handling of whitespace (<a href="https://redirect.github.com/npm/node-semver/issues/585">#585</a>) (<a href="https://github.com/joaomoreno"><code>@joaomoreno</code></a>, <a href="https://github.com/lukekarrys"><code>@lukekarrys</code></a>)</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/npm/node-semver/blob/v5.7.2/CHANGELOG.md">semver's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/npm/node-semver/compare/v5.7.1...v5.7.2">5.7.2</a> (2023-07-10)</h2> <h3>Bug Fixes</h3> <ul> <li><a href="https://github.com/npm/node-semver/commit/2f8fd41487acf380194579ecb6f8b1bbfe116be0"><code>2f8fd41</code></a> <a href="https://redirect.github.com/npm/node-semver/pull/585">#585</a> better handling of whitespace (<a href="https://redirect.github.com/npm/node-semver/issues/585">#585</a>) (<a href="https://github.com/joaomoreno"><code>@joaomoreno</code></a>, <a href="https://github.com/lukekarrys"><code>@lukekarrys</code></a>)</li> </ul> <h2>5.7</h2> <ul> <li>Add <code>minVersion</code> method</li> </ul> <h2>5.6</h2> <ul> <li>Move boolean <code>loose</code> param to an options object, with backwards-compatibility protection.</li> <li>Add ability to opt out of special prerelease version handling with the <code>includePrerelease</code> option flag.</li> </ul> <h2>5.5</h2> <ul> <li>Add version coercion capabilities</li> </ul> <h2>5.4</h2> <ul> <li>Add intersection checking</li> </ul> <h2>5.3</h2> <ul> <li>Add <code>minSatisfying</code> method</li> </ul> <h2>5.2</h2> <ul> <li>Add <code>prerelease(v)</code> that returns prerelease components</li> </ul> <h2>5.1</h2> <ul> <li>Add Backus-Naur for ranges</li> <li>Remove excessively cute inspection methods</li> </ul> <h2>5.0</h2> <ul> <li>Remove AMD/Browserified build artifacts</li> <li>Fix ltr and gtr when using the <code>*</code> range</li> <li>Fix for range <code>*</code> with a prerelease identifier</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/npm/node-semver/commit/f8cc313550691a50d9662d8c94f0c033717efd7d"><code>f8cc313</code></a> chore: release 5.7.2</li> <li><a href="https://github.com/npm/node-semver/commit/2f8fd41487acf380194579ecb6f8b1bbfe116be0"><code>2f8fd41</code></a> fix: better handling of whitespace (<a href="https://redirect.github.com/npm/node-semver/issues/585">#585</a>)</li> <li><a href="https://github.com/npm/node-semver/commit/deb5ad51bf58868fa243c1683775305fe9e0e365"><code>deb5ad5</code></a> chore: <code>@npmcli/template-oss</code><a href="https://github.com/4"><code>@4</code></a>.16.0</li> <li>See full diff in <a href="https://github.com/npm/node-semver/compare/v5.7.1...v5.7.2">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~lukekarrys">lukekarrys</a>, a new releaser for semver since your current version.</p> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/awslabs/aws-dataall/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Feature or Bugfix Release PR with the following list of features. Refer to each PR for the details ### Detail - data-dot-all#498 - data-dot-all#482 - data-dot-all#543 - data-dot-all#524 (which also solves data-dot-all#531) - data-dot-all#532 - data-dot-all#535 - data-dot-all#497 - data-dot-all#515 - data-dot-all#529 - data-dot-all#562 - data-dot-all#455 - data-dot-all#572 - data-dot-all#567 - data-dot-all#573 - data-dot-all#579 - data-dot-all#578 - data-dot-all#582 ### Breaking changes - release notes -⚠️ IMPORTANT: upgrade to a version >V1.5.0 before upgrading to V1.6 to avoid deletion of resources in custom resource deletion -⚠️ IMPORTANT: requires an update of environments and then datasets after upgrading. Either using cdk.json parameter `enable_update_dataall_stacks_in_cicd_pipeline`, waiting for overnight update stack task, or manually updating first environments and then datasets - CloudFront distribution replace for data-dot-all#529 - Additional EC2 permissions in CDK Synth CodeBuild stage for data-dot-all#543 --> this can be avoided by upgrading to v1.5.6 before upgrading to v1.6.0 - local development affected by more restrictive pivotRole trust policy ### Relates V1.6.0 release By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Gezim Musliaj <102723839+gmuslia@users.noreply.github.com> Co-authored-by: Noah Paige <69586985+noah-paige@users.noreply.github.com> Co-authored-by: nikpodsh <124577300+nikpodsh@users.noreply.github.com> Co-authored-by: chamcca <40579012+chamcca@users.noreply.github.com> Co-authored-by: Nikita Podshivalov <nikpodsh@amazon.com> Co-authored-by: dbalintx <132444646+dbalintx@users.noreply.github.com> Co-authored-by: mourya-33 <134511711+mourya-33@users.noreply.github.com>
### Feature or Bugfix - Bugfix ### Detail Fixes data-dot-all#589 by: - using CDK constructs to check the existence of an externalID in Secrets Manager - using boto3 calls using the CDK look up role in the deployment accounts to find an externalID in the Systems Manager Parameter Store ### Relates - data-dot-all#589 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Noah Paige <noahpaig@amazon.com>
### Feature or Bugfix - Bugfix ### Detail In case a custom domain and ACM certificate is configured for CloudFront, these parameters have to be passed differently to the newly introduced CloudFront distribution CDK class, introduced in [v1.6](data-dot-all@84c555e#diff-c65de5ab1eebd2a930807381430fa602793cc9966ab2a064cb29603162377030) ### Relates data-dot-all#603 Testing: Tested by, - creating a fresh deployment with v1.5 with a custom domain and SSL certificate - upgrading to v1.6, with the bugfix content from this PR 2 issues that are detailed in data-dot-all#603 arose (both related to the upgrade process), mitigation method of them is detailed there in the issue. By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - Add missing KMS key for canaries secrets ### Relates - data-dot-all#613 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Feature ### Detail - Extend the restricted NACLs parameter to allow for both the tooling VPC and the backend VPC By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - Remove small bug on the way we define the git release role - managed policies are attached after role creation - NOTE: The fix is already included in the `modularization-main` branch ### Relates - data-dot-all#617 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - get the prefix id list for S3 from the infra region. We need the prefix id to connect the dbmigration stage with the S3 bucket containing the migration scripts (add it in the security groups) ### Relates - data-dot-all#618 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- As part of v1.6 Data.All moved away from storing the externalID as a
rotated secret in Secret Manager and instead placed the external ID in
SSM Parameter Store.
- In the current implementation in v1.6.1 we check if the secret exists
and the ssm parameter does not and if these conditions are met the
secret value is retrieved and a new ssm parameter is set with the same
externalID
- The problem with the above is CDK uses dynamic references to resolve
the secret value (meaning in the first upgrade deployment we set ssm
parameter as ref to secret value and delete secret, in 2nd and so one
deployments it will fail with `Secrets Manager can't find the specified
secret.`)
- Alternatively we can not use the CDK bootstrap role, such as the look
up role, and boto3 SDK commands to retrieve the secret value during
`synth` because IAM permissions out of the box do not allow said actions
- This would theoretically be a way to overcome the dynamic reference
issue mentioned above
- This PR reverts to a more straightforward approach where we create a
new SSM Parameter if one does not exist already for the external ID and
does not reference the previously created secret externalID
- NOTE: In order to keep the same externalID and prevent additional
manual work to update the pivotRole's using this value one would have to
- retain the current externalID in Secret Manager (named
`dataall-externalId-{envname}`) from version <= 1.5X
- Run the upgrade to v1.6.1
- Replace the newly created SSM (parameter named
`/dataall/{envname}/pivotRole/externalId"`) with the original value for
external ID
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
### Feature or Bugfix <!-- please choose --> - Bugfix ### Detail - Add S3 Permissions so CDK Execution Policy is able to access CDK Asset Staging Bucket and auto create Pivot Role if enabled - Add details to instructions of linking environment to make Prerequisite steps more clear  ### Relates - data-dot-all#625 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix <!-- please choose --> - Bugfix ### Detail - Specify `SPARK_VERSION` as an environment variable for `pydeequ` before import - Add IAM Permissions to Dataset IAM Role to Allow for Glue Job logging in CloudWatch - Add LF Permissions to resolve insufficient permissions error thrown when looking for `default` database ### Relates By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
### Feature or Bugfix - Bugfix ### Detail - migration script for upgrade to V1.2 had a mistake and is affecting one customer. Basically the `devStrategy` and `devStages` values were not backfilled which causes nulls in the RDS table that are not allowed as this column should contain only non-null values. In this PR we modify that script for customers that have not updated yet. It is not 100% clear to me whether we should merge it, but I wanted to raise awareness of this issue here. ### Relates - data-dot-all#637 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
dlpzx
approved these changes
Aug 16, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge latest changes from main into modularization-main
It includes changes from #626, #630, #648, #649, and #651
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.