Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

modularization: Mlstudio modularization #448

Closed
wants to merge 81 commits into from
Closed

modularization: Mlstudio modularization #448

wants to merge 81 commits into from

Conversation

dlpzx
Copy link
Contributor

@dlpzx dlpzx commented May 10, 2023 •
edited
Loading

Feature or Bugfix

  • Refactoring

Detail

First part of the modularization of ML Studio

  • Created SageMakerStudioService and move resolvers business logic into the service
  • Redefined resolvers and add request validators
  • Redefined db operations and added them into SageMakerRepository
  • Created sagemaker_studio_client with Sagemaker Studio SDK calls
  • Added CloudFormation stack in the cdk package of the module
  • Modified environment creation and edit api calls and views to read mlStudiosEnabled from environment parameters table
  • Fixed tests and added migration scripts

Additional:

  • Standardized naming of functions and objects to "sagemaker_studio_user" and avoid legacy "Notebook" references
  • Removed unused api definition for getSagemakerStudioUserApps
  • Split migration scripts for environments, worksheets and mlstudio

Relates

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

dependabot bot and others added 15 commits March 15, 2023 11:36
@dependabot
Bump webpack from 5.75.0 to 5.76.1 in /frontend (data-dot-all#371)
171a75c 
Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to
5.76.1.
@dlpzx
Upgrade sqlalchemy 13.16 -> 1.3.24 and starlette 0.19.1 -> 0.25.0, ar…
4002963 
…iadne 0.13 -> 0.17, fastapi 0.78 -> 0.92 (data-dot-all#379)

### Feature or Bugfix
- Bugfix

### Detail
- Upgrade starlette version: vulnerability found in starlette <0.25
(https://security.snyk.io/vuln/SNYK-PYTHON-STARLETTE-3319937). It does
not affect data.all as we do not use `python-multipart` but nevertheless
it is better to be in a non-vulnerable version.
- Upgrade sqlalchemy version: the vulnerability is not stopping the CICD
pipeline, but by upgrading we are able to use the latest version of
alembic and we can revert the pinning of the version which happened in
data-dot-all#354
- Upgrade ariadne to version 0.17.0: needed to support starlette 0.25.0
Higher version of ariadne==0.18.0 removes `PLAYGROUND_HTML` constant
that we use in testing (Check
[docs](https://ariadnegraphql.org/docs/0.17/constants-reference))
- Upgrade fastapi version to 0.92.0: needed to support starlette 0.25.0
(Version that supports this particular version of starlette,
[docs](https://fastapi.tiangolo.com/release-notes/#0920))
### Relates
- data-dot-all#378 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx
Add dependency in dataset stack (data-dot-all#385)
79f0e4c 
### Feature or Bugfix
- Bugfix

### Detail
- Added dependency to dataset S3 Bucket for the dataset crawler

### Relates
- data-dot-all#384 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@wolanlu
feat: generate url with dynamically domain name for quicksight embede…
95c8619 
…d dashboards (data-dot-all#380)

feat: generate url with dynamically domain name for quicksight embeded
dashboards

### Feature or Bugfix
- Feature

### Detail
Previously one had to configure statically data.all domain name inside
QuickSight to allow dashboards to be embeddable inside data.all. With
new api it is possible to dynamically set in inside request. User no
longer has to configure QuickSight before using it.

### Relates

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@AmrSaber
fix: dev docker images base (data-dot-all#387)
f677371 
### Feature or Bug-fix
- Bug-fix

### Detail
The latest version of dev docker images for FE and BE no-longer has
`amazon-linux-extras`, this update changes the based of the docker image
to use tag `2` (which is consistent with the rest of the images) instead
of `latest` (which is a bad practice anyway -- see 2.4
[here](https://sysdig.com/blog/dockerfile-best-practices))


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx
Added missing groupUri from get credentials (data-dot-all#391)
e01e014 
### Feature or Bugfix
- Bugfix

### Detail
- Get credentials access token was missing groupUri input variable, as a
result all users appeared as Unauthorized

### Relates
- data-dot-all#389 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx
388 race condition occurs when adding folder to shared items in share…
9057116 
…s and errors out (data-dot-all#392)

### Feature or Bugfix
- Bugfix

### Detail
- The creation of S3 access points is asynchronous and can take more
than 5 seconds to complete. When the share managers tries attaching the
policy to the access points it fails in certain cases. This PR replaces
the waiting time of 5 seconds for a while loop that checks that the
access points has been created and if not it waits for 30s

### Relates
- data-dot-all#388 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx
hotfix: Revert PR on custom url quicksight embedding sessions (data-d…
d191b26 
…ot-all#403)

### Feature or Bugfix
- Bugfix

### Detail
In PR data-dot-all#380 domains that are not custom domains are not taken into
account. Reverting changes back and will continue that feature on the
side.

### Relates
- data-dot-all#400 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@noah-paige
401 shared dbs worksheet list (data-dot-all#402)
a17f12a 
### Feature or Bugfix
- Bugfix

### Detail
- Fix Worksheet View to only show shares to a environment-team specific
to the team's IAM role (not consumption role)

### Relates
- [data-dot-all#401 ]

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx
Fix sharing update (data-dot-all#404)
6460986 
### Feature or Bugfix
- Bugfix

### Detail
When we import a dataset stack, if the S3 location was already
registered data.all does not create a storage location. The issue is
that for datasets where data.all needs to create a storage location:
1. the first time that it creates the stack it detects that there is no
storage location and it creates the corresponding CFN resource
2. the first time that it UPDATES the stack it detects the storage
location from 1. and it deletes the CFN resource
3. the next time that it UPDATES the stack it does not detect any
storage location (it was deleted in 2.) and it creates the CFN resource
again.

To fix this behavior, in V1.5 we will use Lambda custom resource to
check the storage location and avoid CFN resources. But for previous
versions, this PR includes:
- in the method that checks the existence of an storage location, we
filter by the roleArn of the location. If the roleArn is the
`dataallPivotRole` then we assume that it was created by the dataset,
which means that `existing_storage_location = False`

I tested locally but with actual stacks being created. No additional
policies are needed

### Relates

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx @kukushking
@dosiennik @noah-paige @degoldner
V1.5.0 Features (data-dot-all#409)
219553f 
### Feature or Bugfix
- V1.5.0 Features. Check each PR for a complete description of the
feature.

### Detail
- data-dot-all#292 
- data-dot-all#355 
- data-dot-all#337 
- data-dot-all#427 
- data-dot-all#431 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

---------

Co-authored-by: kukushking <kukushkin.anton@gmail.com>
Co-authored-by: Dariusz Osiennik <osiend@amazon.com>
Co-authored-by: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Co-authored-by: Dennis Goldner <107395339+degoldner@users.noreply.github.com>
@dependabot
Bump flask from 2.0.3 to 2.3.2 in /backend (data-dot-all#439)
7ad099f 
Bumps [flask](https://github.com/pallets/flask) from 2.0.3 to 2.3.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/flask/releases">flask's
releases</a>.</em></p>
<blockquote>
<h2>2.3.2</h2>
<p>This is a security fix release for the 2.3.x release branch.</p>
<ul>
<li>Security advisory: <a
href="https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq">https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq</a>,
CVE-2023-30861</li>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/29?closed=1">https://github.com/pallets/flask/milestone/29?closed=1</a></li>
</ul>
<h2>2.3.1</h2>
<p>This is a fix release for the 2.3.x release branch.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/28?closed=1">https://github.com/pallets/flask/milestone/28?closed=1</a></li>
</ul>
<h2>2.3.0</h2>
<p>This is a feature release, which includes new features, removes
previously deprecated code, and adds new deprecations. The 2.3.x branch
is now the supported fix branch, the 2.2.x branch will become a tag
marking the end of support for that branch. We encourage everyone to
upgrade, and to use a tool such as <a
href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all
dependencies and control upgrades. Test with warnings treated as errors
to be able to adapt to deprecation warnings early.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/24?closed=1">https://github.com/pallets/flask/milestone/24?closed=1</a></li>
</ul>
<h2>2.2.4</h2>
<p>This is a fix release for the 2.2.x release branch.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/27?closed=1">https://github.com/pallets/flask/milestone/27?closed=1</a></li>
</ul>
<h2>2.2.3</h2>
<p>This is a fix release for the 2.2.x release branch.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/26?closed=1">https://github.com/pallets/flask/milestone/26?closed=1</a></li>
</ul>
<h2>2.2.2</h2>
<p>This is a fix release for the <a
href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a>
feature release.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/25?closed=1">https://github.com/pallets/flask/milestone/25?closed=1</a></li>
</ul>
<h2>2.2.1</h2>
<p>This is a fix release for the <a
href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a>
feature release.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/23?closed=1">https://github.com/pallets/flask/milestone/23?closed=1</a></li>
</ul>
<h2>2.2.0</h2>
<p>This is a feature release, which includes new features and removes
previously deprecated code. The 2.2.x branch is now the supported bug
fix branch, the 2.1.x branch will become a tag marking the end of
support for that branch. We encourage everyone to upgrade, and to use a
tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a>
to pin all dependencies and control upgrades.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/19?closed=1">https://github.com/pallets/flask/milestone/19?closed=1</a></li>
</ul>
<h2>2.1.3</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/flask/blob/main/CHANGES.rst">flask's
changelog</a>.</em></p>
<blockquote>
<h2>Version 2.3.2</h2>
<p>Released 2023-05-01</p>
<ul>
<li>Set <code>Vary: Cookie</code> header when the session is accessed,
modified, or refreshed.</li>
<li>Update Werkzeug requirement to &gt;=2.3.3 to apply recent bug
fixes.</li>
</ul>
<h2>Version 2.3.1</h2>
<p>Released 2023-04-25</p>
<ul>
<li>Restore deprecated <code>from flask import Markup</code>.
:issue:<code>5084</code></li>
</ul>
<h2>Version 2.3.0</h2>
<p>Released 2023-04-25</p>
<ul>
<li>
<p>Drop support for Python 3.7. :pr:<code>5072</code></p>
</li>
<li>
<p>Update minimum requirements to the latest versions:
Werkzeug&gt;=2.3.0, Jinja2&gt;3.1.2,
itsdangerous&gt;=2.1.2, click&gt;=8.1.3.</p>
</li>
<li>
<p>Remove previously deprecated code. :pr:<code>4995</code></p>
<ul>
<li>The <code>push</code> and <code>pop</code> methods of the deprecated
<code>_app_ctx_stack</code> and
<code>_request_ctx_stack</code> objects are removed. <code>top</code>
still exists to give
extensions more time to update, but it will be removed.</li>
<li>The <code>FLASK_ENV</code> environment variable, <code>ENV</code>
config key, and <code>app.env</code>
property are removed.</li>
<li>The <code>session_cookie_name</code>,
<code>send_file_max_age_default</code>, <code>use_x_sendfile</code>,
<code>propagate_exceptions</code>, and
<code>templates_auto_reload</code> properties on <code>app</code>
are removed.</li>
<li>The <code>JSON_AS_ASCII</code>, <code>JSON_SORT_KEYS</code>,
<code>JSONIFY_MIMETYPE</code>, and
<code>JSONIFY_PRETTYPRINT_REGULAR</code> config keys are removed.</li>
<li>The <code>app.before_first_request</code> and
<code>bp.before_app_first_request</code> decorators
are removed.</li>
<li><code>json_encoder</code> and <code>json_decoder</code> attributes
on app and blueprint, and the
corresponding <code>json.JSONEncoder</code> and <code>JSONDecoder</code>
classes, are removed.</li>
<li>The <code>json.htmlsafe_dumps</code> and <code>htmlsafe_dump</code>
functions are removed.</li>
<li>Calling setup methods on blueprints after registration is an error
instead of a
warning. :pr:<code>4997</code></li>
</ul>
</li>
<li>
<p>Importing <code>escape</code> and <code>Markup</code> from
<code>flask</code> is deprecated. Import them
directly from <code>markupsafe</code> instead. :pr:<code>4996</code></p>
</li>
<li>
<p>The <code>app.got_first_request</code> property is deprecated.
:pr:<code>4997</code></p>
</li>
<li>
<p>The <code>locked_cached_property</code> decorator is deprecated. Use
a lock inside the
decorated function if locking is needed. :issue:<code>4993</code></p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/flask/commit/f3b8f570545200c87465d18386f3fc9f2258307a"><code>f3b8f57</code></a>
release version 2.3.2</li>
<li><a
href="https://github.com/pallets/flask/commit/c990bba94ab9bc81adf2d33e83c9a9628a2098f2"><code>c990bba</code></a>
update min test env</li>
<li><a
href="https://github.com/pallets/flask/commit/adedb2a64ea7703369bc89021710b439ee79f8dc"><code>adedb2a</code></a>
Merge pull request <a
href="https://redirect.github.com/pallets/flask/issues/5101">#5101</a>
from pallets/update-werkzeug</li>
<li><a
href="https://github.com/pallets/flask/commit/e1aedecdc689cc9a79131851dbdabf6c3bc49c9e"><code>e1aedec</code></a>
update werkzeug</li>
<li><a
href="https://github.com/pallets/flask/commit/37badc3ce8b0665e3454547839196a676729309f"><code>37badc3</code></a>
update changelog</li>
<li><a
href="https://github.com/pallets/flask/commit/70f906c51ce49c485f1d355703e9cc3386b1cc2b"><code>70f906c</code></a>
Merge pull request from GHSA-m2qf-hxjv-5gpq</li>
<li><a
href="https://github.com/pallets/flask/commit/8705dd39c4fa563ea0fe0bf84c85da8fcc98b88d"><code>8705dd3</code></a>
set <code>Vary: Cookie</code> header consistently for session</li>
<li><a
href="https://github.com/pallets/flask/commit/9532cba45d2339e90ebf04f178b1e4f2064e7328"><code>9532cba</code></a>
fix mypy finding</li>
<li><a
href="https://github.com/pallets/flask/commit/0bc7356ce1ae11e633426902aba76d525f4523da"><code>0bc7356</code></a>
start version 2.3.2</li>
<li><a
href="https://github.com/pallets/flask/commit/f07fb2b607c1eaa724ca9bfe43e2dc20d97d34de"><code>f07fb2b</code></a>
Merge pull request <a
href="https://redirect.github.com/pallets/flask/issues/5086">#5086</a>
from pallets/release-2.3.1</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/flask/compare/2.0.3...2.3.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=flask&package-manager=pip&previous-version=2.0.3&new-version=2.3.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/awslabs/aws-dataall/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot @nikpodsh
Bump flask from 2.0.3 to 2.3.2 in /backend/dataall/cdkproxy (data-dot…
e4b3e73 
…-all#438)

Bumps [flask](https://github.com/pallets/flask) from 2.0.3 to 2.3.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/flask/releases">flask's
releases</a>.</em></p>
<blockquote>
<h2>2.3.2</h2>
<p>This is a security fix release for the 2.3.x release branch.</p>
<ul>
<li>Security advisory: <a
href="https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq">https://github.com/pallets/flask/security/advisories/GHSA-m2qf-hxjv-5gpq</a>,
CVE-2023-30861</li>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-2</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/29?closed=1">https://github.com/pallets/flask/milestone/29?closed=1</a></li>
</ul>
<h2>2.3.1</h2>
<p>This is a fix release for the 2.3.x release branch.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-1</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/28?closed=1">https://github.com/pallets/flask/milestone/28?closed=1</a></li>
</ul>
<h2>2.3.0</h2>
<p>This is a feature release, which includes new features, removes
previously deprecated code, and adds new deprecations. The 2.3.x branch
is now the supported fix branch, the 2.2.x branch will become a tag
marking the end of support for that branch. We encourage everyone to
upgrade, and to use a tool such as <a
href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all
dependencies and control upgrades. Test with warnings treated as errors
to be able to adapt to deprecation warnings early.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0">https://flask.palletsprojects.com/en/2.3.x/changes/#version-2-3-0</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/24?closed=1">https://github.com/pallets/flask/milestone/24?closed=1</a></li>
</ul>
<h2>2.2.4</h2>
<p>This is a fix release for the 2.2.x release branch.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-4</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/27?closed=1">https://github.com/pallets/flask/milestone/27?closed=1</a></li>
</ul>
<h2>2.2.3</h2>
<p>This is a fix release for the 2.2.x release branch.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-3</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/26?closed=1">https://github.com/pallets/flask/milestone/26?closed=1</a></li>
</ul>
<h2>2.2.2</h2>
<p>This is a fix release for the <a
href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a>
feature release.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-2</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/25?closed=1">https://github.com/pallets/flask/milestone/25?closed=1</a></li>
</ul>
<h2>2.2.1</h2>
<p>This is a fix release for the <a
href="https://github.com/pallets/flask/releases/tag/2.2.0">2.2.0</a>
feature release.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-1</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/23?closed=1">https://github.com/pallets/flask/milestone/23?closed=1</a></li>
</ul>
<h2>2.2.0</h2>
<p>This is a feature release, which includes new features and removes
previously deprecated code. The 2.2.x branch is now the supported bug
fix branch, the 2.1.x branch will become a tag marking the end of
support for that branch. We encourage everyone to upgrade, and to use a
tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a>
to pin all dependencies and control upgrades.</p>
<ul>
<li>Changes: <a
href="https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0">https://flask.palletsprojects.com/en/2.2.x/changes/#version-2-2-0</a></li>
<li>Milestone: <a
href="https://github.com/pallets/flask/milestone/19?closed=1">https://github.com/pallets/flask/milestone/19?closed=1</a></li>
</ul>
<h2>2.1.3</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/flask/blob/main/CHANGES.rst">flask's
changelog</a>.</em></p>
<blockquote>
<h2>Version 2.3.2</h2>
<p>Released 2023-05-01</p>
<ul>
<li>Set <code>Vary: Cookie</code> header when the session is accessed,
modified, or refreshed.</li>
<li>Update Werkzeug requirement to &gt;=2.3.3 to apply recent bug
fixes.</li>
</ul>
<h2>Version 2.3.1</h2>
<p>Released 2023-04-25</p>
<ul>
<li>Restore deprecated <code>from flask import Markup</code>.
:issue:<code>5084</code></li>
</ul>
<h2>Version 2.3.0</h2>
<p>Released 2023-04-25</p>
<ul>
<li>
<p>Drop support for Python 3.7. :pr:<code>5072</code></p>
</li>
<li>
<p>Update minimum requirements to the latest versions:
Werkzeug&gt;=2.3.0, Jinja2&gt;3.1.2,
itsdangerous&gt;=2.1.2, click&gt;=8.1.3.</p>
</li>
<li>
<p>Remove previously deprecated code. :pr:<code>4995</code></p>
<ul>
<li>The <code>push</code> and <code>pop</code> methods of the deprecated
<code>_app_ctx_stack</code> and
<code>_request_ctx_stack</code> objects are removed. <code>top</code>
still exists to give
extensions more time to update, but it will be removed.</li>
<li>The <code>FLASK_ENV</code> environment variable, <code>ENV</code>
config key, and <code>app.env</code>
property are removed.</li>
<li>The <code>session_cookie_name</code>,
<code>send_file_max_age_default</code>, <code>use_x_sendfile</code>,
<code>propagate_exceptions</code>, and
<code>templates_auto_reload</code> properties on <code>app</code>
are removed.</li>
<li>The <code>JSON_AS_ASCII</code>, <code>JSON_SORT_KEYS</code>,
<code>JSONIFY_MIMETYPE</code>, and
<code>JSONIFY_PRETTYPRINT_REGULAR</code> config keys are removed.</li>
<li>The <code>app.before_first_request</code> and
<code>bp.before_app_first_request</code> decorators
are removed.</li>
<li><code>json_encoder</code> and <code>json_decoder</code> attributes
on app and blueprint, and the
corresponding <code>json.JSONEncoder</code> and <code>JSONDecoder</code>
classes, are removed.</li>
<li>The <code>json.htmlsafe_dumps</code> and <code>htmlsafe_dump</code>
functions are removed.</li>
<li>Calling setup methods on blueprints after registration is an error
instead of a
warning. :pr:<code>4997</code></li>
</ul>
</li>
<li>
<p>Importing <code>escape</code> and <code>Markup</code> from
<code>flask</code> is deprecated. Import them
directly from <code>markupsafe</code> instead. :pr:<code>4996</code></p>
</li>
<li>
<p>The <code>app.got_first_request</code> property is deprecated.
:pr:<code>4997</code></p>
</li>
<li>
<p>The <code>locked_cached_property</code> decorator is deprecated. Use
a lock inside the
decorated function if locking is needed. :issue:<code>4993</code></p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/flask/commit/f3b8f570545200c87465d18386f3fc9f2258307a"><code>f3b8f57</code></a>
release version 2.3.2</li>
<li><a
href="https://github.com/pallets/flask/commit/c990bba94ab9bc81adf2d33e83c9a9628a2098f2"><code>c990bba</code></a>
update min test env</li>
<li><a
href="https://github.com/pallets/flask/commit/adedb2a64ea7703369bc89021710b439ee79f8dc"><code>adedb2a</code></a>
Merge pull request <a
href="https://redirect.github.com/pallets/flask/issues/5101">#5101</a>
from pallets/update-werkzeug</li>
<li><a
href="https://github.com/pallets/flask/commit/e1aedecdc689cc9a79131851dbdabf6c3bc49c9e"><code>e1aedec</code></a>
update werkzeug</li>
<li><a
href="https://github.com/pallets/flask/commit/37badc3ce8b0665e3454547839196a676729309f"><code>37badc3</code></a>
update changelog</li>
<li><a
href="https://github.com/pallets/flask/commit/70f906c51ce49c485f1d355703e9cc3386b1cc2b"><code>70f906c</code></a>
Merge pull request from GHSA-m2qf-hxjv-5gpq</li>
<li><a
href="https://github.com/pallets/flask/commit/8705dd39c4fa563ea0fe0bf84c85da8fcc98b88d"><code>8705dd3</code></a>
set <code>Vary: Cookie</code> header consistently for session</li>
<li><a
href="https://github.com/pallets/flask/commit/9532cba45d2339e90ebf04f178b1e4f2064e7328"><code>9532cba</code></a>
fix mypy finding</li>
<li><a
href="https://github.com/pallets/flask/commit/0bc7356ce1ae11e633426902aba76d525f4523da"><code>0bc7356</code></a>
start version 2.3.2</li>
<li><a
href="https://github.com/pallets/flask/commit/f07fb2b607c1eaa724ca9bfe43e2dc20d97d34de"><code>f07fb2b</code></a>
Merge pull request <a
href="https://redirect.github.com/pallets/flask/issues/5086">#5086</a>
from pallets/release-2.3.1</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/flask/compare/2.0.3...2.3.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=flask&package-manager=pip&previous-version=2.0.3&new-version=2.3.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/awslabs/aws-dataall/network/alerts).

</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nikita Podshivalov <nikpodsh@amazon.com>
@akaitoua @akaitoua-sa
solve deployment bug data-dot-all#433 CloudFront logs does not enable…
2a319ca 
… ACL access (data-dot-all#437)

### Feature or Bugfix
- Bugfix

### Detail
Solved bug 433, starting from April 2023 S3 default configurations
changed, the default for s3 is set to disable ACL. Which is giving an
issue for cloudfront logging on s3. The solution was to change the
ownership of the object to object writer (enabling ACL for object writer
as stated in cloudfront documentation).

### Relates
[- <URL or Ticket>](data-dot-all#433)

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

Co-authored-by: akaitoua-sa <126820454+akaitoua-sa@users.noreply.github.com>
@dlpzx
Modify docker-compose yaml to read region and default region from env… (
13a2fc0 
data-dot-all#446)

### Feature or Bugfix
- Bugfix
- Refactoring

### Detail
Added `AWS_REGION` to the environment variables of the Docker containers
for local development. Set both`AWS_DEFAULT_REGION` and `AWS_REGION` to
their values set on the terminal where `docker-compose up` is run.
If these values are not set, `eu-west-1` is used as default
Another PR with better instructions to the github pages documentation
(deploy locally) will follow.

### Relates

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@dlpzx dlpzx added the type: modularization Code refactoring project label May 10, 2023
@dlpzx dlpzx requested review from nikpodsh and dbalintx May 11, 2023 07:59
@dlpzx dlpzx marked this pull request as ready for review May 11, 2023 07:59
nikpodsh
nikpodsh reviewed May 12, 2023
View reviewed changes
backend/dataall/modules/mlstudio/services/permissions.py Outdated
ENVIRONMENT_ALL.append(LIST_ENVIRONMENT_SGMSTUDIO_USERS)
ENVIRONMENT_INVITED.append(CREATE_SGMSTUDIO_USER)
ENVIRONMENT_INVITED.append(LIST_ENVIRONMENT_SGMSTUDIO_USERS)
ENVIRONMENT_INVITATION_REQUEST.append(CREATE_SGMSTUDIO_USER)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LIST_ENVIRONMENT_SGMSTUDIO_USERS should be in ENVIRONMENT_INVITATION_REQUEST. This will allow us to remove the code related to ml studios permissions from db.api.Environment#validate_permissions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nikpodsh I am not sure about this, maybe I understood it wrong, let's see :)
ENVIRONMENT_ALL = all permissions in the environment, given to environment Admin group
ENVIRONMENT_INVITED = all permissions that can actually be granted to invited groups in an environment. Used in db.api.Environment#validate_permissions, here we add the selectable + some additional ones and then check against ENVIRONMENT_INVITED
ENVIRONMENT_INVITATION_REQUEST = all permissions that are selectable to invited groups in an environment. Used in backend.dataall.db.api.environment.Environment.list_group_invitation_permissions to plot selectable permissions.

I understand that those additional ones are the ones that you want to get rid of, but we cannot do this without checking because we plot the results of list_group_invitation_permissions in the frontend

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we deleted this line in environment.py:

if permissions.CREATE_SGMSTUDIO_NOTEBOOK in g_permissions:
            g_permissions.append(permissions.LIST_ENVIRONMENT_SGMSTUDIO_NOTEBOOKS)

We will always have issue in this line:

if g_permissions not in permissions.ENVIRONMENT_INVITED:
       exceptions.PermissionUnauthorized(
                action='INVITE_TEAM', group_name=group, resource_uri=uri
            )

P.S. just noticed that we don't raise the error (just create it). And in many places too

nikpodsh
nikpodsh reviewed May 12, 2023
View reviewed changes
backend/dataall/modules/mlstudio/services/services.py Outdated
def list_sagemaker_studio_users(*, filter: dict) -> dict:
with _session() as session:
return SageMakerStudioRepository(session).paginated_sagemaker_studio_users(
username=context.username,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be context() :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am changing to use get_context directly to point out that it is a function

dlpzx added 19 commits May 25, 2023 18:11
@dlpzx
Fixing migration scripts
5ab0bcf
@dlpzx
Added mlstudio module skeleton - api package except resolvers, skelet…
2e60acf 
…on for db and services
@dlpzx
mlstudio - moved permissions
ba8b023
@dlpzx
ml studio - started to create db models, repositories, services
df72352
@dlpzx
Added create sagemaker studio service, added sm_studio_client
78deecb
@dlpzx
Renaming and advancing with db.repository, services and resolvers (list)
f39da0a
@dlpzx
Rename permissions to SGM_USER instead of SGM_NOTEBOOK
8b39a99
@dlpzx
More services and methods - presigned url...
5d817cb
@dlpzx
Completed services-db-resolvers. Renamed to naming convention sagemak…
a6627f2 
…er_studio_user --> will need migration scripts
@dlpzx
Linting
cf1c6c5
@dlpzx
Cleanup, extract non class method in sagemaker client
48146b5
@dlpzx
Review comments
c8dc09f
@dlpzx
Linting and minor errors
b098e9f
@dlpzx
Fixing migration scripts
4176228
@dlpzx
Finish SGM_STUDIO migration scripts, split migration scripts
345baae
@dlpzx
Typo in Makefile
3c6cf8b
@dlpzx
Naming conventions
d96a3b3
@dlpzx
Renaming, fixed migration scripts and remove sagemaker_base
4a163da
@dlpzx
mlstudio review
4ce293a
@dlpzx dlpzx force-pushed the mlstudio-mod-part1 branch from c402ee8 to 4ce293a Compare May 25, 2023 16:14
@dlpzx dlpzx marked this pull request as draft May 30, 2023 10:22
@dlpzx dlpzx closed this May 30, 2023
@dlpzx
Copy link
Contributor Author

dlpzx commented May 30, 2023

Closed to avoid merging conflicts from modularization-main. A new PR will be opened

@dlpzx dlpzx mentioned this pull request May 30, 2023
Merged
dlpzx added a commit that referenced this pull request Jun 14, 2023
@dlpzx @dependabot
@wolanlu @AmrSaber @noah-paige @kukushking @dosiennik @degoldner @nikpodsh @akaitoua @akaitoua-sa @gmuslia @rbernotas @kimengu-david @dbalintx
modularization: MLstudio modularization (#486)
f2505b5 
### Feature or Bugfix
- Refactoring

### Detail
First part of the modularization of ML Studio 

- Created `SageMakerStudioService` and move `resolvers` business logic
into the service
- Redefined `resolvers` and add request validators
- Redefined `db` operations and added them into `SageMakerRepository`
- Created `sagemaker_studio_client` with Sagemaker Studio SDK calls
- Created `ec2_client` with EC2 SDK calls
- Added CloudFormation stack in the `cdk` package of the module
- Added CloudFormation stack as an environment extension in the `cdk`
package of the module
- Modified environment creation and edit api calls and views to read
`mlStudiosEnabled` from environment parameters table
- Fixed tests and added migration scripts

Additional:
- Standardized naming of functions and objects to
"sagemaker_studio_user" and avoid legacy "Notebook" references
- Removed unused api definition for `getSagemakerStudioUserApps` 
- Cleaned up unused frontend views and API methods
- Split migration scripts for environments, worksheets and mlstudio
- linting Worksheets module
- prettier frontend

### Relates
#448 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: wolanlu <101870655+wolanlu@users.noreply.github.com>
Co-authored-by: Amr Saber <amr.m.saber.mail@gmail.com>
Co-authored-by: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Co-authored-by: kukushking <kukushkin.anton@gmail.com>
Co-authored-by: Dariusz Osiennik <osiend@amazon.com>
Co-authored-by: Dennis Goldner <107395339+degoldner@users.noreply.github.com>
Co-authored-by: Nikita Podshivalov <nikpodsh@amazon.com>
Co-authored-by: Abdulrahman Kaitoua <abdulrahman.kaitoua@polimi.it>
Co-authored-by: akaitoua-sa <126820454+akaitoua-sa@users.noreply.github.com>
Co-authored-by: Gezim Musliaj <102723839+gmuslia@users.noreply.github.com>
Co-authored-by: Rick Bernotas <97474536+rbernotas@users.noreply.github.com>
Co-authored-by: David Mutune Kimengu <57294718+kimengu-david@users.noreply.github.com>
Co-authored-by: nikpodsh <124577300+nikpodsh@users.noreply.github.com>
Co-authored-by: dbalintx <132444646+dbalintx@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dbalintx dbalintx dbalintx left review comments

@nikpodsh nikpodsh nikpodsh left review comments

Assignees

@dlpzx dlpzx

Labels
type: modularization Code refactoring project
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.