fix: pin version of npm@9 for VPC facing architecture #724
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Build succeeds for me with the changes in my VPC facing deployment |
dbalintx
approved these changes
Sep 4, 2023
noah-paige
added a commit
that referenced
this pull request
Jun 25, 2024
commit 4cb33324
Author: Admin/noahpaig-Isengard <Admin/noahpaig-Isengard>
Date: Mon Sep 11 2023 12:12:11 GMT-0400 (Eastern Daylight Time)
Conflicts resolved in the console.
commit 9e9b0e9
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Mon Sep 11 2023 10:57:24 GMT-0400 (Eastern Daylight Time)
Merge branch 'main' into main-v2
commit 0eefd78
Author: dlpzx <dlpzx@amazon.com>
Date: Mon Sep 11 2023 10:55:04 GMT-0400 (Eastern Daylight Time)
Upgrade css-tools and revert format of yarn.lock
commit 88acc19
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Mon Sep 11 2023 08:56:30 GMT-0400 (Eastern Daylight Time)
Modularization/fix glossary associations (#742)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Unable to load Glossary Term Associations for `DataStorageLocation`
- Pass the correct target_type of `Folder` when setting Glossary Term
Links
- Unable to load Glossary Term Associations when either Dashboards or
Datasets Module Disabled
- We have inline fragments on the `getGlossary` graphQL API that will
throw an error if the ObjectType is not defined (i.e. when Dashboards or
Datasets are disabled)
- Since we only require a common field returned by any of the Term
Association Objects (we only need `label` field) we can remove the
inline fragments so there is no need to have object types defined when
loading the glossary term associations
- Also cleaned up some of the other API Get calls on glossary that
return additional information that is unused
**NOTE: This PR is branched off of #731 and should be merged ONLY AFTER
the prior is to `main-v2`**
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
NA
```
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: dlpzx <dlpzx@amazon.com>
Co-authored-by: dlpzx <71252798+dlpzx@users.noreply.github.com>
commit 9158691
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Mon Sep 11 2023 01:34:04 GMT-0400 (Eastern Daylight Time)
DA v2: fix glossaries permissions and refactor catalog module (#731)
### Feature or Bugfix
- Bugfix
- Refactoring
### Detail
- moved glossaries permissions from core to module.catalog
- refractored catalog module to follow services, resolvers, db layer
design
- fix list_term_associations
- remove Columns from glossaries registry --> they cannot be tagged
- clean up unused code in glossaries
### Relates
V2.0.0 release
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
`N/A`
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 75171c0
Author: nikpodsh <124577300+nikpodsh@users.noreply.github.com>
Date: Fri Sep 08 2023 09:44:08 GMT-0400 (Eastern Daylight Time)
Add dependency for Worksheets (#740)
### Feature or Bugfix
Added datasets as dependency for worksheets
### Security
N/A
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 8d95241
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri Sep 08 2023 05:17:08 GMT-0400 (Eastern Daylight Time)
DA v2: Modularized structure of feeds and votes (#735)
### Feature or Bugfix
- Refactoring
### Detail
- Adjust `feed` module to the new code layer design
- Adjust `vote` module to the new code layer design
- Clean up unused code
### Relates
- V2.0
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
`N/A`
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit a4b831e
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Thu Sep 07 2023 12:43:38 GMT-0400 (Eastern Daylight Time)
DA v2: Fix Handling of Delete Dataset Resources (#737)
### Feature or Bugfix
<!-- please choose -->
- Bugfix in V2.0 data.all code
### Detail
- When deleting a dataset - we were redefining the `uri` variable used
to track the datasetUri. This caused errors with:
- Folders to not be removed from opensearch catalog
- Folders to not be removed from opensearch catalog
- Deletion of outstanding shares with no shared items on the dataset (if
they exist)
- Deletion of Glossary Term Links on the Dataset or Dataset Tables (if
they exist)
- Deletion of Resource Policies on the Dataset
- Deletion of the Dataset CloudFormation Stack (if enabled)
- The above handling of deletion should be resolved as of this PR
### Relates
- #733
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
NA
```
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b43a247
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Thu Sep 07 2023 09:21:39 GMT-0400 (Eastern Daylight Time)
Fix/cdkpipeline local deploy (#730)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Fix bug with local deployments creation of CDK Pipelines. Error Caused
By:
- Not registering `ImportMode.CDK_CLI_EXTENSION` since `load_modules()`
for this import mode type was only being called for AWS Deployments
- Incorrect path specified for the location of the cdk/ddk app created
for CDK Pipelines in local deployments
-
### Relates
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
NA
```
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 51890cb
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed Sep 06 2023 13:08:11 GMT-0400 (Eastern Daylight Time)
DA v2: Fix Get Query on Chat/Feed Feature (#729)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- If 1 of the dataset, dashboard, or datapipeline modules was disabled,
the query to get Feed (i.e. the Chat Feature) would fail with an error
similar to error:
- `[GraphQL error]: Message: Unknown type 'DataPipeline'. Did you mean
'DatasetLink'?, Location: [object Object], Path: undefined`
- This is due to the fixed inline fragments on the `getFeed(...)` query
that do not exists such as `... on DatasetTable {...}`
- In this PR we remove the fragments as the `label` return value is not
used in the `FeedComments` View
### Relates
- [#723](#723)
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit fe3ed03
Author: Mo <136477937+itsmo-amzn@users.noreply.github.com>
Date: Wed Sep 06 2023 05:53:00 GMT-0400 (Eastern Daylight Time)
Fix frontend issues (#727)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- disabled `datasets` tab in `environment` view when `datasets` module
is disabled
- disabled module list item in environment features when module is
disabled
- fix issue with landing route when `catalog` module is disabled
- `worksheets` tab enablement rules
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 0c22580
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue Sep 05 2023 11:09:46 GMT-0400 (Eastern Daylight Time)
DA v2: fix default permissions and update migrations scripts (#728)
### Feature or Bugfix
- Bugfix
### Detail
There are some issues with the permissions that appear in the invitation
request
Original:
Fresh deployment (with mlStudio module disabled):
<img width="500" alt="image"
src="https://github.com/awslabs/aws-dataall/assets/71252798/996aae76-3069-4562-9baf-d7255d009650">
With a pre-existing deployment:
old deployment:
```
Invite other teams
Add consumption roles
create networks
create pipelines
create notebooks
Request datasets access
create datasets
create redshift ---> removed! already done
create ML Studio ---> renamed! already done
```
The following are new or wrong permissions in fresh deployments /
backwards
```
List datasets on this environment / LIST_ENVIRONMENT_DATASETS
Run athena queries / RUN_ATHENA_QUERY
List datasets shared with this environments (TYPO!) / LIST_ENVIRONMENT_SHARED_WITH_OBJECTS??
nothing (mlstudio disabled) / LIST_ENVIRONMENT_SGMSTUDIO_USERS
```
This PR includes 3 fixes:
1) `RUN_ATHENA_QUERY`: Good to add, but was not there before so we need
to update the description in the migration scrips
2) `LIST_ENVIRONMENT_DATASETS`, `LIST_ENVIRONMENT_SHARED_WITH_OBJECTS`:
they are needed by default, so we can add a new list
`ENVIRONMENT_INVITED_DEFAULT` and add them there instead of adding them
in the list that is used for the toogle menu.
3) `LIST_ENVIRONMENT_SGMSTUDIO_USERS`: this permission is not used, we
just need to remove it
In addition some permissions have been renamed. I used the
`migrations/versions/4a0618805341_rename_sgm_studio_permissions.py`
script as it already handles renames
Testing:
[X] - Local testing of renaming and descriptions
[X] - AWS testing of the permissions that appear on screen
[ ] - AWS testing with an invited group - check that they can list
datasets and shares in environment
This is the end result for a deployment with the dashboards modules
disabled:
### Relates
- V2
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
`N/A`
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 12db96a
Author: nikpodsh <124577300+nikpodsh@users.noreply.github.com>
Date: Tue Sep 05 2023 08:18:52 GMT-0400 (Eastern Daylight Time)
Add conditions on catalog indexer tasks (#725)
The catalog indexer fails if all modules are disabled. Here is a small
improvement to fix it
Checked against:
1) all disabled
2) all enabled
3) datasets enabled only
4) dashboards enabled only
### Security
`N/A` . it's a small improvement for disable/enabling tasks based on
modules. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 2295e51
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Mon Sep 04 2023 09:03:18 GMT-0400 (Eastern Daylight Time)
fix: pin version of npm@9 for VPC facing architecture (#724)
### Feature or Bugfix
- Bugfix
### Detail
In the [latest release of
npm](https://github.com/npm/cli/releases/tag/v10.0.0) 4 days ago, node
16 is no longer supported. Which leads to failure of the frontend image
building when using VPC-facing architecture. In this PR the version of
npm is fixed to the previous version. It solves the issue and it is a
quick fix, but we need to upgrade to node 18. There are GitHub issues (
#655 #38 ...) dedicated to the node version upgrade, which should be
done in a separate PR.
I tried upgrading the version of node directly in the docker image, but
it lead to deployment errors (different ones)
### Relates
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
N/A for all
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit fb7632d
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri Sep 01 2023 09:43:56 GMT-0400 (Eastern Daylight Time)
DA V2: Fix Environment Stack CDKProxy Register by default (#717)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- When all modules are disabled (or all modules that depend on
EnvironmentSetup for their CDK Module Interface) then the Environment
Stack is not imported properly or registered as a stack in the
StackManager
- This PR adds EnvironmentSetup to the `__init__.py` file of the
`base/cdkproxy/stacks/` so Environment stack will always be registered
by default
- It also separates `mlstudio_stack` and mlstudio_extension` to avoid
circular dependencies when importing `EnvironmentSetup` and `stack` in
the same file
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
NA
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit bf77f7c
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri Sep 01 2023 04:45:00 GMT-0400 (Eastern Daylight Time)
DA v2: fix issues with modularization pipelines (#705)
### Feature or Bugfix
- Refactoring
- datapipelines module is not following design guidelines. `api`
=validation of input, `services`=check permissions and business logic
- there is a lot of unused code
### Detail
- move business logic to `services`
- implement validation of parameters in `api`
- clean up unused code in `datapipelines`: cat, ls and the `aws` clients
### Relates
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
N/A
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Balint David <dbalintx@amazon.com>
commit af860f6
Author: dbalintx <132444646+dbalintx@users.noreply.github.com>
Date: Thu Aug 31 2023 12:16:17 GMT-0400 (Eastern Daylight Time)
Fix canary user pw creation (#718)
In rare cases the current way of generating a Canary user password in
Cognito can result in a string containing no numerical values, hence
following error is thrown during deployment ( requires
**enable_cw_canaries** config parameter set to True in cdk.json):
`botocore.errorfactory.InvalidPasswordException: An error occurred
(InvalidPasswordException) when calling the AdminCreateUser operation:
Password did not conform with password policy: Password must have
numeric characters`
Changing the password creation to contain at least 1 uppercase and 1
numerical character.
### Feature or Bugfix
- Bugfix
### Detail
- <feature1 or bug1>
- <feature2 or bug2>
### Relates
- <URL or Ticket>
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
n/a
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 43eb082
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed Aug 30 2023 09:07:15 GMT-0400 (Eastern Daylight Time)
Add env URI to role share query (#706)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- If I have Group_1 that is invited to Env1 and Env2 and has existing
shares for that group to Env1 - I still cannot remove the EnvGroup from
Env2 (where no shares exists)
- When we count resources to validate before removing group - for share
objects in the `count_principal_resources()` we only filter by
principalId and principalType but we also need to filter by
environmentUri
- This PR fixes query to add environmentURI filter when counting
envGroup resources before removing groups
### Relates
### Security
NA
```
Please answer the questions below briefly where applicable, or write `N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 4ced415
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Tue Aug 29 2023 14:49:18 GMT-0400 (Eastern Daylight Time)
DA v2: Fix Params for RAM Invitations Table Sharing (#713)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- LF Cross Account Table shares was originally unable to find the
correct RAM Share Invitiation because it was filtering on incorrect
`Sender` and `Receiver` Accounts
- Typically led to a `Insufficient Glue Permissions on GrantPermissions
Operation` when attempting to share a table in v2 data.all
### Relates
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
NA
```
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit ad53c33
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Mon Aug 28 2023 03:15:25 GMT-0400 (Eastern Daylight Time)
DA v2: Dataset IAM Role Default DB Glue Permissions (#701)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- In the case where the `default` Glue Database does not already exist
- Glue Script is failing with "DATASET IAM ROLE is not authorized to
perform: glue:CreateDatabase on resource XXX"
- Root Casue
- The Glue Job needs to verify that the default database exists and if
it does not it needs to create the default DB
- Adding the glue:CreateDatabase permissions on the catalog and only for
the default DB resolves the issue
- POTENTIAL ISSUE:
- In the Custom Resource of Dataset Stack we add permission for the
dataset IAM Role to the default DB if it exists
- In the scenario where default DB does not already exist and we create
Dataset1 and Dataset2
- Then Dataset1 Role creates default DB when running profiling job and
the Dataset2 IAM Role never gets proper LF Permissions
- On testing - I tried a Dataset Stack update but it determines "No
Change Set" on the Cloudformation Stack so the Custom Resource does not
re-run to add the required permissions
- A potential solution is to pass a randomly generated uuid as part of
the Custom Resource properties to trigger a run of the Lambda each time?
### Relates
- <URL or Ticket>
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: dlpzx <dlpzx@amazon.com>
commit e097d08
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Thu Aug 24 2023 15:41:40 GMT-0400 (Eastern Daylight Time)
DA v2: Worksheets View need default export (#703)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Worksheet view is not loading with error:
```
Element type is invalid. Received a promise that resolves to: [missing argument]. Lazy element type must resolve to a class or function.[missing argument]
```
- From React
[Docs](https://legacy.reactjs.org/docs/code-splitting.html#reactlazy):
```
React.lazy takes a function that must call a dynamic import(). This must return a Promise which resolves to a module with a default export containing a React component.
```
### Relates
### Security
NA
```
Please answer the questions below briefly where applicable, or write `N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 9156189
Author: nikpodsh <124577300+nikpodsh@users.noreply.github.com>
Date: Thu Aug 24 2023 04:45:43 GMT-0400 (Eastern Daylight Time)
Fix chats. (#698)
Fixed chats. Worksheets are no longer eligible as a chat entity.
Security `N/A`, just removing a few lines of frontend code. Based on
[OWASP 10](https://owasp.org/Top10/en/).
Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
Is the input sanitized?
What precautions are you taking before deserializing the data you
consume?
Is injection prevented by parametrizing queries?
Have you ensured no eval or similar functions are used?
Does this PR introduce any functionality or component that requires
authorization?
How have you ensured it respects the existing AuthN/AuthZ mechanisms?
Are you logging failed auth attempts?
Are you using or adding any cryptographic features?
Do you use a standard proven implementations?
Are the used keys controlled by the customer? Where are they stored?
Are you introducing any new policies/roles/users?
Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 9ab933a
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Thu Aug 24 2023 02:25:28 GMT-0400 (Eastern Daylight Time)
DA v2: List Datasets Pagination (#700)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- When owning 3 data.all dataset with shares created and viewing the
Datasets Tab in the data.all UI --> There were multiple pages showing a
different number of datasets on each page (expected behavior to only
have 1 page in the datasets tab showing all 3 datasets since the default
page size is set to 10)
- This was happening because when we call the `listDatasets` API we are
returning a query that joins datasets with existing shares to ensure
that we list all datasets that a user either owns, stewards, or is
shared to
- But we did not filter the query for distinct datasetUris --> Meaning
if I own 1 dataset with 3 existing shares each on 1 table in the dataset
and I the query to find all user datasets - it would return 4 records to
me instead of just 1 for the dataset I own
### Security
NA
```
Please answer the questions below briefly where applicable, or write `N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
```
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 12d0596
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Thu Aug 24 2023 02:24:07 GMT-0400 (Eastern Daylight Time)
DA v2: Fix Lakeformation Registered Location Check Dataset Stack (#699)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- In v2 code, when creating/updating a dataset stack it would go back
and forth creating and deleting the `Lakeformation::Resource` construct
in Cloudformation which was registering/un-registering the resource
- This led to issues querying the data in Athena with a `Permissions
denied on S3 Path` error
- The root cause was when performing
`check_existing_lf_registered_location()` in the dataset stack synth we
were passing the wrong variable to resolve the AWS Account Id for the
role arn returned from the API call `lakeformation.describe_resource()`
### Relates
- #672
- #671
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
NA
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 9a250e5
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed Aug 23 2023 11:58:42 GMT-0400 (Eastern Daylight Time)
Fix Delete Env Validation on Consumption Role Resource (#693)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Resolve incorrect parameters passed when counting consumption role
resources before handling deletes of environments
### Relates
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 37c8cb7
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed Aug 23 2023 11:58:25 GMT-0400 (Eastern Daylight Time)
re create pivot roles on upgrade to v2 (#696)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Change name of pivot role policies created via CDK to handle upgrades
from v1.6.2 to v2.X
- Originally throwing an error on updates to the environment stack
`res-pivotrole-cdk-policy-2 already exists in stack
`arn:aws:cloudformation:eu-west-1:xxxxxxxx:stack/PIVOTROLESTACKNAME`
- Add permissions required by Custom CDK Exec Role to create policy
versions for stack upgrades
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b9b64d1
Author: nikpodsh <124577300+nikpodsh@users.noreply.github.com>
Date: Wed Aug 23 2023 11:56:41 GMT-0400 (Eastern Daylight Time)
Fix auth issue happening on the frontend (#689)
Fix of auth issue in frontend
commit e0b287f
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed Aug 23 2023 09:29:46 GMT-0400 (Eastern Daylight Time)
Fix Env Parameters Migration Script (#692)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Resolve errors in migration script for environemnt parameters when
upgrading from data.all v1.6.2 with pre-existing environments to
beta-2.0
### Relates
- #691
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: dlpzx <dlpzx@amazon.com>
commit 2fb7d97
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue Aug 22 2023 05:09:41 GMT-0400 (Eastern Daylight Time)
Add missing frontend path in albfront stage (#686)
### Feature or Bugfix
- Bugfix
### Detail
- Add frontend to path to dockerfile in albfront stage
### Relates
- V2.0
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 568a8fe
Author: nikpodsh <124577300+nikpodsh@users.noreply.github.com>
Date: Tue Aug 22 2023 04:36:42 GMT-0400 (Eastern Daylight Time)
Main v2 (#683)
A new PR without conflicts
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Amr Saber <amr.m.saber.mail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dlpzx <71252798+dlpzx@users.noreply.github.com>
Co-authored-by: wolanlu <101870655+wolanlu@users.noreply.github.com>
Co-authored-by: dbalintx <132444646+dbalintx@users.noreply.github.com>
Co-authored-by: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Co-authored-by: kukushking <kukushkin.anton@gmail.com>
Co-authored-by: Dariusz Osiennik <osiend@amazon.com>
Co-authored-by: Dennis Goldner <107395339+degoldner@users.noreply.github.com>
Co-authored-by: Abdulrahman Kaitoua <abdulrahman.kaitoua@polimi.it>
Co-authored-by: akaitoua-sa <126820454+akaitoua-sa@users.noreply.github.com>
Co-authored-by: Gezim Musliaj <102723839+gmuslia@users.noreply.github.com>
Co-authored-by: Rick Bernotas <97474536+rbernotas@users.noreply.github.com>
Co-authored-by: David Mutune Kimengu <57294718+kimengu-david@users.noreply.github.com>
Co-authored-by: Mohit Arora <1666133+blitzmohit@users.noreply.github.com>
Co-authored-by: Balint David <dbalintx@amazon.com>
Co-authored-by: chamcca <40579012+chamcca@users.noreply.github.com>
Co-authored-by: Dhruba <117375130+marjet26@users.noreply.github.com>
Co-authored-by: Srinivas Reddy <srinivasreddych@outlook.com>
Co-authored-by: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Co-authored-by: dlpzx <dlpzx@amazon.com>
Co-authored-by: Noah Paige <noahpaig@amazon.com>
Co-authored-by: Mo <136477937+itsmo-amzn@users.noreply.github.com>
Co-authored-by: Raul Cajias <cajias@users.noreply.github.com>
Co-authored-by: Maryam Khidir <maryamolaide95@gmail.com>
Co-authored-by: Jorge Iglesias Garcia <44316552+jorgeig-space@users.noreply.github.com>
Co-authored-by: Jorge Iglesias Garcia <jorgeig@amazon.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Feature or Bugfix
Detail
In the latest release of npm 4 days ago, node 16 is no longer supported. Which leads to failure of the frontend image building when using VPC-facing architecture. In this PR the version of npm is fixed to the previous version. It solves the issue and it is a quick fix, but we need to upgrade to node 18. There are GitHub issues ( #655 #38 ...) dedicated to the node version upgrade, which should be done in a separate PR.
I tried upgrading the version of node directly in the docker image, but it lead to deployment errors (different ones)
Relates
Security
Please answer the questions below briefly where applicable, or write
N/A. Based onOWASP 10.
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
evalor similar functions are used?N/A for all
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.