Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get Latest in main to v2m1m0 #771

Merged
merged 10 commits into from
Sep 19, 2023
Merged

Get Latest in main to v2m1m0 #771

merged 10 commits into from
Sep 19, 2023

Conversation

noah-paige
Copy link
Contributor

Feature or Bugfix

  • NA

Detail

  • Get latest code in main to v2m1m0 branch to keep in sync

Relates

  • NA

Security

Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10.

NA

- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
  - Is the input sanitized?
  - What precautions are you taking before deserializing the data you consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
  - How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

dlpzx and others added 10 commits September 13, 2023 12:00
… merge package (#751)

### Feature or Bugfix
- Feature
- Bugfix

### Detail
- add npm-audit github workflow on PR
- add semgrep worflow on PR and on schedule
- ignore semgrep issues with explanation. Those to be fixed will be
fixed in #739 and
#738
- remove make security checks that uses safety library and rename it as
linting
- upgrade all pacakges, add package-lock and pin merge to version 2.1.1

### Relates

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3
to 1.2.5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/jonschlinkert/word-wrap/releases">word-wrap's
releases</a>.</em></p>
<blockquote>
<h2>1.2.5</h2>
<p><strong>Changes</strong>:</p>
<p>Reverts default value for <code>options.indent</code> to two spaces
<code>' '</code>.</p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/jonschlinkert/word-wrap/compare/1.2.4...1.2.5">https://github.com/jonschlinkert/word-wrap/compare/1.2.4...1.2.5</a></p>
<h2>1.2.4</h2>
<h2>What's Changed</h2>
<ul>
<li>Remove default indent by <a
href="https://github.com/mohd-akram"><code>@​mohd-akram</code></a> in <a
href="https://redirect.github.com/jonschlinkert/word-wrap/pull/24">jonschlinkert/word-wrap#24</a></li>
<li>🔒fix: CVE 2023 26115 (2) by <a
href="https://github.com/OlafConijn"><code>@​OlafConijn</code></a> in <a
href="https://redirect.github.com/jonschlinkert/word-wrap/pull/41">jonschlinkert/word-wrap#41</a></li>
<li>:lock: fix: CVE-2023-26115 by <a
href="https://github.com/aashutoshrathi"><code>@​aashutoshrathi</code></a>
in <a
href="https://redirect.github.com/jonschlinkert/word-wrap/pull/33">jonschlinkert/word-wrap#33</a></li>
<li>chore: publish workflow by <a
href="https://github.com/OlafConijn"><code>@​OlafConijn</code></a> in <a
href="https://redirect.github.com/jonschlinkert/word-wrap/pull/42">jonschlinkert/word-wrap#42</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/mohd-akram"><code>@​mohd-akram</code></a> made
their first contribution in <a
href="https://redirect.github.com/jonschlinkert/word-wrap/pull/24">jonschlinkert/word-wrap#24</a></li>
<li><a
href="https://github.com/OlafConijn"><code>@​OlafConijn</code></a> made
their first contribution in <a
href="https://redirect.github.com/jonschlinkert/word-wrap/pull/41">jonschlinkert/word-wrap#41</a></li>
<li><a
href="https://github.com/aashutoshrathi"><code>@​aashutoshrathi</code></a>
made their first contribution in <a
href="https://redirect.github.com/jonschlinkert/word-wrap/pull/33">jonschlinkert/word-wrap#33</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4">https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/207044ebda1dd3809d15b6000a48409266536771"><code>207044e</code></a>
1.2.5</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/98943154855b0dd79b707462b9202614990c7f61"><code>9894315</code></a>
revert default indent</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/f64b188c7261d26b99e1e2075d6b12f21798e83a"><code>f64b188</code></a>
run verb to generate README</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/03ea08256ba0c8e8b02b1b304f0f5bd2b1863207"><code>03ea082</code></a>
Merge pull request <a
href="https://redirect.github.com/jonschlinkert/word-wrap/issues/42">#42</a>
from jonschlinkert/chore/publish-workflow</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/420dce9a2412b21881202b73a3c34f0edc53cb2e"><code>420dce9</code></a>
Merge pull request <a
href="https://redirect.github.com/jonschlinkert/word-wrap/issues/41">#41</a>
from jonschlinkert/fix/CVE-2023-26115-2</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/bfa694edf55bb84ff84512f13da6d68bf7593f06"><code>bfa694e</code></a>
Update .github/workflows/publish.yml</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/ace0b3c78f81aaf43040bab3bc91d3c5546d3fd2"><code>ace0b3c</code></a>
chore: bump version to 1.2.4</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/6fd727594676f3e1b196b08a320908bec2f4ca02"><code>6fd7275</code></a>
chore: add publish workflow</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/30d6daf60fce429f5f559252fa86ee78200652c4"><code>30d6daf</code></a>
chore: fix test</li>
<li><a
href="https://github.com/jonschlinkert/word-wrap/commit/655929cabea6299dddf3b4a21fc3713fca701b48"><code>655929c</code></a>
chore: remove package-lock</li>
<li>Additional commits viewable in <a
href="https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.5">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=word-wrap&package-manager=npm_and_yarn&previous-version=1.2.3&new-version=1.2.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/awslabs/aws-dataall/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Feature or Bugfix
- Feature

### Detail
Add ASH, Bandit and CDK Nag security checks.

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

---------

Co-authored-by: dlpzx <dlpzx@amazon.com>
### Feature or Bugfix
<!-- please choose -->
- Bugfix


### Detail
- Fix `ModuleName` for `MLSTUDIO` 
- MLStudio in the Environment Overview `Features` Card will not load
since the current `ModuleName.ML_STUDIO` property does not exist

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
NA
```
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
  - Is the input sanitized?
  - What precautions are you taking before deserializing the data you consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
  - How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?
```

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
### Feature or Bugfix
- Bugfix

### Detail
- add OIDC role for CDK-nag actions
- upgrade cdk-nag actions to latest version (V2 deprecated soon)

### Relates

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
### Feature or Bugfix
- Bugfix

### Detail
yarn.lock and package-lock.json react differently when defining specific
versions for a package in package.json
- yarn.lock works when the pinned packages are added to the resolutions
section
- package-lock.json when they are in the overrides section

When generating each of these accommodate the package.json accordingly.

### Relates
Dependabot and npm audit

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
### Feature or Bugfix
<!-- please choose -->
- Bugfix

### Detail
- Move `tests/modules/test_loader.py` to `tests/base/test_loader.py` as
originally the test_loader tests were being excluded because of the
`ignore_module_tests_if_not_active()` function in `tests/conftests.py`

- Added `@pytest.mark.skipif(...)` decorators to skip tests if the
module features are not enabled in `config.json`
- Originally this was failing on some of the `test_dataset_location.py`
tests when `modules.datasets.features.file_actions` was set to `False`
in `config.json`

### Relates

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

N/A
```
- Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
  - Is the input sanitized?
  - What precautions are you taking before deserializing the data you consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires authorization?
  - How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?
```

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
### Feature or Bugfix
- Enhancement

### Detail
Remove overly permissive permissions in the optional cdk execution
bootstrapping policy

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?

NA

- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?

NA

- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?

NA

- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?

Adding additional restrictions to permission set for cdk execution
bootstrapping policy when bootstrapping / linking environments


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

---------

Co-authored-by: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Co-authored-by: dlpzx <dlpzx@amazon.com>
### Feature or Bugfix
- Bugfix

### Detail
See issue #766 
Alternatives considered:
1. use [ok-to-test](https://github.com/imjohnbo/ok-to-test) --> requires
authentication, preferred is GitHub App (not possible for data.all),
other options: personal access token or OAuth app token
2. run cdk-nag action only on minor release branches (v2m2m0 to main)
branch
3. run cdk-nag on `pull_request_target` after the PR is merged
4. run cdk-nag on schedule
5. Use other than OIDC but I think the issue could still be there as it
has to due with permissions on the repo
6. Avoid the need for credentials

This last one is the cleanest and safest. We need to mock the context of
the cdk app either:
- passing context as part of the CLI command `cdk synth --context
key=value` --> not possible as we need to pass more complex params
- creating a json object in the CLI --> cumbersome
- CHOSEN: pass the context directly in the declaration of the `App`

In addition other changes had to be made:
- remove need for SSM calls in app.py if GithubActions are running
- try/except in getting the S3 prefixes

⚠️ STILL NEEDS TESTING

### Relates
#766 

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
…te-sanitization (#762)

### Feature or Bugfix
- Refactoring

### Detail
Use `replaceAll` instead of `replace` command in parsing the custom SAML
groups. This way all appearances of `[` and `]` are replaced.
I did not want to modify the command more as it is touching the
integration with other IdPs.

### Relates
- #739 

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized? ---> ⭐ exactly this is what this PR is trying
to improve.
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@noah-paige noah-paige merged commit dbbef3c into v2m1m0 Sep 19, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants