Skip to content

test: GHA to Execute Test without Docker Running #5290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sriram-mv merged 15 commits into from
Jun 8, 2023
Merged

test: GHA to Execute Test without Docker Running #5290

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
1e9931a
test: Test without Docker running
mildaniel Jun 7, 2023
5cb6490
Add build test
mildaniel Jun 7, 2023
cb39abd
Run install
mildaniel Jun 7, 2023
6fec858
Remove success condition
mildaniel Jun 7, 2023
41a8abe
Add continue on error
mildaniel Jun 7, 2023
1325067
Add continue on error
mildaniel Jun 7, 2023
79cc8fb
Separate tests
mildaniel Jun 7, 2023
2a298a3
Fix test name
mildaniel Jun 7, 2023
d880ae0
Require new test
mildaniel Jun 7, 2023
ca6d73d
Address comments
mildaniel Jun 7, 2023
dcfddb1
Attempt to parameterize for windows
mildaniel Jun 7, 2023
92d167d
Attempt to parameterize for windows
mildaniel Jun 7, 2023
42c0746
Attempt to parameterize for windows
mildaniel Jun 7, 2023
055427a
Set samdev in environment
mildaniel Jun 7, 2023
3f657ca
Move skip to top of test class
mildaniel Jun 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,14 @@ jobs:
- make-pr
- integration-tests
- smoke-and-functional-tests
- docker-disabled
steps:
- name: report-failure
if : |
needs.make-pr.result != 'success' ||
needs.integration-tests.result != 'success' ||
needs.smoke-and-functional-tests.result != 'success'
needs.smoke-and-functional-tests.result != 'success' ||
needs.docker-disabled.result != 'success'
run: exit 1
- name: report-success
run: exit 0
Expand Down Expand Up @@ -187,3 +189,43 @@ jobs:
run: make init
- name: Run functional & smoke tests
run: pytest -vv -n 4 tests/functional tests/smoke

docker-disabled:
name: Docker-disabled Tests / ${{ matrix.os }}
if: github.repository_owner == 'aws'
runs-on: ${{ matrix.os }}
env:
SAM_CLI_DEV: "1"
strategy:
matrix:
os:
- ubuntu-latest
- windows-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
# These are the versions of Python that correspond to the supported Lambda runtimes
python-version: |
3.10
3.9
3.8
3.7
- name: Init samdev
run: make init
- name: Stop Docker Linux
if: ${{ matrix.os == 'ubuntu-latest' }}
run: sudo systemctl stop docker
- name: Stop Docker Windows
if: ${{ matrix.os == 'windows-latest' }}
shell: pwsh
run: stop-service docker
- name: Check Docker not Running
run: docker info
id: run-docker-info
continue-on-error: true
- name: Report failure
if: steps.run-docker-info.outcome == 'success'
run: exit 1
- name: Run tests without Docker
run: pytest -vv tests/integration/buildcmd/test_build_cmd.py -k TestBuildCommand_PythonFunctions_WithoutDocker
Copy link
Contributor

@jfuss jfuss Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Should this be a make command at all?
  2. Should we be running this with SAM_CLI_DEV=1 like https://github.com/aws/aws-sam-cli/blob/develop/Makefile#L20?

Copy link
Contributor Author

@mildaniel mildaniel Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Not too sure what you mean, it's not currently a make command.
  2. 👍

Copy link
Contributor

@jfuss jfuss Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On one: I was asking if this should be a make command. Not a big deal.

Copy link
Contributor Author

@mildaniel mildaniel Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha. Don't think it needs to be. We have other tests in our actions we run directly through pytest. Don't think we need to abstract them all with make.

62 changes: 47 additions & 15 deletions tests/integration/buildcmd/test_build_cmd.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -398,34 +398,64 @@ def _validate_skipped_built_function(
"overrides",
"runtime",
"codeuri",
"use_container",
"check_function_only",
"prop",
),
[
("template.yaml", "Function", True, "python3.7", "Python", False, False, "CodeUri"),
("template.yaml", "Function", True, "python3.8", "Python", False, False, "CodeUri"),
("template.yaml", "Function", True, "python3.9", "Python", False, False, "CodeUri"),
("template.yaml", "Function", True, "python3.10", "Python", False, False, "CodeUri"),
("template.yaml", "Function", True, "python3.7", "PythonPEP600", False, False, "CodeUri"),
("template.yaml", "Function", True, "python3.8", "PythonPEP600", False, False, "CodeUri"),
("template.yaml", "Function", True, "python3.7", "Python", "use_container", False, "CodeUri"),
("template.yaml", "Function", True, "python3.8", "Python", "use_container", False, "CodeUri"),
("template.yaml", "Function", True, "python3.9", "Python", "use_container", False, "CodeUri"),
("template.yaml", "Function", True, "python3.10", "Python", "use_container", False, "CodeUri"),
("template.yaml", "Function", True, "python3.7", "Python", False, "CodeUri"),
("template.yaml", "Function", True, "python3.8", "Python", False, "CodeUri"),
("template.yaml", "Function", True, "python3.9", "Python", False, "CodeUri"),
("template.yaml", "Function", True, "python3.10", "Python", False, "CodeUri"),
("template.yaml", "Function", True, "python3.7", "PythonPEP600", False, "CodeUri"),
("template.yaml", "Function", True, "python3.8", "PythonPEP600", False, "CodeUri"),
],
)
class TestBuildCommand_PythonFunctions(BuildIntegPythonBase):
class TestBuildCommand_PythonFunctions_WithoutDocker(BuildIntegPythonBase):
Copy link
Contributor

@super132 super132 Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we skip all the builds tests without docker?

Copy link
Contributor Author

@mildaniel mildaniel Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are tests that don't use Docker, so we don't need to skip them.

overrides = True
runtime = "python3.9"
codeuri = "Python"
check_function_only = False
use_container = False

@pytest.mark.flaky(reruns=3)
def test_with_default_requirements(self):
self._test_with_default_requirements(
self.runtime,
self.codeuri,
self.use_container,
self.test_data_path,
do_override=self.overrides,
check_function_only=self.check_function_only,
)


@skipIf(SKIP_DOCKER_TESTS or SKIP_DOCKER_BUILD, SKIP_DOCKER_MESSAGE)
@parameterized_class(
(
"template",
"FUNCTION_LOGICAL_ID",
"overrides",
"runtime",
"codeuri",
"check_function_only",
"prop",
),
[
("template.yaml", "Function", True, "python3.7", "Python", False, "CodeUri"),
("template.yaml", "Function", True, "python3.8", "Python", False, "CodeUri"),
("template.yaml", "Function", True, "python3.9", "Python", False, "CodeUri"),
("template.yaml", "Function", True, "python3.10", "Python", False, "CodeUri"),
],
)
class TestBuildCommand_PythonFunctions_WithDocker(BuildIntegPythonBase):
overrides = True
runtime = "python3.9"
codeuri = "Python"
use_container = "use_container"
check_function_only = False

@pytest.mark.flaky(reruns=3)
def test_with_default_requirements(self):
if self.use_container and (SKIP_DOCKER_TESTS or SKIP_DOCKER_BUILD):
self.skipTest(SKIP_DOCKER_MESSAGE)
self._test_with_default_requirements(
self.runtime,
self.codeuri,
Expand Down Expand Up @@ -465,7 +495,9 @@ def test_with_default_requirements(self):
),
],
)
class TestBuildCommand_PythonFunctions_CDK(TestBuildCommand_PythonFunctions):
class TestBuildCommand_PythonFunctions_CDK(TestBuildCommand_PythonFunctions_WithoutDocker):
use_container = False

@pytest.mark.flaky(reruns=3)
def test_cdk_app_with_default_requirements(self):
self._test_with_default_requirements(
Expand Down