Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jest, Playwright and pytest tests #207

Merged
merged 56 commits into from
Jun 4, 2022
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
8688979
Add jest and playwright testing
fcollonval Feb 12, 2022
e61eebc
Fix working directory
fcollonval Feb 12, 2022
7cf1ecc
Use Jest 26
fcollonval Feb 12, 2022
9796df4
Don't lint tests files
fcollonval Feb 12, 2022
f294cbd
Fix manifest
fcollonval Feb 12, 2022
135f598
Add update snapshots action
fcollonval Feb 12, 2022
71dd282
Add documentation
fcollonval Feb 12, 2022
2e7a683
Add pytest
fcollonval Feb 12, 2022
e0a9907
Fix cookiecutter
fcollonval Feb 12, 2022
7d87d2f
Print log and stop jupyter lab
fcollonval Feb 12, 2022
86871cd
First install jupyter lab
fcollonval Feb 12, 2022
b2c2221
partial revert to test stuck CI task
fcollonval Feb 12, 2022
eee23be
Fix pytest
fcollonval Feb 12, 2022
fe03c32
Ensure server log redirection
fcollonval Feb 12, 2022
df4af43
Move ui tests config in subfolder
fcollonval Feb 14, 2022
4514ab3
Add documentation for pytest
fcollonval Feb 14, 2022
2bb2088
Run integration tests in post job using distribution artifact
fcollonval Feb 14, 2022
0012738
Apply review suggestions
fcollonval Feb 17, 2022
38fbe46
Update check test
fcollonval Feb 18, 2022
145662c
[skip ci] Update documentation
fcollonval Feb 18, 2022
f9a1fe8
Remove galata and playwright from main package.json
fcollonval Feb 18, 2022
94000be
Merge branch '3.0' into fix/Add-testing
fcollonval Feb 22, 2022
c1593ee
Update galata
fcollonval May 27, 2022
70adf00
Add jest and playwright testing
fcollonval Feb 12, 2022
26c7284
Fix working directory
fcollonval Feb 12, 2022
976d6b4
Use Jest 26
fcollonval Feb 12, 2022
8b0cd6e
Don't lint tests files
fcollonval Feb 12, 2022
ad4838e
Fix manifest
fcollonval Feb 12, 2022
f36e3e2
Add update snapshots action
fcollonval Feb 12, 2022
b504266
Add documentation
fcollonval Feb 12, 2022
3b6c6bb
Add pytest
fcollonval Feb 12, 2022
79e527e
Fix cookiecutter
fcollonval Feb 12, 2022
37fd2c2
Print log and stop jupyter lab
fcollonval Feb 12, 2022
d4b941b
First install jupyter lab
fcollonval Feb 12, 2022
6a01635
partial revert to test stuck CI task
fcollonval Feb 12, 2022
5aa176b
Fix pytest
fcollonval Feb 12, 2022
0a8f3ab
Ensure server log redirection
fcollonval Feb 12, 2022
74c1d9c
Move ui tests config in subfolder
fcollonval Feb 14, 2022
b29009f
Add documentation for pytest
fcollonval Feb 14, 2022
9ff452d
Run integration tests in post job using distribution artifact
fcollonval Feb 14, 2022
07c0902
Apply review suggestions
fcollonval Feb 17, 2022
5190921
Update check test
fcollonval Feb 18, 2022
672041f
[skip ci] Update documentation
fcollonval Feb 18, 2022
c2bf000
Remove galata and playwright from main package.json
fcollonval Feb 18, 2022
9e1ee79
Update galata
fcollonval May 27, 2022
bc386fb
Fixed rebase
3coins May 28, 2022
b869015
Merge branch 'fix/Add-testing' into 3coins-add-testing
fcollonval May 30, 2022
8d1fb7f
Update add testing
fcollonval May 30, 2022
061054a
Fix prettier
fcollonval May 30, 2022
51320d2
Fix mac job
fcollonval May 30, 2022
27805cf
Apply suggestions
fcollonval Jun 2, 2022
bac5026
Add configurable for setting up the tests
fcollonval Jun 2, 2022
6cdfdcf
Fix syntax
fcollonval Jun 2, 2022
d71c65b
Fix for new cookiecutter
fcollonval Jun 2, 2022
4b8717e
Minor fixes
fcollonval Jun 2, 2022
40df2a2
Conditioned type jest
fcollonval Jun 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/workflows/check-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Tests validation

on:
push:
branches: ["*.0"]
pull_request:
branches: ["*"]
schedule:
- cron: "0 0 * * *"

jobs:
tests:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Base Setup
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Install dependencies
run: python -m pip install -U cookiecutter check-manifest jupyterlab~=3.1

- name: Create the extension
run: |
set -eux
# Trick to use custom parameters
python -m cookiecutter --no-input .

- name: Test the extension
working-directory: myextension
run: |
set -eux
jlpm
jlpm test

- name: Install the extension
working-directory: myextension
run: |
set -eux
python -m pip install .

- name: Install dependencies
working-directory: myextension/ui-tests
env:
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
run: jlpm install

- name: Set up browser cache
uses: actions/cache@v2
with:
path: |
${{ github.workspace }}/pw-browsers
key: ${{ runner.os }}-${{ hashFiles('myextension/ui-tests/yarn.lock') }}

- name: Install browser
run: jlpm playwright install chromium
working-directory: myextension/ui-tests

- name: Execute integration tests
working-directory: myextension/ui-tests
run: |
jlpm playwright test

- name: Upload Playwright Test report
if: always()
uses: actions/upload-artifact@v2
with:
name: myextension-playwright-tests
path: |
myextension/ui-tests/test-results
myextension/ui-tests/playwright-report
31 changes: 17 additions & 14 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,18 @@ jobs:
# Trick to use custom parameters
python -c "from cookiecutter.main import cookiecutter; import json, os; f=open('cookiecutter.json'); d=json.load(f); f.close(); d['labextension_name']=os.getenv('NAME'); cookiecutter('.', extra_context=d, no_input=True)"
pushd ${PYNAME}
pip install jupyterlab
python -m pip install jupyterlab
jlpm
jlpm stylelint-config-prettier-check
jlpm lint:check
pip install -e .
python -m pip install -e .
jupyter labextension develop . --overwrite
jupyter labextension list
jupyter labextension list 2>&1 | grep -ie "${NAME}.*OK"
python -m jupyterlab.browser_check

jupyter labextension uninstall ${NAME}
pip uninstall -y ${NAME} jupyterlab
python -m pip uninstall -y ${NAME} jupyterlab

git init && git add .
check-manifest -v
Expand Down Expand Up @@ -154,11 +154,14 @@ jobs:
# Trick to use custom parameters
python -c "from cookiecutter.main import cookiecutter; import json; f=open('cookiecutter.json'); d=json.load(f); f.close(); d['kind']='server'; cookiecutter('.', extra_context=d, no_input=True)"
cd myextension
pip install -e .
pip install jupyterlab
python -m pip install -e .[test]
python -m pip install jupyterlab
jupyter labextension develop . --overwrite
jupyter server extension enable myextension

# Check unit tests are passing
python -m pytest -vv -r ap --cov myextension

- name: Check pip develop method
run: |
set -eux
Expand All @@ -177,7 +180,7 @@ jobs:
jupyter labextension build ./myextension

jupyter labextension uninstall myextension
pip uninstall -y myextension jupyterlab
python -m pip uninstall -y myextension jupyterlab

python -c "import shutil; shutil.rmtree('myextension')"

Expand All @@ -186,11 +189,11 @@ jobs:
# Trick to use custom parameters
python -c "from cookiecutter.main import cookiecutter; import json; f=open('cookiecutter.json'); d=json.load(f); f.close(); d['kind']='server'; cookiecutter('.', extra_context=d, no_input=True)"
cd myextension
pip install jupyterlab
python -m pip install jupyterlab
jupyter lab clean --all
python -m build --sdist
cd dist
pip install myextension-0.1.0.tar.gz
python -m pip install myextension-0.1.0.tar.gz

- name: Check install tarball method
run: |
Expand All @@ -205,7 +208,7 @@ jobs:
python -m jupyterlab.browser_check

cp myextension/dist/*.tar.gz myextension.tar.gz
pip uninstall -y myextension jupyterlab
python -m pip uninstall -y myextension jupyterlab
rm -rf myextension
shell: bash

Expand Down Expand Up @@ -248,8 +251,8 @@ jobs:
sudo rm -rf $(which node)
sudo rm -rf $(which node)

pip install myextension.tar.gz
pip install jupyterlab
python -m pip install myextension.tar.gz
python -m pip install jupyterlab
jupyter labextension list 2>&1 | grep -ie "myextension.*OK"
jupyter server extension list
jupyter server extension list 2>&1 | grep -ie "myextension.*OK"
Expand Down Expand Up @@ -279,17 +282,17 @@ jobs:
# Trick to use custom parameters
python -c "from cookiecutter.main import cookiecutter; import json; f=open('cookiecutter.json'); d=json.load(f); f.close(); d['kind']='theme'; cookiecutter('.', extra_context=d, no_input=True)"
pushd mytheme
pip install jupyterlab
python -m pip install jupyterlab
jlpm
jlpm lint:check
pip install -e .
python -m pip install -e .
jupyter labextension develop . --overwrite
jupyter labextension list
jupyter labextension list 2>&1 | grep -ie "mytheme.*OK"
python -m jupyterlab.browser_check

jupyter labextension uninstall mytheme
pip uninstall -y mytheme jupyterlab
python -m pip uninstall -y mytheme jupyterlab

git init && git add .
check-manifest -v
Expand Down
4 changes: 3 additions & 1 deletion hooks/post_gen_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@ def remove_path(path: str) -> None:
for f in (
"{{ cookiecutter.python_name }}/handlers.py",
"src/handler.ts",
"jupyter-config"
"jupyter-config",
"conftest.py",
"{{ cookiecutter.python_name }}/tests"
):
remove_path(PROJECT_DIRECTORY / f)

Expand Down
3 changes: 3 additions & 0 deletions {{cookiecutter.python_name}}/.eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,6 @@ dist
coverage
**/*.d.ts
tests

**/__tests__
ui-tests
104 changes: 91 additions & 13 deletions {{cookiecutter.python_name}}/.github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ on:
jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2
Expand All @@ -19,30 +20,45 @@ jobs:
- name: Install dependencies
run: python -m pip install -U jupyterlab~=3.1 check-manifest

- name: Build the extension
- name: Lint the extension
run: |
set -eux
jlpm
jlpm lint:check
python -m pip install .
jlpm run lint:check

- name: Test the extension
run: |
set -eux
jlpm run test

- name: Build the extension
run: |
set -eux
python -m pip install .[test]
{% if cookiecutter.kind.lower() == 'server' %}
pytest -vv -r ap --cov {{ cookiecutter.python_name }}
jupyter server extension list
jupyter server extension list 2>&1 | grep -ie "{{ cookiecutter.python_name }}.*OK"
{% endif %}
jupyter labextension list
jupyter labextension list 2>&1 | grep -ie "{{ cookiecutter.labextension_name }}.*OK"
python -m jupyterlab.browser_check

- name: Package the extension
run: |
set -eux
check-manifest -v

pip install build
python -m build --sdist
cp dist/*.tar.gz myextension.tar.gz
python -m build
pip uninstall -y "{{ cookiecutter.python_name }}" jupyterlab
rm -rf myextension

- uses: actions/upload-artifact@v2
- name: Upload extension packages
uses: actions/upload-artifact@v2
with:
name: myextension-sdist
path: myextension.tar.gz
name: extension-artifacts
path: dist/{{ cookiecutter.python_name }}*
if-no-files-found: error

test_isolated:
needs: build
Expand All @@ -54,18 +70,80 @@ jobs:
- name: Install Python
uses: actions/setup-python@v2
with:
python-version: '3.8'
python-version: '3.9'
architecture: 'x64'
- uses: actions/download-artifact@v2
with:
name: myextension-sdist
name: extension-artifacts
- name: Install and Test
run: |
set -eux
# Remove NodeJS, twice to take care of system and locally installed node versions.
sudo rm -rf $(which node)
sudo rm -rf $(which node)
pip install myextension.tar.gz
pip install jupyterlab

pip install "jupyterlab~=3.1" {{ cookiecutter.python_name }}*.whl

{% if cookiecutter.kind.lower() == 'server' %}
jupyter server extension list
jupyter server extension list 2>&1 | grep -ie "{{ cookiecutter.python_name }}.*OK"
{% endif %}
jupyter labextension list
jupyter labextension list 2>&1 | grep -ie "{{ cookiecutter.labextension_name }}.*OK"
python -m jupyterlab.browser_check --no-chrome-test

integration-tests:
name: Integration tests
needs: build
runs-on: ubuntu-latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while ubuntu is a nice, predictable, well-behaved platform, windows is frankly where, from an overall integration testing perspective, one is most likely to get real data and catch evil things.

other fun things to find extremely predictable bugs:

  • installing/running/serving from paths with non-ascii characters and spaces
  • non-c:-drive installs
  • running with a non-/ base url


env:
PLAYWRIGHT_BROWSERS_PATH: ${{ "{{ github.workspace }}" }}/pw-browsers

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Base Setup
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Download extension package
uses: actions/download-artifact@v2
with:
name: extension-artifacts

- name: Install the extension
run: |
set -eux
python -m pip install "jupyterlab~=3.1" {{ cookiecutter.python_name }}*.whl

- name: Install dependencies
working-directory: ui-tests
env:
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
run: jlpm install
{% raw %}
- name: Set up browser cache
uses: actions/cache@v2
with:
path: |
${{ github.workspace }}/pw-browsers
key: ${{ runner.os }}-${{ hashFiles('ui-tests/yarn.lock') }}
{% endraw %}
- name: Install browser
run: jlpm playwright install chromium
working-directory: ui-tests

- name: Execute integration tests
working-directory: ui-tests
run: |
jlpm playwright test

- name: Upload Playwright Test report
if: always()
uses: actions/upload-artifact@v2
with:
name: {{ cookiecutter.python_name }}-playwright-tests
path: |
ui-tests/test-results
ui-tests/playwright-report
fcollonval marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Update Playwright Snapshots

on:
issue_comment:
types: [created, edited]

permissions:
contents: write
pull-requests: write

jobs:
{# Escape double curly brace #}
{% raw %}
update-snapshots:
if: ${{ github.event.issue.pull_request && contains(github.event.comment.body, 'please update playwright snapshots') }}
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2
with:
token: ${{ secrets.GITHUB_TOKEN }}

- name: Configure git to use https
run: git config --global hub.protocol https

- name: Checkout the branch from the PR that triggered the job
run: hub pr checkout ${{ github.event.issue.number }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Install dependencies
run: |
set -eux
jlpm
python -m pip install .

- uses: jupyterlab/maintainer-tools/.github/actions/update-snapshots@v1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
# Playwright knows how to start JupyterLab server
start_server_script: 'null'
test_folder: ui-tests
{% endraw %}
4 changes: 4 additions & 0 deletions {{cookiecutter.python_name}}/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ node_modules/
*.tsbuildinfo
{{cookiecutter.python_name}}/labextension

# Integration tests
ui-tests/test-results/
ui-tests/playwright-report/

# Created by https://www.gitignore.io/api/python
# Edit at https://www.gitignore.io/?templates=python

Expand Down
Loading