Skip to content

refactor: Python Inspector Overhaul Part 1 - Buildsystem migration #210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 12 additions & 20 deletions .github/workflows/docs-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,32 +6,24 @@ jobs:
build:
runs-on: ubuntu-20.04

strategy:
max-parallel: 4
matrix:
python-version: [3.9]

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
- uses: actions/checkout@v4
- name: Install uv and set up Python
uses: astral-sh/setup-uv@22695119d769bdb6f7032ad67b9bca0ef8c4a174 # v5.4.0
with:
python-version: ${{ matrix.python-version }}

- name: Give permission to run scripts
run: chmod +x ./docs/scripts/doc8_style_check.sh
version: "latest"
pyproject-file: "pyrpoject.toml"
enable-cache: true

- name: Install Dependencies
run: pip install -e .[docs]
run: |
uv sync --all-groups

- name: Check Sphinx Documentation build minimally
working-directory: ./docs
run: sphinx-build -E -W source build
run: |
uv run sphinx-build -E -W source build

- name: Check for documentation style errors
working-directory: ./docs
run: ./scripts/doc8_style_check.sh


run: |
uv run hatch run validate-docs
94 changes: 43 additions & 51 deletions .github/workflows/pypi-release.yml
Original file line number Diff line number Diff line change
@@ -1,83 +1,75 @@
name: Create library release archives, create a GH release and publish PyPI wheel and sdist on tag in main branch


# This is executed automatically on a tag in the main branch

# Summary of the steps:
# - build wheels and sdist
# - upload wheels and sdist to PyPI
# - create gh-release and upload wheels and dists there
# TODO: smoke test wheels and sdist
# TODO: add changelog to release text body

# WARNING: this is designed only for packages building as pure Python wheels
name: Build and Publish

on:
workflow_dispatch:
pull_request:
push:
tags:
- "v*.*.*"
- "v*"

jobs:
build-pypi-distribs:
name: Build and publish library to PyPI
runs-on: ubuntu-20.04
build_and_upload:
name: Build and Upload Archive
runs-on: ubuntu-24.04

steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.9

- name: Install pypa/build
run: python -m pip install build --user

- name: Build a binary wheel and a source tarball
run: python -m build --sdist --wheel --outdir dist/

- name: Upload built archives
uses: actions/upload-artifact@v4
- name: Install uv
uses: astral-sh/setup-uv@22695119d769bdb6f7032ad67b9bca0ef8c4a174 # v5.4.0
with:
enable-cache: true
pyproject-file: "pyproject.toml"

- name: Build 📦 package
run: |
uv sync
uv build
shell: bash

- name: Upload artifacts
if: github.event_name == 'push'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # 4.6.2
with:
name: pypi_archives
path: dist/*

path: dist/python_inspector-*-py3-none-any.whl
overwrite: true

create-gh-release:
name: Create GH release
needs:
- build-pypi-distribs
runs-on: ubuntu-20.04
runs-on: ubuntu-24.04
needs: build_and_upload

steps:
- name: Download built archives
uses: actions/download-artifact@v4
if: github.event_name == 'push'
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # 4.2.1
with:
name: pypi_archives
path: dist

- name: Create GH release
uses: softprops/action-gh-release@v1
if: github.event_name == 'push'
uses: softprops/action-gh-release@c95fe1489396fe8a9eb87c0abf8aa5b2ef267fda # 2.2.1
with:
draft: true
files: dist/*


create-pypi-release:
name: Create PyPI release
needs:
- create-gh-release
runs-on: ubuntu-20.04

needs: create-gh-release
runs-on: ubuntu-24.04
steps:
- name: Download built archives
uses: actions/download-artifact@v4
with:
name: pypi_archives
path: dist

- name: Publish to PyPI
if: startsWith(github.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@release/v1
- name: Install uv
if: github.event_name == 'push'
uses: astral-sh/setup-uv@22695119d769bdb6f7032ad67b9bca0ef8c4a174 # v5.4.0
with:
password: ${{ secrets.PYPI_API_TOKEN }}
enable-cache: true
pyproject-file: "pyproject.toml"

- name: Publish
if: github.event_name == 'push'
env:
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
run: |
uv publish dist/*
26 changes: 26 additions & 0 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Run Pytest with Astral Uv
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We run tests using a common azure pipelines and we are not using GH workflows for running tests. The base script would need to be updated in the https://github.com/aboutcode-org/skeleton first and ensure that this works fine.
At this stage all tests are failing also. See https://dev.azure.com/nexB/python-inspector/_build/results?buildId=15693&view=results

And there we run on linux, windows and mac in all cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, i know.
But now i'm on chicken and egg problem. I can't change skeleton without be approved and not break other projects, and at same time i need to see tests working.

Having the tests on Github cause no harm, so we can keep it.
If i have an easy way to update the skeleton without break the rest of the projects and i can effective use on the PR, would be fine.


on:
workflow_dispatch:
pull_request:

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4
- name: Install uv and set up Python ${{ matrix.python-version }}
uses: astral-sh/setup-uv@22695119d769bdb6f7032ad67b9bca0ef8c4a174 # v5.4.0
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
uv sync --group=test
- name: Run Pytest
run: |
uv run pytest -vvs
30 changes: 30 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
default_language_version:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Introducing pre-commit should be its own PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not mandatory and is not turned on by default, so no harm.
I really appreciate if we keep it to follow the README.

python: python3.9
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: check-yaml
- id: check-added-large-files

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.15.0
hooks:
- id: mypy
additional_dependencies:
- pydantic
- types-PyYaml==6.0.12.12
- types-toml
- types-requests
args: [--config, pyproject.toml]

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.11.2"
hooks:
- id: ruff

- repo: https://github.com/astral-sh/uv-pre-commit
# uv version.
rev: 0.6.10
hooks:
- id: uv-lock
65 changes: 60 additions & 5 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,73 @@ The goal of python-inspector is to be a comprehensive library
that can handle every style of Python package layouts, manifests and lockfiles.


Developing
----------

- `Install Astral Uv <https://docs.astral.sh/uv/getting-started/installation/>`_. For convenience:

- Regular: ``pip install uv``
- Isolated ( if you have pipx installed): ``pipx install uv``

- (Optional) Configure pre-commit for commit linter checks:
.. code-block:: bash

pre-commit install
pre-commit install --hook-type commit-msg

- Run from development tree. A virtual .venv will be created if you not have one

- You can run using uv direct
.. code-block:: bash

uv run python-inspector --help

- Or if you have a virtual env activated do:
.. code-block:: bash

uv sync # One single time
python-inspector --help


Testing
--------

- Run the tests with::
- Run the tests. Tests have a special dependency group with their requirements for text exclusively:

pytest -vvs
.. code-block:: bash
uv sync --group=testing

- These are live tests to regenrate the tests with updated data run::
If you want to use Uv ( which enable possibility to use multiple python versions)

.. code-block:: bash

uv run -p 3.9 pytest -vvs

Or if you have a virtualenv activated with the deps installed

.. code-block:: bash

uv run -p 3.9 pytest -vvs

- These are live tests to regenerate the tests with updated data run::

.. code-block:: shell

uv sync
PYINSP_REGEN_TEST_FIXTURES=yes uv run pytest -vvs


Documentation
-------------

.. code-block:: bash

uv sync --all-groups
hatch run validate-docs

PYINSP_REGEN_TEST_FIXTURES=yes pytest -vvs

Usage
--------
-----

- Install with pip::

Expand Down Expand Up @@ -86,6 +140,7 @@ This project is funded, supported and sponsored by:
- Google, including the Google Summer of Code and the Google Seasons of Doc programmes
- Mercedes-Benz Group
- Microsoft and Microsoft Azure
- Cariad SE
- AboutCode ASBL
- nexB Inc.

Expand Down
Loading
Loading