Improve Python build process and streamline CI #600
Conversation
thomas-maschler
force-pushed
the
python/spark-3.3
branch
from
d53a96c to
40733ec
Compare
thomas-maschler
changed the title
python/tests/GeotrellisTests.py
Outdated
|
|
||
| import pytest | ||
|
|
||
| on_circle_ci = os.environ.get("CIRCLECI", "false") == "true" |
There was a problem hiding this comment.
Is CircleCI still in use?
| # TODO: Where does this go, do we need it? | ||
| # - name: upload artefacts | ||
| # uses: ./.github/actions/upload_artefacts | ||
|
|
||
| # TODO: Where does this go, do we need it? | ||
| # - uses: actions/cache@v3 | ||
| # with: | ||
| # path: ./dist/* | ||
| # key: dist-${{ github.sha }} | ||
|
|
||
| # TODO: Where does this go? | ||
| # - name: upload wheel | ||
| # working-directory: dist | ||
| # shell: bash | ||
| # run: |
There was a problem hiding this comment.
@echeipesh what is the correct workflow here?
There was a problem hiding this comment.
The actions/cache@v3 is an optimization to pull the assembly from workflow cache. We don't need it. It's alright or even better to rebuild it.
I'm not sure where the wheel goes yet, PyPi eventually but I'll need to figure out how that happens.
the artefacts bit is for debugging failed CI. I never used that part of the CI. So much has changed and I think will change a little more that I'm not worried about it. I'm sure when we need it we will wire it right up.
Overall I think the CI is good and even green. I'm a little surprised we didn't end up needing GDAL for the tests though.
Changes made:
/pythonfolderpoetryandpyproject.tomlto manage dependencies and project metadataMakefileto manage main dev and CI events for both Scala and PythonBlackandiSortNote: This PR allows to build python wheel separately from the python documentation. Building the Python Markdown documentation still fails due to invalid URLs in the docs. Fixing the Python documentation should be covered in its own PR.