Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(CI/CD): bump version 0.7.3 -> 0.8.0 #102

Merged
merged 25 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
bc303c6
chore(CI/CD): bump version 0.7.3 -> 0.8.0
kraina-cicd May 8, 2024
a1a8653
docs: update CHANGELOG.md
kraina-cicd May 8, 2024
108312a
chore: simplify tests and bump minimal libraries versions
RaczeQ May 8, 2024
432c1b0
chore: change python version for smallest dependencies
RaczeQ May 8, 2024
4d43e6c
ci: disable pre-commit for manual tests
RaczeQ May 8, 2024
5e85482
chore: bump pytest-xdist
RaczeQ May 8, 2024
d7ad8f8
chore: bump geopandas version
RaczeQ May 8, 2024
da0ea92
chore: bump typeguard version
RaczeQ May 8, 2024
a8b5f25
chore: bump pytest version
RaczeQ May 8, 2024
b612add
chore: changed changelog
RaczeQ May 8, 2024
2f42657
ci: remove doctests from oldest dependencies test
RaczeQ May 8, 2024
6d144c6
chore: bump pytest-mock version
RaczeQ May 8, 2024
467d3b8
chore: bump pooch version
RaczeQ May 8, 2024
cc90adb
chore: bump tqdm version
RaczeQ May 8, 2024
6e8eb31
chore: bump osmnx version
RaczeQ May 8, 2024
6cdcdf2
chore: bump osmnx and s2 version
RaczeQ May 8, 2024
2751e52
chore: remove h3ronpy from dependencies
RaczeQ May 8, 2024
1ab3e2d
chore: bump osmnx version
RaczeQ May 8, 2024
55ec0be
fix: change h3 to polygon transformation
RaczeQ May 8, 2024
7518ab2
chore: bump psutil version
RaczeQ May 8, 2024
8a2cdcc
chore: bump polars version
RaczeQ May 8, 2024
9fe66ff
chore: bump polars version
RaczeQ May 8, 2024
479a54b
chore: bump pyarrow and pooch version
RaczeQ May 8, 2024
99c4e87
chore: change readme
RaczeQ May 8, 2024
919dfb1
chore: change changelog
RaczeQ May 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 6 additions & 23 deletions .github/workflows/manual_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: ["3.9", "3.10", "3.11", "3.12"]
include:
- os: macos-latest
python-version: "3.12"
- os: macos-13
python-version: "3.12"
- os: windows-latest
python-version: "3.12"
os: [ubuntu-latest, macos-13, macos-latest, windows-latest]
python-version: ["3.12"]
env:
OS: ${{ matrix.os }}
PYTHON: ${{ matrix.python-version }}
Expand All @@ -31,29 +24,21 @@ jobs:
- name: Generate lock with newest dependencies
run: pdm lock --lockfile pdm.newest.lock --strategy no_cross_platform -dG:all
- name: Install quackosm and tests dependencies
run: pdm install --lockfile pdm.newest.lock -dG:all
run: pdm install --lockfile pdm.newest.lock -dG:all --skip=post_install
- name: Run tests with pytest
run: |
pdm run pytest -v -s --durations=20 --doctest-modules --doctest-continue-on-failure quackosm
pdm run pytest -v -s --durations=20 tests/base
pdm run pytest -v -s --durations=20 tests/optional_imports
pdm run pytest -v -s --durations=20 tests/benchmark

run-tests-oldest:
name: Run tests 🛠️ on multiple systems 🖥️ and Python 🐍 versions (oldest dependencies)
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: ["3.9", "3.10", "3.11", "3.12"]
include:
- os: macos-latest
python-version: "3.12"
- os: macos-13
python-version: "3.12"
- os: windows-latest
python-version: "3.12"
os: [ubuntu-latest, macos-13, macos-latest, windows-latest]
python-version: ["3.9"]
env:
OS: ${{ matrix.os }}
PYTHON: ${{ matrix.python-version }}
Expand All @@ -68,10 +53,8 @@ jobs:
- name: Generate lock with oldest dependencies
run: pdm lock --lockfile pdm.oldest.lock --strategy no_cross_platform,direct_minimal_versions -dG:all
- name: Install srai and tests dependencies
run: pdm install --lockfile pdm.oldest.lock -dG:all
run: pdm install --lockfile pdm.oldest.lock -dG:all --skip=post_install
- name: Run tests with pytest
run: |
pdm run pytest -v -s --durations=20 --doctest-modules --doctest-continue-on-failure quackosm
pdm run pytest -v -s --durations=20 tests/base
pdm run pytest -v -s --durations=20 tests/optional_imports
pdm run pytest -v -s --durations=20 tests/benchmark
14 changes: 12 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.8.0] - 2024-05-08

### Added

- Polars library to the main dependencies
Expand All @@ -16,14 +18,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Refactored ways grouping logic from duckdb to polars `LazyFrame` API for faster operations
- Default result file extension from `geoparquet` to `parquet` [#99](https://github.com/kraina-ai/quackosm/issues/99)
- Moved `rich` to the main dependencies [#95](https://github.com/kraina-ai/quackosm/issues/95)
- Set minimal versions of multiple dependencies
- Added tests for minimal dependencies versions

### Fixed

- Steps numbering after encountering `MemoryError`

### Removed

- `h3ronpy` from dependencies and replaced logic with pure `h3` calls

### Deprecated

- Usage of `geoparquet` files as default extension, reusing existing `geoparquet` files from cache will be supported with deprecation warning [#99](https://github.com/kraina-ai/quackosm/issues/99)
- Reusing existing `geoparquet` files from cache will be supported, but will result in deprecation warning [#99](https://github.com/kraina-ai/quackosm/issues/99)

## [0.7.3] - 2024-05-07

Expand Down Expand Up @@ -254,7 +262,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Created QuackOSM repository
- Implemented PbfFileReader

[Unreleased]: https://github.com/kraina-ai/quackosm/compare/0.7.3...HEAD
[Unreleased]: https://github.com/kraina-ai/quackosm/compare/0.8.0...HEAD

[0.8.0]: https://github.com/kraina-ai/quackosm/compare/0.7.3...0.8.0

[0.7.3]: https://github.com/kraina-ai/quackosm/compare/0.7.2...0.7.3

Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,23 +72,25 @@ Required:

- `duckdb (>=0.10.2)`: For all DuckDB operations on PBF files

- `pyarrow (>=13.0.0)`: For parquet files wrangling
- `pyarrow (>=14.0.0)`: For parquet files wrangling

- `pyarrow-ops`: For easy removal of duplicated features in parquet files

- `geoarrow-pyarrow (>=0.1.2)`: For GeoParquet IO operations

- `geopandas`: For returning GeoDataFrames and reading Geo files
- `geopandas (>=0.6)`: For returning GeoDataFrames and reading Geo files

- `shapely (>=2.0)`: For parsing WKT and GeoJSON strings and fixing geometries

- `typeguard`: For internal validation of types
- `polars (>=0.19.4)`: For faster OSM ways grouping operation

- `psutil`: For automatic scaling of parameters based on available resources
- `typeguard (>=3.0)`: For internal validation of types

- `pooch`: For downloading `*.osm.pbf` files
- `psutil (>=5.6.2)`: For automatic scaling of parameters based on available resources

- `tqdm`: For showing progress bars
- `pooch (>=1.6.0)`: For downloading `*.osm.pbf` files

- `rich (>=10.11.0)` & `tqdm (>=4.42.0)`: For showing progress bars

- `requests`: For iterating OSM PBF files services

Expand All @@ -99,15 +101,13 @@ Optional:

- `typer[all] (>=0.9.0)` (click, colorama, rich, shellingham): For CLI

- `osmnx`: For geocoding of strings in CLI
- `osmnx (>=1.3.0)`: For geocoding of strings in CLI

- `h3 (>=4.0.0b1)`: For reading H3 strings in CLI

- `h3ronpy (>=0.18.0)`: For transforming H3 indexes into geometries

- `s2`: For transforming S2 indexes into geometries
- `s2 (>=0.1.9)`: For transforming S2 indexes into geometries

- `python-geohash`: For transforming GeoHash indexes into geometries
- `python-geohash (>=0.8)`: For transforming GeoHash indexes into geometries

## Usage

Expand Down
16 changes: 8 additions & 8 deletions pdm.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 17 additions & 18 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
[project]
name = "QuackOSM"
version = "0.7.3"
version = "0.8.0"
description = "An open-source tool for reading OpenStreetMap PBF files using DuckDB"
authors = [{ name = "Kamil Raczycki", email = "kraczycki@kraina.ai" }]
dependencies = [
"geopandas",
"geopandas>=0.6",
"shapely>=2",
"pyarrow>=13.0.0",
"pyarrow>=14.0.0",
"duckdb>=0.10.2",
"geoarrow-pyarrow>=0.1.2",
"typeguard",
"psutil",
"pooch",
"tqdm",
"typeguard>=3.0.0",
"psutil>=5.6.2",
"pooch>=1.6.0",
"tqdm>=4.42.0",
"beautifulsoup4",
"pyarrow-ops",
"requests",
"polars",
"polars>=0.19.4",
"rich>=10.11.0",
]
requires-python = ">=3.9"
Expand Down Expand Up @@ -48,10 +48,9 @@ Changelog = "https://github.com/kraina-ai/quackosm/blob/main/CHANGELOG.md"
[project.optional-dependencies]
cli = [
"typer[all]>=0.9.0",
"osmnx",
"osmnx>=1.3.0",
"h3>=4.0.0b1",
"h3ronpy>=0.18.0",
"s2",
"s2>=0.1.9",
"python-geohash>=0.8",
]

Expand All @@ -73,16 +72,16 @@ dev = [
"setuptools>=45.0.0",
]
# pdm add -dG lint <library>
lint = ["pre-commit", "mypy", "docformatter[tomli]", "ruff"]
lint = ["pre-commit", "mypy>=1", "docformatter[tomli]", "ruff>=0.1.0"]
# pdm add -dG test <library>
test = [
"pytest",
"pytest>=7.0.0",
"tox-pdm",
"pytest-mock",
"pytest-mock>=3.3.0",
"requests-mock",
"pytest-check",
"pytest-parametrization",
"pytest-xdist",
"pytest-xdist>=3.4.0",
"pytest-doctestplus",
"srai>=0.6.2",
]
Expand All @@ -96,15 +95,15 @@ docs = [
"ipykernel",
"mkdocs-gen-files",
"mkdocs-awesome-pages-plugin",
"mike<2",
"mike>=1,<2",
"black",
"jupyter",
"nbconvert",
"nbformat",
"notebook",
"osmnx",
"matplotlib",
"seaborn",
"seaborn>=0.10.0",
"py-cpuinfo",
"adjustText",
]
Expand Down Expand Up @@ -180,7 +179,7 @@ close-quotes-on-newline = true
wrap-one-line = true

[tool.bumpver]
current_version = "0.7.3"
current_version = "0.8.0"
version_pattern = "MAJOR.MINOR.PATCH[PYTAGNUM]"
commit_message = "chore(CI/CD): bump version {old_version} -> {new_version}"
commit = true
Expand Down
2 changes: 1 addition & 1 deletion quackosm/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
from quackosm.pbf_file_reader import PbfFileReader

__app_name__ = "QuackOSM"
__version__ = "0.7.3"
__version__ = "0.8.0"

__all__ = [
"PbfFileReader",
Expand Down
13 changes: 8 additions & 5 deletions quackosm/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
from click import Argument
from click.exceptions import MissingParameter
from geohash import bbox as geohash_bbox
from h3ronpy.arrow.vector import cells_to_wkb_polygons
from s2 import s2
from shapely import from_geojson, from_wkt
from shapely.geometry import Polygon, box
Expand Down Expand Up @@ -145,10 +144,14 @@ def convert(self, value, param=None, ctx=None): # type: ignore
return None

try:
h3_int_indexes = [h3.str_to_int(h3_cell.strip()) for h3_cell in value.split(",")]
return gpd.GeoSeries.from_wkb(cells_to_wkb_polygons(h3_int_indexes)).unary_union
except Exception:
raise typer.BadParameter(f"Cannot parse provided H3 values: {value}") from None
geometries = [] # noqa: FURB138
for h3_cell in value.split(","):
geometries.append(
Polygon([coords[::-1] for coords in h3.cell_to_boundary(h3_cell.strip())])
)
return gpd.GeoSeries(geometries).unary_union
except Exception as ex:
raise typer.BadParameter(f"Cannot parse provided H3 values: {value}") from ex


class S2GeometryParser(click.ParamType): # type: ignore
Expand Down
Loading