Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reducing dependencies #259

Open
ashleysommer opened this issue Aug 27, 2024 · 2 comments
Open

Reducing dependencies #259

ashleysommer opened this issue Aug 27, 2024 · 2 comments

Comments

@ashleysommer
Copy link
Contributor

ashleysommer commented Aug 27, 2024

I notice Shapely is listed in the dependencies of Prez.

One big issue with having Shapely as a dependency is shapely subsequently depends on Numpy, which is a very large dependency and often needs to be compiled on installation (if the target platform does not use binary wheels).

After searching through the codebase, I see Shapely is only used in one location, that is to convert cql->Polygon->wkt String.

I recently made a PR in the rdf2geojson repo to replace the use of Shapely with a vendorised copy of GeoMet.
https://github.com/Kurrawong/rdf2geojson/pull/1
Note, it is vendorised so we can use it without the sub-dependency on "click", that we don't need. As well as applying some out-of-tree patches to fix multiple WKT serialization errors.

GeoMet is pure-python, is (relatively) small and is great at converting from GeoJSON to WKT and back again (and some other formats). It is compatible with the Python Geo Interface convention (__geo_interface__) so should work for our use case in Prez.


Secondly, uvicorn is included as a non-optional requirement in the dependencies list. Again, the main issue is the sub-dependency on uvloop that again is a rather large dependency that also often needs to be compiled on installation (if the target platform does not use binary wheels).

Those who are familiar with the codebase will know that uvicorn is used in exactly one place in the codebase, that is at the bottom of main.py, if you are running prez as a standalone application it will spawn uvicorn to run the app.

There are lots of other ways of running the FastAPI ASGI app other than uvicorn (eg, in a Azure Function App, a AWS Lambda app, Apache ASGI wrapper, another ASGI runner like hypercorn or daphne). So I believe uvicorn should be an optional dependency, because not everyone is going to want Prez to supply its own server.


The motivation for these changes is to reduce the size of the requirements.txt file generated by poetry:

poetry export --format=requirements.txt > requirements.txt"

This is used when generating the requirements for building the Azure Function App bundle, and I feel the total built bundle size could be smaller.

@ashleysommer
Copy link
Contributor Author

Note, this is the whole requirements.txt that is currently generated when exporting requirements from poetry.

aiocache==0.12.2 ; python_version >= "3.11" and python_version < "4.0"
annotated-types==0.7.0 ; python_version >= "3.11" and python_version < "4.0"
anyio==4.4.0 ; python_version >= "3.11" and python_version < "4.0"
cachetools==5.4.0 ; python_version >= "3.11" and python_version < "4.0"
certifi==2024.7.4 ; python_version >= "3.11" and python_version < "4.0"
click==8.1.7 ; python_version >= "3.11" and python_version < "4.0"
colorama==0.4.6 ; python_version >= "3.11" and python_version < "4.0" and (platform_system == "Windows" or sys_platform == "win32")
dnspython==2.6.1 ; python_version >= "3.11" and python_version < "4.0"
email-validator==2.2.0 ; python_version >= "3.11" and python_version < "4.0"
fastapi-cli==0.0.5 ; python_version >= "3.11" and python_version < "4.0"
fastapi==0.111.1 ; python_version >= "3.11" and python_version < "4.0"
frozendict==2.4.4 ; python_version >= "3.11" and python_version < "4.0"
h11==0.14.0 ; python_version >= "3.11" and python_version < "4.0"
httpcore==1.0.5 ; python_version >= "3.11" and python_version < "4.0"
httptools==0.6.1 ; python_version >= "3.11" and python_version < "4.0"
httpx==0.27.0 ; python_version >= "3.11" and python_version < "4.0"
idna==3.7 ; python_version >= "3.11" and python_version < "4.0"
isodate==0.6.1 ; python_version >= "3.11" and python_version < "4.0"
jinja2==3.1.4 ; python_version >= "3.11" and python_version < "4.0"
lxml==5.3.0 ; python_version >= "3.11" and python_version < "4.0"
markdown-it-py==3.0.0 ; python_version >= "3.11" and python_version < "4.0"
markupsafe==2.1.5 ; python_version >= "3.11" and python_version < "4.0"
mdurl==0.1.2 ; python_version >= "3.11" and python_version < "4.0"
numpy==2.0.1 ; python_version >= "3.11" and python_version < "4.0"
oxrdflib==0.3.7 ; python_version >= "3.11" and python_version < "4.0"
pydantic-core==2.20.1 ; python_version >= "3.11" and python_version < "4.0"
pydantic-settings==2.4.0 ; python_version >= "3.11" and python_version < "4.0"
pydantic==2.8.2 ; python_version >= "3.11" and python_version < "4.0"
pygments==2.18.0 ; python_version >= "3.11" and python_version < "4.0"
pyld==2.0.4 ; python_version >= "3.11" and python_version < "4.0"
pyoxigraph==0.3.22 ; python_version >= "3.11" and python_version < "4.0"
pyparsing==3.1.2 ; python_version >= "3.11" and python_version < "4.0"
python-dotenv==1.0.1 ; python_version >= "3.11" and python_version < "4.0"
python-multipart==0.0.9 ; python_version >= "3.11" and python_version < "4.0"
pyyaml==6.0.2 ; python_version >= "3.11" and python_version < "4.0"
rdflib==7.0.0 ; python_version >= "3.11" and python_version < "4.0"
rich==13.7.1 ; python_version >= "3.11" and python_version < "4.0"
shapely==2.0.5 ; python_version >= "3.11" and python_version < "4.0"
shellingham==1.5.4 ; python_version >= "3.11" and python_version < "4.0"
six==1.16.0 ; python_version >= "3.11" and python_version < "4.0"
sniffio==1.3.1 ; python_version >= "3.11" and python_version < "4.0"
sparql-grammar-pydantic==0.1.2 ; python_version >= "3.11" and python_version < "4.0"
starlette==0.37.2 ; python_version >= "3.11" and python_version < "4.0"
toml==0.10.2 ; python_version >= "3.11" and python_version < "4.0"
typer==0.12.3 ; python_version >= "3.11" and python_version < "4.0"
typing-extensions==4.12.2 ; python_version >= "3.11" and python_version < "4.0"
uvicorn==0.30.6 ; python_version >= "3.11" and python_version < "4.0"
uvicorn[standard]==0.30.6 ; python_version >= "3.11" and python_version < "4.0"
uvloop==0.19.0 ; (sys_platform != "win32" and sys_platform != "cygwin") and platform_python_implementation != "PyPy" and python_version >= "3.11" and python_version < "4.0"
watchfiles==0.23.0 ; python_version >= "3.11" and python_version < "4.0"
websockets==12.0 ; python_version >= "3.11" and python_version < "4.0"

@ashleysommer
Copy link
Contributor Author

Additional note, if Prez adds the rdf2geojson dependency (which it may do when implementing geojson serialization support), you can probably use the GeoMet version that is bundled with rdf2geojson.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🆕 New
Development

No branches or pull requests

1 participant