Request: Expose the import to conda package name mapping table in a more cost-effective way.

[schuylermartin45]

I have a need in conda-recipe-manager to acquire a mapping of import names to conda package names.

Currently, this project supports querying the API for a single string look-up:

conda-forge-metadata/conda_forge_metadata/autotick_bot/import_to_pkg.py

Line 104 in 18af7dc

def map_import_to_package(import_name: str) -> str:

For what we plan to do with conda-recipe-manager, this could easily turn into a very large number (at least in the thousands) of API requests a day. For data that seems to change pretty infrequently, that seems excessive and costly.

I would much rather have a local cache of this mapping data, but periodically update it through a new API in this project.

The current size of the JSON file containing this mapping in cf-countyfair is ~800kb. I have gotten that down to about 300kb by removing some redundant fields. If we expect this list to grow significantly, the new query may need to be built with pagination in mind.

See this conversation for additional context: conda-incubator/conda-recipe-manager#218

tl;dr
I would like to request a new API endpoint that exposes the import mapping data currently available in cf-countyfair.

[maresb]

Regarding the caching, I just added some logic to conda-lock for caching the lookup table.

https://github.com/conda/conda-lock/blob/main/conda_lock/lookup_cache.py
https://github.com/conda/conda-lock/blob/main/tests/test_lookup_cache.py

In practice with the download being so tiny, I'm not sure how worthwhile this is, but I'd be happy to add it to conda-recipe-manager.

[schuylermartin45]

I'm less concerned about the size and more concerned about redundancy and how frequently I will need to hit the API.

The fact that it is so small and doesn't get updated often makes me think a local cache is very reasonable.

[beckermr]

Feel free to use a local cache, but conda-recipe-manager needs to pull its cache updates using an api (that we need to add of course) in this package.