Speedup: Replace modules and use lazy imports

[hugovk]

Checking the import times (see https://medium.com/alan/how-we-improved-our-python-backend-start-up-time-2c33cd4873c8) with:

python -m pip install tuna

python -X importtime -c 'import pypistats; pypistats.recent("pillow")' 2> import0.log
tuna import0.log

This test case is when using default Markdown format and the data has been cached.

main

1.014s total: shows importing pytablewriter (0.433s, 42.7%), pkg_resources (0.270s, 26.7%) and httpx (0.243s, 24.0%) are slow.

Use faster PrettyTable for Markdown

Use PrettyTable for default Markdown, use pytablewriter for the more complex formats. Only import each when needed.

We remove 0.433s for pytablewriter but only add 0.011s for PrettyTable.

Down to 0.627s.

Use faster importlib.metadata for getting version

Instead of importing pkg_resources to get the package version (because we use setuptools_scm), use the much faster importlib.metadata (and backport for pre-3.8).

https://github.com/pypa/setuptools_scm#retrieving-package-version-at-runtime

We remove 0.272s for pkg_resources but only add 0.010s for importlib.metadata.

Down to 0.378s.

Only import httpx if needed

If we happen to be reading a cached data file, we don't need to import httpx to fetch a new one.

Down to 0.139s.

1.014s -> 0.139s = 7.3x faster.

Another test

$ python --version
Python 3.10.0

$ git checkout main
Already on 'main'
Your branch is up to date with 'origin/main'.
$ time python -m timeit -n 1000 'import pypistats; pypistats.recent("pillow")'
1000 loops, best of 5: 18.8 msec per loop
python3 -m timeit -n 1000 'import pypistats; pypistats.recent("pillow")'  91.74s user 1.75s system 94% cpu 1:38.99 total

$ git checkout speedup
Switched to branch 'speedup'
Your branch is up to date with 'origin/speedup'.
$ time python -m timeit -n 1000 'import pypistats; pypistats.recent("pillow")'
1000 loops, best of 5: 426 usec per loop
python3 -m timeit -n 1000 'import pypistats; pypistats.recent("pillow")'  2.21s user 0.50s system 91% cpu 2.951 total

= 44x faster per loop.

And another

$ git checkout main
Already on 'main'
Your branch is up to date with 'origin/main'.
$ time pypistats recent pillow
| last_day  | last_month | last_week |
| --------: | ---------: | --------: |
| 1,362,501 | 35,373,628 | 8,292,701 |

pypistats recent pillow  1.53s user 0.35s system 101% cpu 1.846 total

$ gco speedup
Switched to branch 'speedup'
Your branch is up to date with 'origin/speedup'.
$ time pypistats recent pillow
|  last_day | last_month | last_week |
|----------:|-----------:|----------:|
| 1,362,501 | 35,373,628 | 8,292,701 |

pypistats recent pillow  0.16s user 0.06s system 90% cpu 0.244 total

= 9.5625x faster.

[codecov]

Codecov Report

Merging #289 (501b704) into main (d9ba837) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #289      +/-   ##
==========================================
+ Coverage   99.58%   99.60%   +0.01%     
==========================================
  Files           8        8              
  Lines         726      754      +28     
==========================================
+ Hits          723      751      +28     
  Misses          3        3              

Flag	Coverage Δ
GHA_Ubuntu	99.60% <100.00%> (+0.01%)	⬆️
GHA_Windows	99.60% <100.00%> (+0.01%)	⬆️
GHA_macOS	99.60% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
tests/data/expected_tabulated.py	100.00% <ø> (ø)
tests/test_pypistats_cache.py	100.00% <ø> (ø)
src/pypistats/__init__.py	99.25% <100.00%> (+0.08%)	⬆️
tests/test_pypistats.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d9ba837...501b704. Read the comment docs.