Releases: Hochfrequenz/edi_energy_scraper
v0.2.2
What's Changed
- 🐛 fix negation error in
metadata_has_changed
by @hf-kklein in #43
This should improve the performance because there are less disk write operations in the update run.
Full Changelog: v0.2.1...v0.2.2
v0.2.1
What's Changed
- Improved resilience against
ServerDisconnectedError
s andTimeoutError
s - Some fixed links and documentation
- 🔄Add minimal retry logic; 🕰 Increase timeout to 15min per epoch by @hf-kklein in #42
Full Changelog: v0.2.0...v0.2.1
v0.2.0
What's Changed
v0.2.0 switches to an async download logic based on aiohttp.
This makes mirroring edi-energy.de significantly faster (~4x in manual tests) but also increases the risk of getting blocked when sending to many requests at a time.
In case the server suddenly terminates your connection, try lowering the connection_limit to e.g. 1
which will be very similar to the non-async code in v0.1.x (performance-wise).
Using the aiohttp-requests library comes with the price of temporarily dropped support for Python 3.11 (issue #40).
Also, using async together with BeautifulSoup was not possible in Python 3.7.
- ❌ Drop Support for Python 3.7 by @hf-kklein in #39
- (❌) Temporary drop support for Python 3.11 by @hf-kklein in #41
- 🚀 Make download code async by @hf-kklein in #38
Full Changelog: v0.1.1...v0.2.0
v0.1.1
What's Changed
- 🔥 Remove DOS protection by @hf-kklein in #36
- Switch from ➖PyPdf2 to ➕pypdf by @hf-kklein in #37
- Fix beautifulsoup deprecation warning: Use
string
instead oftext
by @hf-kklein in #35 - Bump beautifulsoup4 from 4.11.1 to 4.11.2 by @dependabot in #33
- 🖤👷♀️ (chore) Switch to Native Black Action; ⬆Bump black from 22.8 to 23.1 by @hf-kklein in #34
Full Changelog: v0.1.0...v0.1.1
v0.1.0
What's Changed
- 🩹 Avoid crash for missing
content-disposition
header by @hf-kklein in #32
Full Changelog: v0.0.8...v0.1.0
v0.0.8
What's Changed
Features
- ⬆ Upgrade to Python 3.11 (both Code and CI) by @hf-kklein in #27
Dependencies
- Bump pypdf2 from 2.10.5 to 2.11.1 by @dependabot in #15
- Bump certifi from 2022.6.15 to 2022.12.7 by @dependabot in #22
- Bump pypdf2 from 2.11.1 to 2.12.1 by @dependabot in #23
- Bump requests from 2.28.1 to 2.28.2 by @dependabot in #26
- Bump pypdf2 from 2.12.1 to 3.0.1 by @dependabot in #25
Internal Tooling / Github Actions
- 🤖Update Github Actions using Dependabot by @hf-kklein in #16
- Bump actions/setup-python from 2 to 4 by @dependabot in #19
- Bump github/codeql-action from 1 to 2 by @dependabot in #17
- Bump actions/checkout from 2 to 3 by @dependabot in #18
New Contributors
- @dependabot made their first contribution in #15
Full Changelog: v0.0.7...v0.0.8
v0.0.7
v0.0.6
What's Changed
- ⬇Make package📦officially compatible with Python 3.7 and Python 3.8 (no actual code changes) by @hf-kklein in #13
Full Changelog: v0.0.5...v0.0.6
v0.0.5
What's Changed
- Set min versions of all dependencies in setup.cfg by @hf-kklein in #7
Full Changelog: v0.0.4...v0.0.5
v0.0.4
What's Changed
- Make Package available to Python 3.9 by @hf-kklein in #6
Full Changelog: v0.0.3...v0.0.4