Skip to content

Releases: Hochfrequenz/edi_energy_scraper

v0.2.2

19 Feb 23:01
92b49c1
Compare
Choose a tag to compare

What's Changed

  • 🐛 fix negation error in metadata_has_changed by @hf-kklein in #43

This should improve the performance because there are less disk write operations in the update run.

Full Changelog: v0.2.1...v0.2.2

v0.2.1

19 Feb 22:49
a80a0b9
Compare
Choose a tag to compare

What's Changed

  • Improved resilience against ServerDisconnectedErrors and TimeoutErrors
  • Some fixed links and documentation
  • 🔄Add minimal retry logic; 🕰 Increase timeout to 15min per epoch by @hf-kklein in #42

Full Changelog: v0.2.0...v0.2.1

v0.2.0

19 Feb 21:34
47ff72b
Compare
Choose a tag to compare

What's Changed

v0.2.0 switches to an async download logic based on aiohttp.
This makes mirroring edi-energy.de significantly faster (~4x in manual tests) but also increases the risk of getting blocked when sending to many requests at a time.
In case the server suddenly terminates your connection, try lowering the connection_limit to e.g. 1 which will be very similar to the non-async code in v0.1.x (performance-wise).

Using the aiohttp-requests library comes with the price of temporarily dropped support for Python 3.11 (issue #40).
Also, using async together with BeautifulSoup was not possible in Python 3.7.

Full Changelog: v0.1.1...v0.2.0

v0.1.1

19 Feb 20:51
e26bb04
Compare
Choose a tag to compare

What's Changed

  • 🔥 Remove DOS protection by @hf-kklein in #36
  • Switch from ➖PyPdf2 to ➕pypdf by @hf-kklein in #37
  • Fix beautifulsoup deprecation warning: Use string instead of text by @hf-kklein in #35
  • Bump beautifulsoup4 from 4.11.1 to 4.11.2 by @dependabot in #33
  • 🖤👷‍♀️ (chore) Switch to Native Black Action; ⬆Bump black from 22.8 to 23.1 by @hf-kklein in #34

Full Changelog: v0.1.0...v0.1.1

v0.1.0

27 Jan 10:50
9cc6552
Compare
Choose a tag to compare

What's Changed

  • 🩹 Avoid crash for missing content-disposition header by @hf-kklein in #32

Full Changelog: v0.0.8...v0.1.0

v0.0.8

20 Jan 07:31
30522b3
Compare
Choose a tag to compare

What's Changed

Features

  • ⬆ Upgrade to Python 3.11 (both Code and CI) by @hf-kklein in #27

Dependencies

Internal Tooling / Github Actions

New Contributors

Full Changelog: v0.0.7...v0.0.8

v0.0.7

28 Sep 14:05
a81e2ee
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.0.6...v0.0.7

v0.0.6

28 Sep 05:51
84b4415
Compare
Choose a tag to compare

What's Changed

  • ⬇Make package📦officially compatible with Python 3.7 and Python 3.8 (no actual code changes) by @hf-kklein in #13

Full Changelog: v0.0.5...v0.0.6

v0.0.5

08 Sep 14:59
7fcdfe6
Compare
Choose a tag to compare

What's Changed

  • Set min versions of all dependencies in setup.cfg by @hf-kklein in #7

Full Changelog: v0.0.4...v0.0.5

v0.0.4

08 Sep 14:17
160be6f
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.0.3...v0.0.4