Sourcery Starbot ⭐ refactored adbar/trafilatura

[SourceryAI]

Thanks for starring sourcery-ai/sourcery ✨ 🌟 ✨

Here's your pull request refactoring your most popular Python repo.

If you want Sourcery to refactor all your Python repos and incoming pull requests install our bot.

Review changes via command line

To manually merge these changes, make sure you're on the master branch, then run:

git fetch https://github.com/sourcery-ai-bot/trafilatura master
git merge --ff-only FETCH_HEAD
git reset HEAD^

[codecov-commenter]

Codecov Report

Merging #121 (f69c668) into master (a3ba05e) will decrease coverage by 0.02%.
The diff coverage is 92.47%.

@@            Coverage Diff             @@
##           master     #121      +/-   ##
==========================================
- Coverage   94.74%   94.71%   -0.03%     
==========================================
  Files          19       19              
  Lines        2663     2630      -33     
==========================================
- Hits         2523     2491      -32     
+ Misses        140      139       -1     

Impacted Files	Coverage Δ
trafilatura/lru.py	100.00% <ø> (ø)
trafilatura/htmlprocessing.py	95.95% <80.00%> (ø)
trafilatura/json_metadata.py	86.32% <87.50%> (-0.23%)	⬇️
trafilatura/sitemaps.py	92.90% <87.50%> (-0.15%)	⬇️
trafilatura/core.py	96.96% <90.00%> (-0.02%)	⬇️
trafilatura/metadata.py	97.83% <91.66%> (-0.02%)	⬇️
trafilatura/cli.py	99.15% <100.00%> (-0.03%)	⬇️
trafilatura/cli_utils.py	91.26% <100.00%> (ø)
trafilatura/downloads.py	93.02% <100.00%> (ø)
trafilatura/external.py	93.02% <100.00%> (-0.24%)	⬇️
... and 6 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3ba05e...f69c668. Read the comment docs.

[sourcery-ai]

Sourcery Code Quality Report

✅  Merging this PR will increase code quality in the affected files by 0.22%.

Quality metrics	Before	After	Change
Complexity	19.06 😞	18.39 😞	-0.67 👍
Method Length	111.92 🙂	111.20 🙂	-0.72 👍
Working memory	11.67 😞	11.67 😞	0.00
Quality	50.53% 🙂	50.75% 🙂	0.22% 👍

Other metrics	Before	After	Change
Lines	5881	5922	41

Changed files	Quality Before	Quality After	Quality Change
tests/comparison.py	49.45% 😞	48.45% 😞	-1.00% 👎
tests/downloads_tests.py	69.89% 🙂	70.68% 🙂	0.79% 👍
tests/eval_authors.py	66.81% 🙂	66.90% 🙂	0.09% 👍
tests/realworld_tests.py	21.30% ⛔	21.15% ⛔	-0.15% 👎
tests/unit_tests.py	65.21% 🙂	65.26% 🙂	0.05% 👍
trafilatura/cli.py	43.03% 😞	44.47% 😞	1.44% 👍
trafilatura/cli_utils.py	60.30% 🙂	60.32% 🙂	0.02% 👍
trafilatura/core.py	33.42% 😞	34.14% 😞	0.72% 👍
trafilatura/downloads.py	65.86% 🙂	66.24% 🙂	0.38% 👍
trafilatura/external.py	71.01% 🙂	72.91% 🙂	1.90% 👍
trafilatura/feeds.py	42.52% 😞	43.24% 😞	0.72% 👍
trafilatura/filters.py	69.05% 🙂	68.55% 🙂	-0.50% 👎
trafilatura/htmlprocessing.py	50.31% 🙂	50.55% 🙂	0.24% 👍
trafilatura/json_metadata.py	32.47% 😞	32.95% 😞	0.48% 👍
trafilatura/lru.py	77.56% ⭐	77.68% ⭐	0.12% 👍
trafilatura/metadata.py	45.18% 😞	46.17% 😞	0.99% 👍
trafilatura/sitemaps.py	57.57% 🙂	58.58% 🙂	1.01% 👍
trafilatura/spider.py	66.04% 🙂	66.38% 🙂	0.34% 👍
trafilatura/utils.py	62.73% 🙂	63.05% 🙂	0.32% 👍
trafilatura/xml.py	49.35% 😞	49.71% 😞	0.36% 👍

Here are some functions in these files that still need a tune-up:

File	Function	Complexity	Length	Working Memory	Quality	Recommendation
trafilatura/core.py	bare_extraction	39 ⛔	477 ⛔	27 ⛔	7.88% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
trafilatura/json_metadata.py	extract_json	131 ⛔	438 ⛔	16 ⛔	8.34% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
trafilatura/metadata.py	examine_meta	43 ⛔	331 ⛔	26 ⛔	8.92% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
tests/realworld_tests.py	test_extract	90 ⛔	1880 ⛔	11 😞	14.80% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions
trafilatura/core.py	extract_content	39 ⛔	363 ⛔	16 ⛔	15.16% ⛔	Refactor to reduce nesting. Try splitting into smaller methods. Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

• ⭐ excellent
• 🙂 good
• 😞 poor
• ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.

---

Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

[adbar]

✔️ seems ready to merge