Deprecated Flag for Uncrawlable Publishers

[addie9800]

This PR adds a flag for uncrawlable publishers. The Publisher Coverage Script is updated to skip publishers marked as deprecated. Also, deprecated publishers are marked as such by strike-through in the supported_publishers.md

[MaxDall]

Thanks for adding! Looks awesome 👍

[MaxDall]

@addie9800 I moved the ignore_deprecated flag from the crawl method to Crawler. The paradigm of crawl is to only include parameters affecting all crawlers (forward, backward). Since the depreciation flag does not serve any purpose regarding backward crawling, I decided to move it to the crawler. What do you think about that?

[addie9800]

@addie9800 I moved the ignore_deprecated flag from the crawl method to Crawler. The paradigm of crawl is to only include parameters affecting all crawlers (forward, backward). Since the depreciation flag does not serve any purpose regarding backward crawling, I decided to move it to the crawler. What do you think about that?

I think it's a good idea. I also updated the docs a bit. Maybe you can also say what you think about that

[MaxDall]

@addie9800 thanks for updating the documentation. Unfortunately the latest documentation commit breaks a lot of links. Also tutorial 4 now seems completely overwhelming and out of focus while tutorial 5 looks like it was cannibalized. The purpose of tutorial 4 is to enable the user to filter for specific articles. search or deprecatrd isn't as nearly as important here. The latter ones are mostly for convenience. I would let tutorial 4 as it was and add the deprecation mechanica to the former tutorial 5.

[addie9800]

Oh, and I thought I had caught the broken links. Well, I undid the previous changes and just appended it to section 5 of the tutorial

[MaxDall]

Thanks a lot! Looks really great :)