sync upstream #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Extract all sta.sh items in a single extractor run. Don't spawn a new StashExtractor for each individual sta.sh item to preserve the current requests.Session and its opened TCP connections.
'/extended_fetch' as well as Deviation webpages now again contain Deviation UUIDs needed to grab Deviation info through the OAuth API, meaning cookies are no longer necessary to grab original files. The only instance were cookies are still needed are scraps marked as "mature", since those entries are hidden for public users. (#655, #657, #660)
- add a 2 second wait time between requests to deviantart.com - catch 403 "Request blocked" errors and wait for 3 minutes until retrying
"Request blocked" can also happen on sta.sh and for *any* HTTP request directed at deviantart.com
- allow 'until' to be a datetime object - do "time calculations" with UTC timestamps - set a default 'reason'
Calls to config.clear() from other tests are removing the API credentials set when importing mastodon.py for the first time.
i.e. /g4/data.php?id=… - get filename & extension from Content-Disposition header - handle all downloadable file types (docx, swf, etc)
'q_\d+' would sometimes also replace something in the 'token' query parameter, invalidating the URL.
There are currently no situations where forwarding gallery-dl's cookies to youtube-dl is necessary, and it only causes problems when forcing youtube-dl for Twitter video downloads while logged in.
- move code into its own function - add enumeration index to filenames - dump responses regardless of status code
* [imagechest] Add new extractor for ImageChest * [imagechest] Fix flake8 compliance issues
The webtoons extractor can extract episode and entire comic (all
episodes) from webtoons.com.
All the logic of the extractors should be trivial except for a couple
of kludges needed:
- `ageGatePass' cookie is always set to avoid possible redirect and stop of
extraction, especially in the comic extractor
- The image URLs returned by the episode extractor could not be fetched
directly and the `Referer:' HTTP header needs to be passed to fetch them
Close #593.
use 'pool:<pool id>' as search tag to get pool posts
Wrap all loggers used by job, extractor, downloader, and postprocessor objects into a (custom) LoggerAdapter that provides access to the underlying job, extractor, pathfmt, and kwdict objects and their properties. __init__() signatures for all downloader and postprocessor classes have been changed to take the current Job object as their first argument, instead of the current extractor or pathfmt. (#574, #575)
Downloading the pre-rendered versions should be a better default than .zip files with individual frames.
- include 'igneous' and 'hath_perks' in Exhentai cookies - add an example of how to write DeviantArt description to file - add a 'path-restrict' mapping from invalid characters in Windows paths to Unicode alternatives (taken from #662)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.