mo-han · mo-han · May 26, 2020 · Mar 31, 2020 · Mar 31, 2020 · Mar 31, 2020
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,63 @@
 # Changelog
 
+## Unreleased
+
+## 1.13.6 - 2020-05-02
+### Additions
+- [patreon] respect filters and sort order in query parameters ([#711](https://github.com/mikf/gallery-dl/issues/711))
+- [speakerdeck] add a new extractor for speakerdeck.com ([#726](https://github.com/mikf/gallery-dl/issues/726))
+- [twitter] add `replies` option ([#705](https://github.com/mikf/gallery-dl/issues/705))
+- [weibo] add `videos` option
+- [downloader:http] add MIME types for `.psd` files ([#714](https://github.com/mikf/gallery-dl/issues/714))
+### Fixes
+- [artstation] improve embed extraction ([#720](https://github.com/mikf/gallery-dl/issues/720))
+- [deviantart] limit API wait times ([#721](https://github.com/mikf/gallery-dl/issues/721))
+- [newgrounds] fix URLs produced by the `following` extractor ([#684](https://github.com/mikf/gallery-dl/issues/684))
+- [patreon] improve file hash extraction ([#713](https://github.com/mikf/gallery-dl/issues/713))
+- [vsco] fix user gallery extraction
+- fix/improve Cloudflare bypass code ([#728](https://github.com/mikf/gallery-dl/issues/728))
+
+## 1.13.5 - 2020-04-27
+### Additions
+- [500px] recognize `web.500px.com` URLs
+- [aryion] support downloading from folders ([#694](https://github.com/mikf/gallery-dl/issues/694))
+- [furaffinity] add extractor for followed users ([#515](https://github.com/mikf/gallery-dl/issues/515))
+- [hitomi] add extractor for tag searches ([#697](https://github.com/mikf/gallery-dl/issues/697))
+- [instagram] add `post_id` and `num` metadata fields ([#698](https://github.com/mikf/gallery-dl/issues/698))
+- [newgrounds] add extractor for followed users ([#684](https://github.com/mikf/gallery-dl/issues/684))
+- [patreon] recognize URLs with creator IDs ([#711](https://github.com/mikf/gallery-dl/issues/711))
+- [twitter] add `reply` metadata field ([#705](https://github.com/mikf/gallery-dl/issues/705))
+- [xhamster] recognize `xhamster.porncache.net` URLs ([#700](https://github.com/mikf/gallery-dl/issues/700))
+### Fixes
+- [gelbooru] improve post ID extraction in pool listings
+- [hitomi] fix extraction of galleries without tags
+- [jaiminisbox] update metadata decoding procedure ([#702](https://github.com/mikf/gallery-dl/issues/702))
+- [mastodon] fix pagination ([#701](https://github.com/mikf/gallery-dl/issues/701))
+- [mastodon] improve account searches ([#704](https://github.com/mikf/gallery-dl/issues/704))
+- [patreon] fix hash extraction from download URLs ([#693](https://github.com/mikf/gallery-dl/issues/693))
+- improve parameter extraction when solving Cloudflare challenges
+
+## 1.13.4 - 2020-04-12
+### Additions
+- [aryion] add `gallery` and `post` extractors ([#390](https://github.com/mikf/gallery-dl/issues/390), [#673](https://github.com/mikf/gallery-dl/issues/673))
+- [deviantart] detect and handle folders in sta.sh listings ([#659](https://github.com/mikf/gallery-dl/issues/659))
+- [hentainexus] add `circle`, `event`, and `title_conventional` metadata fields ([#661](https://github.com/mikf/gallery-dl/issues/661))
+- [hiperdex] add `artist` extractor ([#606](https://github.com/mikf/gallery-dl/issues/606))
+- [mastodon] add access tokens for `mastodon.social` and `baraag.net` ([#665](https://github.com/mikf/gallery-dl/issues/665))
+### Changes
+- [deviantart] retrieve *all* download URLs through the OAuth API
+- automatically read config files in PyInstaller executable directories ([#682](https://github.com/mikf/gallery-dl/issues/682))
+### Fixes
+- [deviantart] handle "Request blocked" errors ([#655](https://github.com/mikf/gallery-dl/issues/655))
+- [deviantart] improve JPEG quality replacement pattern
+- [hiperdex] fix extraction
+- [mastodon] handle API rate limits ([#665](https://github.com/mikf/gallery-dl/issues/665))
+- [mastodon] update OAuth credentials for pawoo.net ([#665](https://github.com/mikf/gallery-dl/issues/665))
+- [myportfolio] fix extraction of galleries without title
+- [piczel] fix extraction of single images
+- [vsco] fix collection extraction
+- [weibo] accept status URLs with non-numeric IDs ([#664](https://github.com/mikf/gallery-dl/issues/664))
+
 ## 1.13.3 - 2020-03-28
 ### Additions
 - [instagram] Add support for user's saved medias ([#644](https://github.com/mikf/gallery-dl/issues/644))

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -0,0 +1,2 @@
+include README.rst CHANGELOG.md LICENSE
+recursive-include docs *.conf
diff --git a/README.rst b/README.rst
@@ -83,8 +83,8 @@ Download a standalone executable file,
 put it into your `PATH <https://en.wikipedia.org/wiki/PATH_(variable)>`__,
 and run it inside a command prompt (like ``cmd.exe``).
 
-- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.13.3/gallery-dl.exe>`__
-- `Linux   <https://github.com/mikf/gallery-dl/releases/download/v1.13.3/gallery-dl.bin>`__
+- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.13.6/gallery-dl.exe>`__
+- `Linux   <https://github.com/mikf/gallery-dl/releases/download/v1.13.6/gallery-dl.bin>`__
 
 These executables include a Python 3.8 interpreter
 and all required Python packages.
@@ -99,6 +99,14 @@ Linux users that are using a distro that is supported by Snapd_ can install *gal
 
     $ snap install gallery-dl
 
+Chocolatey
+----
+
+Windows users that have Chocolatey_ installed can install *gallery-dl* from the Chocolatey Community Packages repository:
+
+.. code:: powershell
+
+    $ choco install gallery-dl
 
 Usage
 =====
@@ -172,6 +180,11 @@ i.e. ``C:\Users\<username>\``)
 
 Values in later configuration files will override previous ones.
 
+Command line options will override all related settings in the configuration file(s),
+e.g. using ``--write-metadata`` will enable writing metadata using the default values
+for all ``postprocessors.metadata.*`` settings, overriding any specific settings in
+configuration files.
+
 
 Authentication
 ==============
@@ -236,7 +249,7 @@ access to *gallery-dl*. Authorize it and you will be shown one or more
 .. _gallery-dl-example.conf: https://github.com/mikf/gallery-dl/blob/master/docs/gallery-dl-example.conf
 .. _configuration.rst:       https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst
 .. _Supported Sites:         https://github.com/mikf/gallery-dl/blob/master/docs/supportedsites.rst
-.. _stable:                  https://github.com/mikf/gallery-dl/archive/v1.13.3.tar.gz
+.. _stable:                  https://github.com/mikf/gallery-dl/archive/v1.13.6.tar.gz
 .. _dev:                     https://github.com/mikf/gallery-dl/archive/master.tar.gz
 
 .. _Python:     https://www.python.org/downloads/
@@ -248,12 +261,13 @@ access to *gallery-dl*. Authorize it and you will be shown one or more
 .. _pyOpenSSL:  https://pyopenssl.org/
 .. _Snapd:      https://docs.snapcraft.io/installing-snapd
 .. _OAuth:      https://en.wikipedia.org/wiki/OAuth
+.. _Chocolatey:      https://chocolatey.org/install
 
 .. |pypi| image:: https://img.shields.io/pypi/v/gallery-dl.svg
     :target: https://pypi.org/project/gallery-dl/
 
-.. |build| image:: https://travis-ci.org/mikf/gallery-dl.svg?branch=master
-    :target: https://travis-ci.org/mikf/gallery-dl
+.. |build| image:: https://travis-ci.com/mikf/gallery-dl.svg?branch=master
+    :target: https://travis-ci.com/mikf/gallery-dl
 
 .. |gitter| image:: https://badges.gitter.im/gallery-dl/main.svg
     :target: https://gitter.im/gallery-dl/main
diff --git a/docs/configuration.rst b/docs/configuration.rst
@@ -1,6 +1,15 @@
 Configuration
 #############
 
+| Configuration files for *gallery-dl* use a JSON-based file format.
+| For a (more or less) complete example with options set to their default values,
+  see `gallery-dl.conf <gallery-dl.conf>`__.
+| For a configuration file example with more involved settings and options,
+  see `gallery-dl-example.conf <gallery-dl-example.conf>`__.
+
+This file lists all available configuration options and their descriptions.
+
+
 Contents
 ========
 
@@ -122,11 +131,14 @@ Description Use an extractor's current target directory as
 extractor.*.path-restrict
 -------------------------
 =========== =====
-Type        ``string``
+Type        ``string`` or ``object``
 Default     ``"auto"``
-Example     ``"/!? (){}"``
-Description Set of characters to replace with underscores (``_``)
-            in generated path segment names.
+Example     | ``"/!? (){}"``
+            | ``{" ": "_", "/": "-", "|": "-", ":": "-", "*": "+"}``
+Description | A string of characters to be replaced with the value of
+              `path-replace <extractor.*.path-replace>`__
+            | or an object mapping invalid/unwanted characters to their replacements
+            | for generated path segment names.
 
             Special values:
 
@@ -135,19 +147,29 @@ Description Set of characters to replace with underscores (``_``)
             * ``"unix"``: ``"/"``
             * ``"windows"``: ``"\\\\|/<>:\"?*"``
 
-            Note: In a set with 2 or more characters, ``[]^-\`` need to be
+            Note: In a string with 2 or more characters, ``[]^-\`` need to be
             escaped with backslashes, e.g. ``"\\[\\]"``
 =========== =====
 
 
+extractor.*.path-replace
+------------------------
+=========== =====
+Type        ``string``
+Default     ``"_"``
+Description The replacement character(s) for
+            `path-restrict <extractor.*.path-restrict>`__
+=========== =====
+
+
 extractor.*.path-remove
 -----------------------
 =========== =====
 Type        ``string``
 Default     ``"\u0000-\u001f\u007f"`` (ASCII control characters)
 Description Set of characters to remove from generated path names.
 
-            Note: In a set with 2 or more characters, ``[]^-\`` need to be
+            Note: In a string with 2 or more characters, ``[]^-\`` need to be
             escaped with backslashes, e.g. ``"\\[\\]"``
 =========== =====
 
@@ -541,7 +563,7 @@ extractor.danbooru.ugoira
 -------------------------
 =========== =====
 Type        ``bool``
-Default     ``true``
+Default     ``false``
 Description Controls the download target for Ugoira posts.
 
             * ``true``: Original ZIP archives
@@ -888,6 +910,16 @@ Description Controls how a user is directed to an OAuth authorization site.
 =========== =====
 
 
+extractor.oauth.cache
+---------------------
+=========== =====
+Type        ``bool``
+Default     ``true``
+Description Store tokens received during OAuth authorizations
+            in `cache <cache.file_>`__.
+=========== =====
+
+
 extractor.oauth.port
 --------------------
 =========== =====
@@ -1169,12 +1201,21 @@ Description Extract tweet text as ``content`` metadata.
 =========== =====
 
 
+extractor.twitter.replies
+-------------------------
+=========== =====
+Type        ``bool``
+Default     ``true``
+Description Extract media from replies to other Tweets.
+=========== =====
+
+
 extractor.twitter.retweets
 --------------------------
 =========== =====
 Type        ``bool``
 Default     ``true``
-Description Extract images from retweets.
+Description Extract media from Retweets.
 =========== =====
 
 
@@ -1223,6 +1264,24 @@ Description Your  `API Key <https://wallhaven.cc/settings/account>`__ to use
 =========== =====
 
 
+extractor.weibo.retweets
+------------------------
+=========== =====
+Type        ``bool``
+Default     ``true``
+Description Extract media from retweeted posts.
+=========== =====
+
+
+extractor.weibo.videos
+----------------------
+=========== =====
+Type        ``bool``
+Default     ``true``
+Description Download video files.
+=========== =====
+
+
 extractor.[booru].tags
 ----------------------
 =========== =====
@@ -1366,7 +1425,7 @@ downloader.ytdl.forward-cookies
 -------------------------------
 =========== =====
 Type        ``bool``
-Default     ``true``
+Default     ``false``
 Description Forward cookies to youtube-dl.
 =========== =====
 

diff --git a/docs/gallery-dl-example.conf b/docs/gallery-dl-example.conf
@@ -48,7 +48,9 @@
             "cookies":
             {
                 "ipb_member_id": "12345",
-                "ipb_pass_hash": "1234567890abcdef"
+                "ipb_pass_hash": "1234567890abcdef",
+                "igneous"      : "123456789",
+                "hath_perks"   : "m1.m2.m3.a-123456789a"
             },
 
             "proxy":
@@ -74,6 +76,19 @@
             }]
         },
 
+        "deviantart":
+        {
+            "include": "gallery,scraps",
+            "metadata": true,
+            "postprocessors": [{
+                "name": "metadata",
+                "mode": "custom",
+                "directory"       : "Descriptions",
+                "content-format"  : "{description}\n",
+                "extension-format": "descr.txt"
+            }]
+        },
+
         "flickr":
         {
             "access-token": "1234567890-abcdef",
@@ -134,6 +149,19 @@
         "foolfuuka": {
             "fireden-onion": {"root": "http://ydt6jy2ng3s3xg2e.onion"},
             "scalearchive":  {"root": "https://archive.scaled.team"  }
+        },
+
+        "replace invalid path characters with unicode alternatives": null,
+        "path-restrict": {
+            "\\": "⧹",
+            "/" : "⧸",
+            "|" : "￨",
+            ":" : "꞉",
+            "*" : "∗",
+            "?" : "？",
+            "\"": "″",
+            "<" : "﹤",
+            ">" : "﹥"
         }
     },
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		include README.rst CHANGELOG.md LICENSE
		recursive-include docs *.conf