From 315118499892c021ee27b4b6ea9c6f43dc3739c0 Mon Sep 17 00:00:00 2001 From: Nekmo Date: Fri, 31 Aug 2018 20:56:41 +0200 Subject: [PATCH] Issue #48: --delay option --- README.rst | 3 ++- docs/usage.rst | 25 ++++++++++++++++++++++--- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/README.rst b/README.rst index feb735f..d6a7153 100644 --- a/README.rst +++ b/README.rst @@ -69,7 +69,8 @@ Features * 404 error pages and detect **fake 404 errors**. * Filter results by **flags**. * Analyze results at end. -* Get new directories using **robots.txt** (NEW!) +* Get new directories using **robots.txt** +* **Delay** between requests (NEW!) Install diff --git a/docs/usage.rst b/docs/usage.rst index caf9d19..40679c5 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -35,6 +35,10 @@ Example:: --not-follow-subdomains The subdomains will be ignored --exclude-sources TEXT Exclude source engines. Possible options: robots, virustotal + -p, --proxies TEXT Set one or more proxies to alternate between + them + -d, --delay FLOAT Delay between requests to avoid bans by the + server --not-allow-redirects Redirectors will not be followed --version --help Show this message and exit. @@ -152,16 +156,31 @@ Other flags: Threads ------- -Dirhunt makes multiple simultaneous requests using threads. By default the number of threads is ``cpu count * 5``. -You can change the threads count using ``--threads `` (``-t ``). For example:: +Dirhunt makes multiple simultaneous requests using threads by default. The default number of threads is +``cpu count * 5``. If you use the ``--delay`` option, the *simultaneous requests mode* is disabled and the number of +threads by default is ``number of proxies``. If you do not use proxies in ``--delay`` mode, the default threads +number is 1. You can change the threads count using ``--threads `` (``-t ``). Usage:: - $ dirhunt --threads + $ dirhunt --threads For example:: $ dirhunt http://domain1/blog/ --threads 10 +Delay +----- +This mode deactivates *simultaneous requests mode* and it activates a waiting time from the end of a request +and the next one begins. By default delay is **disabled**. Use this mode only if the server is restricting requests. +You can improve the performance of this option using multiple proxies. Usage:: + + $ dirhunt --delay + +For example:: + + $ dirhunt http://domain1/blog/ --delay 0.1 + + Timeout ------- By default Dirhunt only waits up to 10 seconds for each url. You can increase or decrease this time using