Skip to content

How to debug errors

Jump to bottom
Thomas D edited this page Nov 11, 2020 · 8 revisions

Bugs encountered by running webscreenshot can come from two sources:

  • A) Bugs in the webscreenshot script itself, which as a reminder, is only a wrapper around different renderers (phantomjs, chrome, chromium, edge chromium, firefox etc.), meaning that webscreenshot executes raw commands around renderers
  • B) Bugs in the renderer itself

Here below are explained some steps and advices in order to identify the root cause of a bug, and get your screenshot.

1. Put webscreenshot in DEBUG verbose mode with the -vv option to see more context, and to see raw commands

$ webscreenshot -vv google.fr
webscreenshot.py version 2.94

[DEBUG][General] Options: Namespace(URL='google.fr', ajax_max_timeouts='1400,1800', cookie=None, crop=None, custom_js=None, format='png', header=None, http_password=None, http_username=None, imagemagick_binary=None, input_file=None, label=False, label_bg_color='NavajoWhite', label_size=60, log_level='DEBUG', multiprotocol=False, no_error_file=False, no_xserver=False, output_directory=u'/tmp/screenshots', port=None, proxy=None, proxy_auth=None, proxy_type=None, quality=75, renderer='phantomjs', renderer_binary=None, single_output_file=None, ssl=False, timeout=30, verbosity=2, window_size='1200,800', workers=4)

[INFO][General] 'google.fr' has been formatted as 'http://google.fr:80' with supplied overriding options
[+] 1 URLs to be screenshot
[DEBUG][http://google.fr:80] Shell command to be executed
'phantomjs --ignore-ssl-errors=true --ssl-protocol=any --ssl-ciphers=ALL "/root/venvs/venvwebscreenshot3kali/local/lib/python2.7/site-packages/webscreenshot/webscreenshot.js" url_capture=http://google.fr:80 output_file="/tmp/screenshots/http_google.fr_80.png" width=1200 height=800 format=png quality=75 ajaxtimeout=1400 maxtimeout=1800'

[DEBUG][http://google.fr:80] Shell command PID 5986 ended normally
[INFO][http://google.fr:80] Screenshot OK

[+] 1 actual URLs screenshot
[+] 0 error(s)

2. Execute a raw command with a debug flag

For instance, the phantomjs renderer accepts a --debug=true option to have a verbose output.

$ phantomjs --debug=true --ignore-ssl-errors=true --ssl-protocol=any --ssl-ciphers=ALL "/root/venvs/venvwebscreenshot3kali/local/lib/python2.7/site-packages/webscreenshot/webscreenshot.js" url_capture=http://google.fr:80 output_file="/tmp/screenshots/http_google.fr_80.png" width=1200 height=800 format=png quality=75 ajaxtimeout=1400 maxtimeout=1800

2020-11-11T17:47:19 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=<filename>' to enable persistent cookie storage)
2020-11-11T17:47:19 [DEBUG] Set  "http"  proxy to:  "" : 1080
2020-11-11T17:47:19 [DEBUG] Phantom - execute: Configuration
2020-11-11T17:47:19 [DEBUG]      0 objectName : ""
2020-11-11T17:47:19 [DEBUG]      1 cookiesFile : ""
2020-11-11T17:47:19 [DEBUG]      2 diskCacheEnabled : "false"
2020-11-11T17:47:19 [DEBUG]      3 maxDiskCacheSize : "-1"
2020-11-11T17:47:19 [DEBUG]      4 diskCachePath : ""
...
2020-11-11T17:47:19 [DEBUG] Phantom - execute: Script & Arguments
2020-11-11T17:47:19 [DEBUG]      script: "/root/venvs/venvwebscreenshot3kali/local/lib/python2.7/site-packages/webscreenshot/webscreenshot.js"
2020-11-11T17:47:19 [DEBUG]      0 arg: "url_capture=http://google.fr:80"
2020-11-11T17:47:19 [DEBUG]      1 arg: "output_file=/tmp/screenshots/http_google.fr_80.png"
2020-11-11T17:47:19 [DEBUG]      2 arg: "width=1200"
2020-11-11T17:47:19 [DEBUG]      3 arg: "height=800"
2020-11-11T17:47:19 [DEBUG]      4 arg: "format=png"
2020-11-11T17:47:19 [DEBUG]      5 arg: "quality=75"
2020-11-11T17:47:19 [DEBUG]      6 arg: "ajaxtimeout=1400"
2020-11-11T17:47:19 [DEBUG]      7 arg: "maxtimeout=1800"
2020-11-11T17:47:19 [DEBUG] Phantom - execute: Starting normal mode
2020-11-11T17:47:19 [DEBUG] WebPage - setupFrame ""
2020-11-11T17:47:19 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r")))
2020-11-11T17:47:19 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r")))
2020-11-11T17:47:19 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r")))
2020-11-11T17:47:19 [DEBUG] WebpageCallbacks - getJsConfirmCallback
2020-11-11T17:47:19 [DEBUG] WebpageCallbacks - getJsConfirmCallback
2020-11-11T17:47:19 [DEBUG] CookieJar - Saved "NID=204=gMk8Wbwgx-E0V
...

Similarly, chrome and chromium renderers can be put in verbose mode with these options --enable-logging=stderr --v=1

$ google-chrome --enable-logging=stderr --v=1 --allow-running-insecure-content --ignore-certificate-errors --ignore-urlfetcher-cert-requests --reduce-security-for-testing --no-sandbox --headless --disable-gpu --hide-scrollbars --incognito -screenshot="/tmp/screenshots/http_google.fr_80.png" --window-size=1200,800 "http://google.fr:80"

[1111/175025.343329:INFO:cpu_info.cc(53)] Available number of cores: 4
[1111/175025.343381:INFO:cpu_info.cc(53)] Available number of cores: 4
[1111/175025.343544:VERBOSE1:zygote_main_linux.cc(218)] ZygoteMain: initializing 0 fork delegates
[1111/175025.343553:VERBOSE1:zygote_main_linux.cc(218)] ZygoteMain: initializing 0 fork delegates
[1111/175025.356417:VERBOSE1:webrtc_internals.cc(118)] Could not get the download directory.
[1111/175025.370477:VERBOSE1:breakpad_linux.cc(2070)] Non Browser crash dumping enabled for: gpu-process
[1111/175025.381743:VERBOSE1:breakpad_linux.cc(2070)] Non Browser crash dumping enabled for: renderer
[1111/175025.411370:ERROR:paint_controller.cc(634)] PaintController::FinishCycle() completed
[1111/175025.444807:VERBOSE1:configured_proxy_resolution_service.cc(845)] PAC support disabled because there is no system implementation
[1111/175025.445869:VERBOSE1:configured_proxy_resolution_service.cc(845)] PAC support disabled because there is no system implementation
[1111/175025.447923:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: http://google.fr/
[1111/175025.526322:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: http://www.google.fr/
...

3. Choose another renderer

If one renderer shows some bugs, try another one by using webscreenshot --renderer-binary and -r options.

Clone this wiki locally