Home

Steven Selph's Scraper

This scraper downloads images and metadata primarily from online databases like thegamesdb for use with EmulationStation It uses hashes of roms to provide the most accurate results.

Supported Systems:

A complete list is HERE.

Platforms Supported:

Anything you can compile Go on. I provide precompiled binaries for:

• Windows
• Mac
• Linux
• Raspberry Pi

Note that if you are using RetroPie, the scraper can easily be used and installed through the RetroPie Setup Script - you can see the documentation for it HERE

Installing:

You can install compiled binaries from the releases page HERE

you can also compile from source as described in the README

Usage

$ cd <rom directory>
$ scraper

ROMs will be scanned and a gamelist.xml file will be created. All images will be placed inside the images folder.

All Systems

$ scraper -scrape_all -thumb_only

MAME/FBA

The scraper now supports MAME/FBA but using file names instead of hashing. Since it uses a different DB and lookup method, several of the command line flags no longer apply. When the -mame flag is used it disables all other databases and the mamedb only has the one size of image so the flags about thumbnails, gdb, ovgdb, etc don't do anything.

$ scraper -mame

You can choose your preference of image type with the mame_img flag. If you prefer marquees but want to fallback to titles then snapshots you can do:

$ scraper -mame -mame_img "m,t,s"

Command Line Flags

There are several command flags you can pass. To see a full list use -help

$ scraper -help

Full List:

Usage of scraper:
  -add_not_found
        If true, add roms that are not found as an empty gamelist entry.
  -append
        If the gamelist file already exist skip files that are already listed and only append new files.
  -console_img string
        Comma seperated order to prefer images, s=snapshot, b=boxart, f=fanart, a=banner, l=logo, 3b=3D boxart. (default "b")
  -console_src string
        Comma seperated order to prefer console sources, ss=screenscraper, ovgdb=OpenVGDB, gdb=theGamesDB (default "gdb")
  -download_images
        If false, don't download any images, instead see if the expected file is stored locally already. (default true)
  -extra_ext string
        Comma separated list of extensions to also include in the scraper.
  -gdb_img string
        Deprecated, see console_img. This will be removed soon.
  -hash_file file
        The file containing hash information.
  -image_dir directory
        The directory to place downloaded images to locally. (default "images")
  -image_path path
        The path to use for images in gamelist.xml. (default "images")
  -image_suffix suffix
        The suffix added after rom name when creating image files. (default "-image")
  -img_format jpg or png
        jpg or png, the format to write the images. (default "jpg")
  -img_workers N
        Use N worker threads to process images. If 0, then use the same value as workers.
  -lang string
        The order to choose for language if there is more than one for a value. (en, fr, es, de, pt) (default "en")
  -mame
        If true we want to run in MAME mode.
  -mame_img string
        Comma separated order to prefer images, s=snap, t=title, m=marquee, c=cabniet, b=boxart, 3b=3D-boxart, fly=flyer. (default "t,m,s,c")
  -mame_src string
        Comma seperated order to prefer mame sources, ss=screenscraper, adb=arcadeitalia, mamedb=mamedb-mirror, gdb=theGamesDB-neogeo (default "mamedb,gdb")
  -max_height height
        The max height of images. Larger images will be resized.
  -max_width width
        The max width of images. Larger images will be resized. (default 400)
  -missing file
        The file where information about ROMs that weren't scraped is added.
  -nested_img_dir
        Use a nested img directory structure that matches rom structure.
  -no_thumb
        Don't add thumbnails to the gamelist.
  -output_file file
        The XML file to output to. (default "gamelist.xml")
  -overview_len N
        If set it will truncate the overview of roms to N characters + ellipsis.
  -refresh
        Information will be attempted to be downloaded again but won't remove roms that are not scraped.
  -region string
        The order to choose for region if there is more than one for a value. xx is a special region that will choose any region. (default "us,wor,eu,jp,fr,xx")
  -retries N
        Retry a rom N times on an error. (default 2)
  -rom_dir directory
        The directory containing the roms file to process. (default ".")
  -rom_path path
        The path to use for roms in gamelist.xml. (default ".")
  -scrape_all
        If true, scrape all systems listed in es_systems.cfg. All dir/path flags will be ignored.
  -skip_check
        Skip the check if thegamesdb.net is up.
  -ss_password password
        The password for registered ScreenScraper users.
  -ss_user username
        The username for registered ScreenScraper users.
  -start_pprof
        If true, start the pprof service used to profile the application.
  -strip_unicode
        If true, remove all non-ascii characters.
  -thumb_only
        Download the thumbnail for both the image and thumb (faster).
  -thumb_suffix suffix
        The suffix added after rom name when creating thumb files. (default "-thumb")
  -update_cache
        If false, don't check for updates on locally cached files. (default true)
  -use_filename
        If true, use the filename minus the extension as the game title in xml.
  -use_gdb
        DEPRECATED, see -console_src : use the theGamesDB as a datasource. (default true)
  -use_nointro_name
        Use the name in the No-Intro DB instead of the one in the GDB. (default true)
  -use_ovgdb
        DEPRECATED, see -console_src : use the OpenVGDB as a datasource.
  -use_ss
        DEPRECATED, see -console_src : use the ScreenScraper.fr as a datasource.
  -version
        Print the release version and exit.
  -workers N
        Use N worker threads to process roms. (default 1)