spcrawler: a command-line tool to backup Sharepoint public installation data from open API endpoint

spcrawler is a command-line tool to backup Sharepoint public installation data from open API endpoint It uses Sharepoint API located at "/_api/web" and dumps all data and resources.

Contents

• 1   Main features
• 2   Installation
  • 2.1   Any OS
  • 2.2   Python version
• 3   Usage
• 4   Commands
  • 4.1   Ping command
  • 4.2   Walk command
  • 4.3   Dump command

1   Main features

• Metadata extraction
• Download all files (resources) from Sharepoint installation

2   Installation

2.1   Any OS

A universal installation method (that works on Windows, Mac OS X, Linux, …, and always provides the latest version) is to use pip:

# Make sure we have an up-to-date version of pip and setuptools:
$ pip install --upgrade pip setuptools

$ pip install --upgrade spcrawler

(If pip installation fails for some reason, you can try easy_install spcrawler as a fallback.)

2.2   Python version

Python version 3.6 or greater is required.

3   Usage

Synopsis:

$ spcrawler [command] [flags]

See also python -m spcrawler and spcrawler [command] --help for help for each command.

4   Commands

4.1   Ping command

Pings API endpoint located at url + "/_api/web" and returns OK if it's available.

Ping asutk.ru API endpoint

$ spcrawler ping --url https://asutk.ru

4.2   Walk command

Lists objects in Sharepoint installation

Walks over FA.ru website objects

$ spcrawler walk --url http://fa.ru

4.3   Dump command

Dumps all objects/lists/data from API to JSON lines files. Stores all data to local path "domainname/data"

Dumps data from FA.ru (Financial university) website

$ spcrawler dump --url http://fa.ru