Skip to content

Commit

Permalink
feat: Add support for running the script with npx
Browse files Browse the repository at this point in the history
  • Loading branch information
jtiala committed May 29, 2023
1 parent 3ecdcf4 commit da1b185
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 13 deletions.
16 changes: 6 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,30 +12,26 @@ Scrape pages, posts, images and other data from a WordPress instance using the W

## Pre-requisites

Node.js v19 or newer (for native fetch support)

## Installation

```bash
npm i
```
Node.js v19 or newer (for native fetch support).

## Usage examples

Recommended way of using wpdl is with `npx`. You can also clone this repo and run locally, just replace `npx wpdl` with `npm run scrape -- `. Note: the double dash is needed to pass the arguments to the script.

Scrape pages and posts

```bash
npm run scrape -- --url https://your-wp-instance.com --pages --posts
npx wpdl --url https://your-wp-instance.com --pages --posts
```

Scrape pages and clean up the html by filtering out all `img` elements and elements with the class `foo`. Also remove all elements without text content. From the json files, remove all the Jetpack and Yoast SEO data.

```bash
npm run scrape -- --url https://your-wp-instance.com --pages --elementFilter img --classFilter foo --jsonFilter "jetpack_*" --jsonFilter "yoast_*" --removeEmptyElements
npx wpdl -- --url https://your-wp-instance.com --pages --elementFilter img --classFilter foo --jsonFilter "jetpack_*" --jsonFilter "yoast_*" --removeEmptyElements
```

To see full usage, run

```bash
npm run help
npx wpdl -h
```
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"license": "MIT",
"main": "wpdl.js",
"type": "module",
"bin": "./src/wpdl.js",
"scripts": {
"scrape": "./src/wpdl.js",
"help": "./src/wpdl.js --help",
Expand Down
4 changes: 1 addition & 3 deletions src/wpdl.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,7 @@ import { error, info } from "./utils/log.js";
import { getSiteNameFromUrl, isValidUrl } from "./utils/url.js";

const argv = yargs(hideBin(process.argv))
.usage(
"Usage: npm run scrape -- --url https://your-wp-instance.com [options]"
)
.usage("Usage: npx wpdl --url https://your-wp-instance.com [options]")
.option("url", {
alias: "u",
type: "string",
Expand Down

0 comments on commit da1b185

Please sign in to comment.