Pass through validator options in oGS options

[lukeocodes]

Hey, I'm just trying to understand why http://localhost:3000 is invalid but 127.0.0.1 is valid? Is it the require_tld that it fails on in validator?

openGraphScraper/tests/unit/utils.spec.js

Line 79 in 3fb4fe1

'http://localhost:3000/',

Before I opened this I found a mention in validator about exactly this, you can disable require_tld to get localhost to validate.

validatorjs/validator.js#675

Is there space here to allow for passing in validator options through the oGS options? i.e.

    const options = { 
      url: 'http://localhost:3000',
      validator: {
        require_tld: false
      }
    }

    ogs(options)
      .then((data) => {
        const { error, result } = data
        if (error) console.log('error:', error)
        console.log('result:', result)
      })
      .catch((error) => {
        consola.error(error)
      })

WDYT? I can give a PR a go to add this.

[jshemas]

Hello! This is a great idea! I added it in open-graph-scraper@4.7.0.

Example:

const ogs = require('open-graph-scraper');

const options = {
  url: 'http://ogp.me/',
  urlValidatorSettings: {
    protocols: ['http', 'https'],
    require_tld: true,
    require_protocol: false,
    require_host: true,
    require_valid_protocol: true,
    allow_underscores: false,
    host_whitelist: false,
    host_blacklist: false,
    allow_trailing_dot: false,
    allow_protocol_relative_urls: false,
    disallow_auth: false,
  },
};

ogs(options)
  .then((data) => {
    const { error, result, response } = data;
    console.log('error:', error); // This is returns true or false. True if there was a error. The error it self is inside the results object.
    console.log('result:', result); // This contains all of the Open Graph results
    console.log('response:', response); // This contains the HTML of page
  });