Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add filter function as an option to linkinator.check() #120

Merged
merged 6 commits into from
Nov 20, 2019
Merged

feat: add filter function as an option to linkinator.check() #120

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
99c99a1
feat: add filterLinks function
zeke Nov 19, 2019
483de84
chore: lint
zeke Nov 19, 2019
4c9543d
chore: rename the filtering function and flip its behavior
zeke Nov 19, 2019
9f1adcc
feat: allow linksToSkip option to be a filter function
zeke Nov 20, 2019
f3bb7ef
chore: lint
zeke Nov 20, 2019
a146f2c
chore: make linksToSkip an async function
zeke Nov 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ Asynchronous method that runs a site wide scan. Options come in the form of an o
- `port` (number) - When the `path` is provided as a local path on disk, the `port` on which to start the temporary web server. Defaults to a random high range order port.
- `recurse` (boolean) - By default, all scans are shallow. Only the top level links on the requested page will be scanned. By setting `recurse` to `true`, the crawler will follow all links on the page, and continue scanning links **on the same domain** for as long as it can go. Results are cached, so no worries about loops.
- `linksToSkip` (array) - An array of regular expression strings that should be skipped during the scan.
- `filter` (function) - A function that's called for each link with the link URL as its only argument. Return `true` to check the link or `false` to skip it.
zeke marked this conversation as resolved.
Show resolved Hide resolved

#### linkinator.LinkChecker()
Constructor method that can be used to create a new `LinkChecker` instance. This is particularly useful if you want to receive events as the crawler crawls. Exposes the following events:
Expand Down
6 changes: 5 additions & 1 deletion src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export interface CheckOptions {
path: string;
recurse?: boolean;
linksToSkip?: string[];
filter?: (arg0: string) => boolean;
}

export enum LinkState {
Expand Down Expand Up @@ -144,7 +145,10 @@ export class LinkChecker extends EventEmitter {
})
.filter(match => !!match);

if (skips.length > 0) {
if (
skips.length > 0 ||
(opts.checkOptions.filter && !opts.checkOptions.filter(opts.url.href))
) {
const result: LinkResult = {
url: opts.url.href,
state: LinkState.SKIPPED,
Expand Down
7 changes: 7 additions & 0 deletions test/fixtures/filter/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<html>
<body>
<a href="https://good.com">I'm good</a>
<a href="http://www.filterme.com">I should be filtered</a>
<a href="https://example.com/filtermetoo">I should also be filtered</a>
</body>
</html>
18 changes: 18 additions & 0 deletions test/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,24 @@ describe('linkinator', () => {
);
});

it('should skip links if passed a filterLink function', async () => {
const scope = nock('https://good.com')
.head('/')
.reply(200);
const results = await check({
path: 'test/fixtures/filter',
filter: href => {
return !href.includes('filterme');
},
});
assert.ok(results.passed);
assert.strictEqual(
results.links.filter(x => x.state === LinkState.SKIPPED).length,
2
);
scope.done();
});

it('should report broken links', async () => {
const scope = nock('http://fake.local')
.head('/')
Expand Down