Remove <url> requirement

[darcyclarke]

❤️ this project and all the hard work that's gone into it! /cc @paulirish et al

I've been writing some custom gatherers/audits recently and wanted to take that work further. Outside of www specific audits, I'd like to use lighthouse to test against local project-specifc files, assets, dependancies, apis, repos etc.. The only issue I find is the requirement of a <url>. I can get passed this by simply providing a local/arbitrary value to the cli but I feel like its an anti-pattern (and I know Paul hates anti-patterns).

Examples of general-purpose gatherers I'd like to write:

• Github (organizations, members, repos, contents etc.)
• Sonarqube (code coverage)
• Contentful (schemas, assets)
• lots more...

Examples of general-purpose audits I'd like to write/run:

• Source code/file analysis (linting, standards, best practices, coverage etc.)
• Maintainability (repository health checks, templates, licensing, versioning etc.)
• Performance (local, staging & prod build metrics)
• lots more...

Why?

I see a ton of value in the way lighthouse is architected (ie. collecting > auditing > scoring) and would prefer not to have 2, 3, 4, 5 different testing/auditing tools to create a dashboard of insights (plus, the lighthouse dashboard is really 🌶).

Along these lines, some discussion (#1806) and work (#3743) have been done in this space around the separation of the audit and gatherer phases (GAAAAr!); I can definitely see a lot more use cases from the community (of the CLI tool) if the underlaying architecture was to be more exposed, talked about & emphasized.

What?

• Make lighthouse <url>-agnostic

How?

• Remove <url> as a requirement and/or
• Create a flag (ex. lighthouse --disable-driver)

Would love to know others thoughts/feelings here.

[darcyclarke]

To top up on these thoughts; it seems like it would be more appropriate to just allow for custom drivers instead of trying to disable the default driver portion of this process all together.

I see now that I'm probably doing the hypothetical "driver" setup/instantiation within the collection step when, in hindsight, it would be better to write a GitDriver or a TestDriver that then provides an API downstream to whatever Gatherers I write that require that functionality. Theoretically I could then use the <url> param as a file path or some kind of other meaningful identifier.

I found a few discussions around custom drivers that didn't really go anywhere (ie. supporting other browsers #2406, device emulation #2063 etc.). Again, I'd love to hear any insight/feedback here.

[patrickhulce]

Thanks for sharing these ideas @darcyclarke! Love the concept of bringing more insights into the Lighthouse umbrella. We're very page load centric right now, so URL is very much apart of our flow, as is the concept of a loaded webpage to gatherers.

It seems like the more appropriate point you might want to hook into is the GatherRunner rather than the driver. The gather runner internally has very strong assumptions about a page load. However, the only contract we expect of the gather runner externally is that is has a single function to accept a config and return an object of all the artifacts. Converting this into a configurable array of runners to use with custom options might enable the type of flexibility you're looking for.

Definitely some interesting ideas to consider here as we plan for the future of the config! 👍

[brendankenny]

In the meantime, GA(AAAr) can also help here. The audit stage just expects an object of artifacts, that's it, so a different program can generate those artifacts and then use -A to use Lighthouse as an audit runner

[paulirish]

Darcy wrote up a nice doc exploring the usecases of using LH as a general purpose (gather->audit->report) tool: https://docs.google.com/document/d/1yauVu9UtCn6K37XFW1TFlp4Rwp6czhU5b8Lczpu6W_Q/edit

We decided that we're not going to take this on and support non-webpage sites as the target of investigation. We do support anyone who wants to fork Lighthouse and explore this, though.

Thanks again for exploring this idea with us.

[paulirish]

Looks like sonar enabled this workflow. Check out "Easily extend sonarwhal with parsers to analyze files such as config files" on https://blogs.windows.com/msedgedev/2018/04/19/sonarwhal-v1-linting-tool-for-web/

[darcyclarke]

@paulirish thanks for the update/tip here Paul! Will check this out for sure.