fix #59, add information crawler

[DerekNonGeneric]

Still need to determine where to reliably locate code samples and descriptions, but this is nearly done. 🎉

I happened to run across this old gh-file-importer.mjs module I wrote a few months ago (still needs revision, but works) and immediately thought about #199. It's still early in the semester so I'm not buried in assignments yet, so sorry for stacking the PRs, but this opportunity doesn't come too often.

Since I can't reliably locate code samples, I'm currently just parsing the Markdown and using the last JavaScript code fence that appears on the page. There does not seem to be a designated place to look for proposal descriptions, so I'm just using lorem ipsum until we can find a better way to do this.

I must say, I severely underestimated this task and am glad that I went for it when I did. Run it via npm run sync:proposal-data and you will see it in action. Once it runs once, the _data/stage3.yml file gets updated with the latest info, which can be observed locally once the site is built using this data. I know we still need to integrate it into the CI as suggested in #30, but early feedback on this PR would be amazing. I'm especially concerned about the logging aspect of this. I left several debug log statements commented out to assist with review, but am still unsure what logging should be occurring while the task runs to completion.

Please keep me posted with suggestions!

/cc @codehag @ljharb

[ljharb]

General comment/question about all the .catch-without-rethrows all over the place?

[DerekNonGeneric]

I was thinking that using the description of the GitHub repo would be acceptable, but that doesn't seem to be the currently followed practice. It's possible that these one-line descriptions are all located in the same place for each proposal and I haven't noticed where to look yet. To illustrate my point, take proposal-static-class-features for example.

Current proposal description in stage3.yml:

This proposal adds Static public fields, Static private methods and Static private fields

Current proposal description in GitHub repo UI:

The static parts of new class features, in a separate proposal

---

Currently, parsing each proposal's README for a one-line description would be complicated since these descriptions seem to be derived from non-standard locations in the document, so that seems like a no-go.

[codehag]

@DerekNonGeneric Wow fantastic work. This was a great surprise to see this morning. I haven't had a chance to dig deeply into this yet, I will try to do that later today or tomorrow. I just wanted to say early -- Thank you so much for the work you are doing. It is very much appreciated.

Review incoming.

[DerekNonGeneric]

Hi @codehag, glad to hear you're pleased with the direction this is headed in!

I haven't had a chance to dig deeply into this yet, I will try to do that later today or tomorrow.

I've been able to clean it up a bit since the initial PR, so you probably dodged having to review even worse code. :)

Looking forward to initial thoughts and suggestions, I know there's still a bit more left until this would be considered complete.

[codehag]

Sorry I haven't had a chance to look at this yet. So no worries -- keep working as you are.

The mozilla work flow won't apply here. We can treat this as a regular repository.

[smorimoto]

This seems to be inactive for quite some time. Could you open a new PR as needed?