Buildbot should not only test "make check" but also "make install"

[nodakai]

... or make dist.

It would be nice to avoid troubles like #19678 .

[chris-morgan]

In the last six months:

#15545
#19678
#19970
#20383

As I recall it there was one other one a few months ago that got fixed before it was filed.

Can we please do something about this?

[brson]

I've added to the list.

[retep998]

There have been a huge amount of issues with the -windows-gnu distribution being broken lately. Having some sort of integration testing for whether the built packages actually work would be really useful.

[edunham]

Currently testing make dist on auto builds in dev. I'll deploy to prod as soon as I get it working.

[nodakai]

@edunham Was it deployed?

[Mark-Simulacrum]

I think this may be somewhat done now, as we build artifacts for each PR. However, I'm not sure, so cc @brson @alexcrichton

[alexcrichton]

Yes it'd still be great to test this!

[Mark-Simulacrum]

To update the current status: make dist's equivalent is run across all dist builders, but we do not check make install. I'm going to mark this as P-medium since I imagine distros would appreciate this.

[steveklabnik]

Triage: I thought maybe something changed here, but am not totally sure.

[Mark-Simulacrum]

No nothing changed here, we're still only running x.py dist, not x.py install on our builders. To me at least it's also not super clear how we'd test the installation...

[aidanhs]

We had a chat about this in the infra meeting today and I'm going to close to reflect our current feelings on the issue - in short, though it seems like a reasonable thing to do, we haven't seen an issue for quite a while and make install isn't a key thing we're looking to protect right now.