Status reported by bot doesn't match status of last Jenkins run

[mmarchini]

I noticed this on several pull requests. Seems like the bot doesn't always update all Check statuses:

[richardlau]

@mmarchini That's a build issue. The containered jobs do some mangling of the name of the job they pass through to the bot and only report the individual axis on a failure. When the job is rerun a success is posted for the job as a whole but not the individual axes so the failure isn't overwritten.

[mmarchini]

Should we report only the result of node-test-pull-request instead? Would be more useful than having a red ❌ on almost every PR.

[richardlau]

Should we report only the result of node-test-pull-request instead? Would be more useful than having a red ❌ on almost every PR.

Almost every PR? This particular issue is particular to the linux-containered job and only if one of the axes failed.

[richardlau]

Anyway AFAIK this is a job config issue, not a bot issue so this should be moved to nodejs/build.

[mmarchini]

I've seen this on several PRs on the past few days. Maybe I was just unlucky and looked at several PRs where linux-containered failed first. I'll pay more attention to see if this happens on other jobs.

Anyway AFAIK this is a job config issue, not a bot issue so this should be moved to nodejs/build.

I'm fine with that (I don't have permission to do it though).

[rvagg]

This might be to do with my change of jenkins-workspace machines. I know I put the first one into the GitHub bot config but don't recall doing the second (might have, my memory is fuzzy). If someone has access to that machine and wants to check that both new 18.04 workspace machines' IPs are in there that would be helpful. Otherwise I'll check tomorrow when in front of my computer.

[rvagg]

Did someone touch the github-bot server? the config file is right, in sync with what we have here in roles/github-bot/templates/environment-file.j2 which has the 2 IP addresses that should be currently pinging it and it says the github-bot service has been running since 7 hours ago. I've done another reload just in case, maybe the problem will be fixed?

[mmarchini]

@rvagg I don't think the problem is related to the recent changes on jenkins-workspace machines. These changes were last week, right? I've been seeing it for a while.

Almost every PR? This particular issue is particular to the linux-containered job and only if one of the axes failed.

@richardlau you're right, I'm only seeing it on linux-containered, but it's frequent enough to be annoying. I just landed three PRs, two of which this happened. The combo of flaky tests not marked as flaky with the misconfiguration you mentioned is probably the reason I'm seeing this often. I think I'm also a bit unlucky, looking at the other PRs ready to land with ❌ on GitHub, most of them don't have this issue 😄.

Would be happy to help fix this, but I'd need some pointers (I also don't think I have enough access to see how things are set up).

[mmarchini]

probably not actionable (similar to #2224)