test_end_to_end.py:test_issue_1231c fails on Python 3.6

[mnaberez]

https://travis-ci.org/Supervisor/supervisor/jobs/622789487

This test seems to consistently fail on Travis now but it was working when #1235 was merged. I restarted the build a few times. I only observed the failure on 3.6 and only on that line.

Related #1235
cc @vsajip

[vsajip]

Really weird. I stepped into pexpect to see what supervisord was sending. I haven't yet found out why, but the supervisord process is terminating prematurely after writing this out:

Error: Another program is already listening on a port that one of our HTTP servers is configured to use. Shut this program down first before starting supervisord.
For help, use /home/vinay/projects/supervisor/supervisor/supervisord.py -h

It's not immediately clear what's going on. With Python 3.7, at the same point, we see

Unlinking stale socket /tmp/issue-1231.sock
2020-02-10 20:43:51,341 INFO RPC interface 'supervisor' initialized
2020-02-10 20:43:51,341 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2020-02-10 20:43:51,343 INFO supervisord started with pid 37469
2020-02-10 20:43:52,351 INFO spawned: 'hello' with pid 37472
2020-02-10 20:43:54,323 INFO success: hello entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

[vsajip]

I've done a git bisect, but that's inconclusive - it just identifies the bad commit as the one where I added test_issue_1231c, on 28 May 2019. So if it was working with Python 3.6 at that time, it doesn't any more. But the Python version I'm using predates that (22 Oct 2018), so I would guess that it's something else in the environment that's problematic (given that 3.7, etc. aren't failing).

[vsajip]

OK, I've attempted a fix in #1329.

[mnaberez]

@vsajip Thanks for looking into this! I didn't run it locally yet but I looked at your patch #1329 and I agree with the changes, especially ensuring process shutdown in each test.

I would guess that it's something else in the environment that's problematic (given that 3.7, etc. aren't failing).

After thinking about this comment, I dug around and found that a new version of pexpect (4.8.0) was released just recently (dates, changelog). I pinned to pexpect==4.7.0 and CI passes on 3.6 now. I restarted it several times and it passed every time.

[vsajip]

Oh, it passes on 3.6 without the #1329 changes? Good. But it's probably wise to implement those changes anyway, at some point 😄

[mnaberez]

Oh, it passes on 3.6 without the #1329 changes?

Yes.

But it's probably wise to implement those changes anyway, at some point 😄

Agreed.

[mnaberez]

Closing this issue since it's not a bug in Supervisor and CI passes again since fd12a98 (build). We can continue improving the test suite over on #1329.