Catch errors setting SO_REUSEPORT

[tilgovi]

Fix #1480

[tilgovi]

The py36 coverage seems totally spurious.

[aconrad]

Would it be possible that the tests are not guaranteed to run in a deterministic way where the code path for this branching logic could be different for some test runs? That's usually what the issue is. I've seen this same exact coverage report in other PRs of gunicorn.

[aconrad]

@tilgovi Just so you know, I put a breakpoint just before raise LimitRequestLine(len(data), limit) and ran tox -e py35 in a loop and indeed, sometimes the breakpoint hits, and sometimes it doesn't (most of the time in my experience). My guess is that a dictionary is at play somewhere, the ordering of the keys is less deterministic in Python 3 than it is in Python 2 (but it might be something else too).

[aconrad]

Here's what I actually ran (so I don't have to wait so long for all the tests to run):

while true; do tox -e py35 -- tests/test_invalid_requests.py; done

[tilgovi]

I think it's more deliberate than that. Search random in https://github.com/benoitc/gunicorn/blob/master/tests/treq.py

[aconrad]

Hah, there you go. I think it would be a good opportunity to fix the tests so they are deterministic (and also write additional tests to cover both sides of the coin)

[berkerpeksag]

Or a simpler version:

if err[0] not in (errno.ENOPROTOOPT, errno.EINVAL):
    raise

[tilgovi]

Updated.

[benoitc]

i'm wondering if instead of raising an error, we couldn't simply try to start without this option and log a warning. Thoughts?

[tilgovi]

Looking at the errors that setsockopt(2) can return, I think anything else is probably very bad.

[benoitc]

fair enough :)

[abitduck]

It's OK with this patch~

[hartym]

Fixes #1480 on my system.

[benoitc]

@aconrad what do you mean by more deterministic?

[benoitc]

let's wait if we have some feedback but 👍 for the merge anyway

[aconrad]

@benoitc Couverture.io showed us that the code path execution of each test run may be different each time (not guaranteed, non-deterministic), likely due to the random module used in tests https://github.com/benoitc/gunicorn/blob/master/tests/treq.py, as @tilgovi suggested.

As we can see from the coverage report, line 215 or line 218 of file https://github.com/benoitc/gunicorn/blob/master/gunicorn/http/message.py#L215-L218 may be executed during test runs (unpredictable):

https://app.couverture.io/coverage/github/benoitc/gunicorn/9e076f4246c80f2091163242e7c985ad6ccb78c7/py36

So if the culprit is indeed the random module, we should set a proper seed value so we can guarantee repeatable outcomes of the tests. We might want to set a PYTHONHASHSEED env variable which tox describes here (even though it's a Python feature, not a tox one): https://tox.readthedocs.io/en/latest/example/basic.html#special-handling-of-pythonhashseed

Once fixed (e.g. line 215 is guaranteed to be the code path taken) then we should write additional tests to make sure that line 218 is also covered eventually.

I hope this helps.

[hartym]

Fixes #1480 on my system.