DoIP improvements all-around

[ferdinandjarisch]

This PR contains multiple fixes that:

• Improve logging output of intermediate results and summaries
• Streamline how connection errors are treating in RoutingActivationType/SourceAddress enumeration
• Avoid stalling enumerations due to dangling socket connections when enumerating SourceAddresses

[rumpelsepp]

What' the rational of b8c3ce0? This code seems unfortunate to me. Why is writing random data to a connection needed to close it?

[ferdinandjarisch]

What' the rational of b8c3ce0? This code seems unfortunate to me. Why is writing random data to a connection needed to close it?

As mentioned in the log message and according to what you wrote yourself, the rationale is to "to help our kernel realize the connection is closed to avoid TIME_WAIT".

Apparently this is due to implementation details of asyncio's open_connection, having a stream reader/writer on both ends that do not directly communicate, iirc.
Writing EOF alone is not sufficient.

In any case, without this change the DoIP discoverer occupies up to 13.000 Ports while enumerating, as they are not properly closed.

[ferdinandjarisch]

With recent changes this PR now fully surpasses #506

Additional content includes:

• Fixes treatment of DiagnosticMessageACKs to match requirements from Standard
• Exposes DoIP protocol version to target string
• Auto-detects protocol_version during discovery
• Queries additional content from DoIP entity via DoIPEntityStatusRequests

[ferdinandjarisch]

And even more feats:

• --tcp-connect-delay flag to slow down enumerations, which can be useful for not-so-powerful DoIP entities
• Dynamic scanning of RoutingActivationType and SourceAddress, depending on what exactly the DoIP entity prefers

Apart from that some minor fixups for previous commits; so once done this needs to be rebased

[rumpelsepp]

For the record, discussed via matrix. ACK, but these points are open:

1. Use asyncio for special UDP socket; supported since Python 3.11: https://docs.python.org/3/library/asyncio-eventloop.html#working-with-socket-objects-directly

socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setblocking(False)
sock.bind(("0.0.0.0", 0))
loop = asyncio.get_running_loop()
data, addr = await loop.sock_recvfrom(sock, …)
await loop.sock_sendto(sock, …)

2. Due to a future refactor, avoid defining new class variables.
3. Make the socket option SO_LINGER optional. This is a special case for the discovery scanner and might cause unexpected repercussions for other scanners.

[ferdinandjarisch]

All comments should be addressed.

Maybe let it rest for a day to let me double check tomorrow, otherwise it should be good to merge.

[ferdinandjarisch]

And for the record: this MR fully surpasses and thus closes #506