aiohttp doesn't comply autobahn test suite

[asvetlov]

Thanks @Nothing4You for raising the question and @njsmith for the fix suggestion.

are autobahn tests not used anymore?

Richard Schwab @Nothing4You Oct 20 21:30
also, is ws_connect only supported using a ClientSession?

Andrew Svetlov @asvetlov 01:26
autobahn workflow required manual steps. If you want to integrate them into aiohttp test suite -- you are welcome!
is ws_connect only supported using a ClientSession

Sorry, I don't follow. What did you expect? ClientSession is the root for client-side API


Richard Schwab @Nothing4You 01:52
forgot that a session is also required for other stuff
thought it was just for convenience
i just noticed that the autobahn tests have code that doesn't work anymore

Nathaniel J. Smith @njsmith 06:19
running the autobahn tests is kind of annoying, but wsproto has an automated harness that you might be able to steal parts of: https://github.com/python-hyper/wsproto/tree/master/compliance

At least we should run autobahn manually and fix it; ideally automate it.

[webknjaz]

running the autobahn tests is kind of annoying, but wsproto has an automated harness that you might be able to steal parts of: https://github.com/python-hyper/wsproto/tree/master/compliance

@anesabml I think this could be a starting point for automating the test runs in our CIs ☝️

[anesabml]

@webknjaz it's a great example, and it got me thinking if we could just use docker-py / python-on-whales instead of pytest-docker-compose to start the docker containers.
(I am sorry if this is taking too long but I just want to make sure that I am doing the right thing)

[webknjaz]

Yeah, maybe. Honestly, I'd be happy to avoid a vendor lock and make sure that it also works with a rootless podman too. But if the libs don't support it, hitting up docker directly is fine by me.

[Dreamsorcerer]

What's still needed to get these tests passing? It's rather a shame that >10% of our CI time is spent running xfail tests. Maybe it's better to disable these tests for now? Or, only run them in specific scenarios (like push to master, so it doesn't run on PRs)?