ethstats plug-in fails with HTTP error 504 if not given full URI

[veox]

• trinity version: trinity-v0.1.0-alpha.16-15-g3eb6dc5d (current master: commit 3eb6dc5)
• OS: Arch Linux
• Python Version (python --version): 3.7.0
• Environment (output of pip freeze): MISSING

What is wrong?

PR #1336 squash-merged work from PR #1216, implementing the ethstats plug-in.

I've tried it with an actual "secret token" I have for wss://ropsten-stats.parity.io/, and there's always a 504 when trying to connect via WebSocket:

   ERROR  09-29 23:36:59       EthstatsService  Unexpected error in <trinity.plugins.builtin.ethstats.ethstats_service.EthstatsService object at 0x7f4ac6dafa58>, exiting
Traceback (most recent call last):                                                                     
  File "/home/veox/src/py-evm/p2p/service.py", line 103, in run                                        
    await self._run()                                                                                  
  File "/home/veox/src/py-evm/trinity/plugins/builtin/ethstats/ethstats_service.py", line 68, in _run  
    async with websockets.connect(self.server_url) as websocket:                                       
  File "/home/veox/src/py-evm/.virtualenv/py-evm/lib/python3.7/site-packages/websockets/py35/client.py", line 2, in __aenter__
    return await self                                                                                  
  File "/home/veox/src/py-evm/.virtualenv/py-evm/lib/python3.7/site-packages/websockets/py35/client.py", line 19, in __await_impl__
    extra_headers=protocol.extra_headers,                                                              
  File "/home/veox/src/py-evm/.virtualenv/py-evm/lib/python3.7/site-packages/websockets/client.py", line 263, in handshake
    raise InvalidStatusCode(status_code)                                                               
websockets.exceptions.InvalidStatusCode: Status code not 101: 504                                      
Traceback (most recent call last):                                                                     
  File "/home/veox/src/py-evm/p2p/service.py", line 103, in run                                        
    await self._run()                                                                                  
  File "/home/veox/src/py-evm/trinity/plugins/builtin/ethstats/ethstats_service.py", line 68, in _run  
    async with websockets.connect(self.server_url) as websocket:                                       
  File "/home/veox/src/py-evm/.virtualenv/py-evm/lib/python3.7/site-packages/websockets/py35/client.py", line 2, in __aenter__
    return await self                                                                                  
  File "/home/veox/src/py-evm/.virtualenv/py-evm/lib/python3.7/site-packages/websockets/py35/client.py", line 19, in __await_impl__
    extra_headers=protocol.extra_headers,
  File "/home/veox/src/py-evm/.virtualenv/py-evm/lib/python3.7/site-packages/websockets/client.py", line 263, in handshake
    raise InvalidStatusCode(status_code)
websockets.exceptions.InvalidStatusCode: Status code not 101: 504

EDIT: As a sanity check, I've tried with my previous setup of jsonrpcproxy+eth-net-intelligence-api, and it still works. So the URL/secret are correct.

How can it be fixed

See comment.

Either:

• add api to the URL in the plug-in ourselves; or
• instruct the user (in docs?) to specify the full endpoint path and rename "URL" to "URI".

[veox]

This may be due to the fact that I'm actually connection to WSS. websockets' docs have this to say on SSL/TLS.

EDIT: Although, this seems to be irrelevant:

For example, you can set the ssl keyword argument to a SSLContext to enforce some TLS settings. When connecting to a wss:// URI, if this argument isn’t provided explicitly, it’s set to True, which means Python’s default SSLContext is used.

[voith]

@veox I'm not really sure about the error but just want to point out that the websockets version pinned in setup.py is 5.0.1 and that version is not compatible with python3.7. python 3.7 support in websockets was added in v6.0.0

[veox]

@voith Yes, I'm trying now with websockets==6.0 and web3==4.6.0, and still getting the same.

---

EDIT: Well, I don't see anything obvious, so I guess the best approach is to run an ethstats server oneself, to get access to the "other side" of the logs.

Perhaps after reviewing the references in issue #1187, to see if this is a known quirk in ethstats.

[evgeniuz]

I still think this may be caused by the fact that this endpoint uses SSL. I was testing against local eth-netstats server (and that one obviously doesn't use SSL). Will try to look more closely into this.

[evgeniuz]

Tried to test this with SSL enabled, but it works for me.

@veox BTW, if you pass wss://ropsten-stats.parity.io/ as server URL, can you please try passing wss://ropsten-stats.parity.io/api, as that is endpoint clients should connect to.

[veox]

Ah! You are correct, missing "api" in the URI was the culprit. :) It now works as expected.

For ref (and possible future doc), I'm testing it as:

PYTHONWARNINGS='ignore' trinity \
    --network-id=3 --sync-mode=light --discv5 --max-peers=NUMBER \
    --ethstats \
    --ethstats-server-url="wss://ropsten-stats.parity.io/api" \
    --ethstats-server-secret="SECRET" \
    --ethstats-node-id="veox-trinity-dev-lightclient" \
    --ethstats-node-contact="veox"

[veox]

I've added suggestions to "how it can be fixed" section in issue description.

[evgeniuz]

I think second approach is better, as it is more explicit. Will add note about /api to --help output and rename parameters to URI.

[pipermerriam]

Also, I think that we can auto select a default for the URI if the node is running against ropsten/mainnet or really any network ID for which we know the right default URI.