ws_client::update() does ignore continuation frame op-code returned by WebSocket::recv_data_frame() #2375
Comments
olk
changed the title
olk
changed the title
olk
changed the title
|
@olk Thanks for reporting this issue! Would you like to open a PR to fix it? /help |
|
@roycaihw: GuidelinesPlease ensure that the issue body includes answers to the following questions:
For more details on the requirements of such an issue, please see here and ensure that they are met. If this request no longer meets these requirements, the label can be removed In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
the
help wanted
I've a patch but unfortunately it doesn't solve the issue. It might be that the fix is wrong or the problem is caused by somewhere else. |
I'm using Apache Airflow 2.10.5 and facing randomly corrupted return values from the KubernetesPodOperator. The returned JSON is validated inside function
extract_xcom_json()(providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/utils/pod_manager.py) viajson.loads(result). This results in a error "json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 604 (char 603)"There have been similar bug reports: #2226 (comment)
The JSONDecodeError indicates that not the complete document was transferred.
I've added debug output (priitng the frame as it is written to the channel) in WSClient::update():
As you see the frame/JSON document gets truncated.
The function
extract_xcom_json()callsws::ws_client::update()(timeout=None). This function runs websocket's poll()-function. For the the polled socketop_code, frame = self.sock.recv_data_frame()is invoked. The code inws_client::update()does only deal with op-codes ABNF.OPCODE_CLOSE, ABNF.OPCODE_BINARY and ABNF.OPCODE_TEXT while op-code ABNF.OPCODE_CONT (continuation of fragmented data from previous text or binary frame) is ignored.From description of WebSocket's
recv_data_frame():"The message frame is always a frame object, whose frame.opcode is the opcode of the LATEST-received frame (so in the case of fire_cont_frame=True and IF there is fragmentation, the frame.opcode is cont if the current frame is a cont frame else text/binary, or in the case of fire_cont_frame=False (default), the frame.opcode is text or binary if there was no fragmentation of the current message, or cont if this was the final message of a fully received and fully concatenated fragmented message. The frame.data holds the data of either the current frame (if fire_cont_frame=True) or the concatenated data for all the frames if fire_cont_frame=False (default)."
I believe that ABNF.OPCODE_CONT must be handled by function
ws_client::update()too!Environment:
Kubernetes version (
kubectl version):Client Version: v1.32.2
Kustomize Version: v5.5.0
Server Version: v1.31.6+k3s1
OS (e.g., MacOS 10.13.6):
Debian 12
Python version (
python --version)Python 3.9
The text was updated successfully, but these errors were encountered: