Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

okx: decouple outbound trading functionality from multiplexer #1600

Open
wants to merge 20 commits into
base: master
Choose a base branch
from

Conversation

shazbert
Copy link
Collaborator

@shazbert shazbert commented Aug 2, 2024

PR Description

  • The multiplexer was a good intermediary but added in overhead and code complexity. This PR shifts the burden of websocket message match management to stream.Match using the function SendMessageReturnResponse.

  • It also shifts the burden of JSON unmarshalling to the request calling routine and the message []byte is shifted via pointer, which has a higher probability of improved websocket message handling.

  • The function design attempts to mirror REST request flow. This is probably going to be a multi-stage decoupling for multi-connection management. All done thanks Sam.

  • For matching signature, have opted to use Unix timestamp common.Counter. There was slight overhead from common.GenerateRandomString.

  • Removes WsRequestSemaphore due to upstream changes incorporating request rate limiter.

  • Adds TODO to export request rate limiting functionality so that we can couple rate limits between REST and websocket connections.

  • Waiting for exchanges: Okx Update #1420

Type of change

Please delete options that are not relevant and add an x in [] as item is complete.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How has this been tested

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration and
also consider improving test coverage whilst working on a certain feature or package.

  • go test ./... -race
  • golangci-lint run
  • Test X

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation and regenerated documentation via the documentation tool
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally and on Github Actions with my changes
  • Any dependent changes have been merged and published in downstream modules

@shazbert shazbert added the review me This pull request is ready for review label Aug 2, 2024
@shazbert shazbert requested a review from a team August 2, 2024 06:11
@shazbert shazbert self-assigned this Aug 2, 2024
Copy link

codecov bot commented Aug 2, 2024

Codecov Report

Attention: Patch coverage is 26.49254% with 197 lines in your changes missing coverage. Please review.

Project coverage is 37.17%. Comparing base (4de4e3d) to head (f9a7beb).
Report is 12 commits behind head on master.

Files with missing lines Patch % Lines
exchanges/okx/okx_websocket_requests.go 35.03% 102 Missing ⚠️
exchanges/okx/okx_types.go 9.52% 38 Missing ⚠️
exchanges/okx/okx.go 0.00% 30 Missing ⚠️
exchanges/okx/okx_websocket.go 15.00% 15 Missing and 2 partials ⚠️
exchanges/okx/okx_wrapper.go 47.36% 8 Missing and 2 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1600      +/-   ##
==========================================
+ Coverage   37.09%   37.17%   +0.08%     
==========================================
  Files         414      415       +1     
  Lines      180264   179960     -304     
==========================================
+ Hits        66860    66902      +42     
+ Misses     105549   105203     -346     
  Partials     7855     7855              
Files with missing lines Coverage Δ
exchanges/okx/okx_wrapper.go 30.63% <47.36%> (-0.45%) ⬇️
exchanges/okx/okx_websocket.go 44.05% <15.00%> (+11.39%) ⬆️
exchanges/okx/okx.go 20.27% <0.00%> (+0.26%) ⬆️
exchanges/okx/okx_types.go 37.03% <9.52%> (-3.42%) ⬇️
exchanges/okx/okx_websocket_requests.go 35.03% <35.03%> (ø)

... and 9 files with indirect coverage changes

---- 🚨 Try these New Features:

@shazbert shazbert requested a review from samuael August 2, 2024 06:27
@samuael
Copy link
Collaborator

samuael commented Aug 2, 2024

Thank you for your changes on the Okx exchange and for inviting me to review them. This PR is very optimized and very clean. The only proposal I have on this PR is to eliminate the Multiplexer by handling the incoming data by checking possible login error codes from the incoming data and handling the data based on that filter; Since it is only used by the "login" operation.

For instance, these are the possible incoming login error codes that can be used to filter login errors.

  1. 60022, 60023, 60024, 60026, 63999, 60032, 60011, 60009, 60005, 60021, and 60031; as specified in here https://www.okx.com/docs-v5/en/#error-code-websocket-public

@samuael
Copy link
Collaborator

samuael commented Aug 2, 2024

remove_multiplexer.patch

This is the change I am referring to. You might need to adjust a few things, but I have attempted to implement my intended modifications to eliminate the multiplexer.

@shazbert
Copy link
Collaborator Author

shazbert commented Aug 2, 2024

@samuael I added in your patch and did some adjustments to a type and the matching signature. Thanks heaps for looking into this.

@shazbert shazbert added bug and removed bug labels Aug 2, 2024
@samuael
Copy link
Collaborator

samuael commented Aug 2, 2024

@samuael I added in your patch and did some adjustments to a type and the matching signature. Thanks heaps for looking into this.

The pleasure is mine and thanks, Boss @shazbert. I hope to see this merged soon so that I can proceed with changes to the Okx Update PR.

Copy link
Collaborator

@gloriousCode gloriousCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

higher probability of improved websocket message handling

👀 well does it? Otherwise remove such a claim from your description 🚨

Cool stuff otherwise

exchanges/okx/okx_websocket_requests.go Outdated Show resolved Hide resolved
exchanges/okx/okx_websocket.go Outdated Show resolved Hide resolved
exchanges/okx/okx_websocket.go Show resolved Hide resolved
@shazbert shazbert requested a review from gloriousCode August 15, 2024 03:29
@shazbert
Copy link
Collaborator Author

higher probability of improved websocket message handling

👀 well does it? Otherwise remove such a claim from your description 🚨

Cool stuff otherwise

It was speculation, I can remove that wording? So this PR removes what was happening before including the inline JSON calls in the websocket reader/handler, I haven't tested it in a benchmark. That was the reason I suggested it had an higher probability of improved websocket message handling. I can see now why those choice of words are not entirely correct.

@shazbert
Copy link
Collaborator Author

shazbert commented Sep 4, 2024

Opted to label this as medium priority to highlight that it's complementary to #1420 and it's updates. This does not necessarily need to be merged first but it reduces a lot of code complexity.

Copy link
Collaborator

@gloriousCode gloriousCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I AM MOSTLY HAPPY THANK YOU

exchanges/okx/okx_websocket_requests.go Outdated Show resolved Hide resolved
exchanges/okx/okx_websocket.go Outdated Show resolved Hide resolved
exchanges/okx/okx_websocket_requests.go Outdated Show resolved Hide resolved
exchanges/okx/okx_websocket.go Outdated Show resolved Hide resolved
@shazbert shazbert requested a review from gloriousCode October 10, 2024 17:07
Comment on lines 213 to 215
if intermediary.Code > 1 {
return fmt.Errorf("error code: %d message: %s", intermediary.Code, intermediary.Message)
}
Copy link
Collaborator

@gloriousCode gloriousCode Oct 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Order code 2 was previously handled. Your comments on partial success in those functions likely won't get hit with a response of 2

Also Code 1 is failure

edit: hang on, this might be REST

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good catch; I bumped it to 2 so that anything beyond that can fail here, 1&2 codes don't actually need to fail here. None of my testing actually returned a code 2 that's why I must have forgotten to put it in. Example:

Okx wss://ws.okx.com:8443/ws/v5/private: Sending message: {"id":"1","op":"batch-orders","args":[{"instId":"BTC-USDT","tdMode":"cash","ccy":"USDT","side":"buy","ordType":"limit","sz":"0.0001","px":"-1"},{"instId":"BTC-USDT","tdMode":"cash","ccy":"USDT","side":"buy","ordType":"limit","sz":"0.0001","px":"20000"}]}
Okx wss://ws.okx.com:8443/ws/v5/private: Received response [1/1]: {"id":"1","op":"batch-orders","code":"2","msg":"","data":[{"tag":"","ts":"1728858915989","ordId":"","clOrdId":"","sCode":"51000","sMsg":"Parameter px  error"},{"tag":"","ts":"1728858915990","ordId":"1891010883570696192","clOrdId":"","sCode":"0","sMsg":"Order successfully placed."}],"inTime":"1728858915989134","outTime":"1728858915990899"}

Correct code 1 is a failure but the data holds the specifics of the error in sCode and sMsg and that can be checked in the calling function.

Copy link
Collaborator

@gloriousCode gloriousCode Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that you're trying to put the onus on the caller, but you also don't give the caller the opportunity to do so since you only return the Data property and nothing else. Then with this:

if len(orderResp) != 1 {
		return nil, errNoValidResponseFromServer
	}

There could be a Message and Code (of value 1) from the parent which could contain details but its instead thrown out and you don't know why.

How about the check is Seems like this can't apply

	if intermediary.Code != 0 && intermediary.Code != 2 {

edit: I want a better handling than errNoValidResponseFromServer

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All code 1 and code 2 API examples don't produce a message; 1 is a total failure and 2 is a partial failure which have data returned with specific scode and smsg to each action. From testing anything above 2 which has a message component; I have only seen request errors pertaining to operation string, connection authentication issues and they don't send any data back for the caller. The only reason I added the length check was to protect against a possible panic.

Happy to add in whatever you want though send me a diff on what you propose and the response you have found anything while testing and I will update it.

@shazbert shazbert added nomerge requires dependency This pull request is dependent on another, so it can't be merged until the dependent one is merged and removed blocked medium priority labels Nov 21, 2024
@shazbert shazbert removed the review me This pull request is ready for review label Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
nomerge requires dependency This pull request is dependent on another, so it can't be merged until the dependent one is merged
Projects
Status: In review
Development

Successfully merging this pull request may close these issues.

3 participants