Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config: AssetEnabled upgrade #1735

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

gbjk
Copy link
Collaborator

@gbjk gbjk commented Nov 29, 2024

Config:

  • Moves AssetEnabled from being a bool pointer to just being a bool

Exchanges:

  • Rename StoreAssetPairFormat to StoreAssetPairStore
  • StorePairs(enabled) set assetEnabled if pairs list is not empty
  • StorePairs(enabled) add pairs to Available as well
    • This is a convenience for all the test that have to do both, as well as users. Can be dropped if contentious
  • Fix error formatting for DisableAssetWebsocketSupport logging
  • Various assertifications in tests
  • Simplify exchange's defaultPairFormatting handling and make it somewhat consistent.
    • Could do more, but there's a larger overhaul of this coming
  • SetAssetEnabled will not error on no-op (already disabled or enabled)

Bitfinex:

  • Fix tests for MarginFunding subs

Type of change

Please delete options that are not relevant and add an x in [] as item is complete.

  • New feature (non-breaking change which adds functionality)

How has this been tested

  • go test ./... -race
  • golangci-lint run

@gbjk gbjk self-assigned this Nov 29, 2024
@gbjk gbjk requested a review from thrasher- November 29, 2024 06:19
@gbjk gbjk added the review me This pull request is ready for review label Nov 29, 2024
@gbjk gbjk force-pushed the feature/config_assetenabled branch 2 times, most recently from 3697ede to 5fb08d2 Compare November 30, 2024 06:05
@gloriousCode gloriousCode added the nomerge requires dependency This pull request is dependent on another, so it can't be merged until the dependent one is merged label Dec 1, 2024
@gbjk gbjk force-pushed the feature/config_assetenabled branch from 026f88f to 272827a Compare December 2, 2024 01:28
@gbjk gbjk mentioned this pull request Dec 4, 2024
3 tasks
@gbjk gbjk force-pushed the feature/config_assetenabled branch from 942b49b to e2d81d0 Compare December 9, 2024 08:13
@gbjk gbjk removed the nomerge requires dependency This pull request is dependent on another, so it can't be merged until the dependent one is merged label Dec 9, 2024
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick test on gateio, when I set everything but spot assets to false for asset enabled it still fetches and subscribes all assets. Doesn't seem to be happening on master, once resolved I will do a more in-depth check.

@gbjk gbjk force-pushed the feature/config_assetenabled branch from c092bfa to 65c05a0 Compare December 13, 2024 03:28
@gbjk
Copy link
Collaborator Author

gbjk commented Dec 13, 2024

Quick test on gateio, when I set everything but spot assets to false for asset enabled it still fetches and subscribes all assets. Doesn't seem to be happening on master, once resolved I will do a more in-depth check.

Fixed gbjk@65c05a063

@gbjk gbjk force-pushed the feature/config_assetenabled branch from 65c05a0 to 4a4fb99 Compare December 13, 2024 03:44
@gbjk gbjk mentioned this pull request Dec 17, 2024
5 tasks
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great cleanup! tested and runs great. Just some minor nits.

if err != nil {
log.Errorln(log.ExchangeSys, err)
log.Errorf(log.ExchangeSys, "%s error storing `%s` default asset formats: %s", {{.Variable}}.Name, asset.Spot, err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you wrap all these details on the error paths in StoreAssetPairStore? That way we don't need to have this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole direction here was the opposite of this change request.
The ideology I'm following is that functions return the error they encounter, without their own context.
If you call a go standard library function, you'll get the same thing.
It's down to the caller to decide if it wants to give context to that error, and what that context looks like from it's perspective.
From the perspective of the function caller, they know what they called, and so the error returned should be undecorated.

Case in point: StoreAssetPairStore is not default asset formats so these callers all needed to clarify the context they were calling that function in.

This is why you'll often see me decorate errors one level above where GCT has previously been doing it.
Sometimes it can feel slightly weighty, in this example where we copy the same context everywhere.

We could definitely add a common error for errStoringDefaultAssetFormats though, and I think that's warranted.
I'll circle back to this after the others and see if you've replied :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding one more bit of context:
This implies that functions must also add their own context.
e.g. If StoreAssetPairStore encountered some error interacting with a database, like "Record not found", We'd expect that error to come without context.
So StoreAssetPairStore should add the context error searching for glove size in database: %w.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From an implementation perspective, maintaining this standard can introduce toil during implementation and reviews, especially since any failure in SetDefaults would inherently cause the exchange to malfunction. Differentiating between errors in this context seems redundant. Still suggest wrapping asset and exchange for the exchange method.
But I will leave this open for other peoples perspective. @thrasher- @gloriousCode

config/config_test.go Outdated Show resolved Hide resolved
currency/manager_test.go Show resolved Hide resolved
engine/rpcserver_test.go Outdated Show resolved Hide resolved
exchanges/bitmex/bitmex_wrapper.go Outdated Show resolved Hide resolved
Copy link

codecov bot commented Dec 27, 2024

Codecov Report

Attention: Patch coverage is 71.68142% with 64 lines in your changes missing coverage. Please review.

Project coverage is 37.38%. Comparing base (105591b) to head (bfa72e2).
Report is 36 commits behind head on master.

Files with missing lines Patch % Lines
exchanges/exchange.go 44.44% 10 Missing and 15 partials ⚠️
exchanges/binance/binance_wrapper.go 33.33% 4 Missing and 2 partials ⚠️
exchanges/bitfinex/bitfinex_wrapper.go 66.66% 3 Missing and 2 partials ⚠️
exchanges/bitmex/bitmex_wrapper.go 66.66% 3 Missing and 2 partials ⚠️
exchanges/kraken/kraken_wrapper.go 66.66% 3 Missing and 2 partials ⚠️
exchanges/deribit/deribit_wrapper.go 42.85% 2 Missing and 2 partials ⚠️
exchanges/btse/btse_wrapper.go 76.92% 2 Missing and 1 partial ⚠️
exchanges/bybit/bybit_wrapper.go 25.00% 2 Missing and 1 partial ⚠️
exchanges/huobi/huobi_wrapper.go 80.00% 2 Missing and 1 partial ⚠️
exchanges/kucoin/kucoin_wrapper.go 80.00% 2 Missing and 1 partial ⚠️
... and 1 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1735      +/-   ##
==========================================
+ Coverage   37.13%   37.38%   +0.24%     
==========================================
  Files         414      416       +2     
  Lines      180198   178480    -1718     
==========================================
- Hits        66925    66723     -202     
+ Misses     105413   104001    -1412     
+ Partials     7860     7756     -104     
Files with missing lines Coverage Δ
backtester/engine/setup.go 58.59% <100.00%> (+0.70%) ⬆️
config/config.go 87.30% <ø> (+0.25%) ⬆️
config/versions/v3.go 100.00% <100.00%> (ø)
currency/manager.go 93.66% <100.00%> (-0.35%) ⬇️
exchanges/sharedtestvalues/sharedtestvalues.go 76.59% <100.00%> (+11.28%) ⬆️
exchanges/binanceus/binanceus_wrapper.go 41.04% <60.00%> (+0.15%) ⬆️
exchanges/btse/btse_wrapper.go 44.63% <76.92%> (+0.85%) ⬆️
exchanges/bybit/bybit_wrapper.go 46.63% <25.00%> (+3.42%) ⬆️
exchanges/huobi/huobi_wrapper.go 39.60% <80.00%> (-0.37%) ⬇️
exchanges/kucoin/kucoin_wrapper.go 31.86% <80.00%> (+0.12%) ⬆️
... and 6 more

... and 45 files with indirect coverage changes

@gbjk gbjk requested a review from shazbert December 27, 2024 02:52
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current changes tACK. 🚀

@gloriousCode gloriousCode added the szrc shazbert review complete label Dec 29, 2024
@gbjk gbjk force-pushed the feature/config_assetenabled branch from 5293a16 to c1cbeff Compare January 8, 2025 08:27
gbjk added 2 commits January 9, 2025 15:55
Because this isn't really a file, it's a byte slice
@gbjk gbjk mentioned this pull request Jan 9, 2025
3 tasks
@gbjk gbjk force-pushed the feature/config_assetenabled branch from c1cbeff to bfa72e2 Compare January 10, 2025 06:53
@gbjk gbjk requested a review from gloriousCode January 10, 2025 06:54
@gbjk
Copy link
Collaborator Author

gbjk commented Jan 10, 2025

I'm going to rebase this onto #1770, because I think that's kinda necessary for Downgrade to be testable.

gbjk added 14 commits January 10, 2025 14:03
This became more messy with Disabling something that's defaulted to
disabled.
Taking an idealogical stance against erroring that what you want to have
done is already done.
Previously we were calling it "Format", but accepting everything from
the PairStore.
We were also defaulting to turning the Asset on.

Now callers need to get their AssetEnabled set as they want it, so
there's no magic

This change also moves responsibility for error wrapping outside to the
caller.
Previously we ignored the field and just turned on everything.
I think that was because we couldn't get at the old value.
In either case, we have the option to do better, and respect the
assetEnabled value
@gbjk gbjk force-pushed the feature/config_assetenabled branch from bfa72e2 to 65d4423 Compare January 10, 2025 07:10
@shazbert shazbert added nomerge requires dependency This pull request is dependent on another, so it can't be merged until the dependent one is merged and removed szrc shazbert review complete labels Jan 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
nomerge requires dependency This pull request is dependent on another, so it can't be merged until the dependent one is merged review me This pull request is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants