Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC of generated code for type-safe SQL #1

Draft
wants to merge 3,375 commits into
base: master
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
3375 commits
Select commit Hold shift + click to select a range
b71e6ba
services/horizon: Add liquidity_pools to TruncateIngestStateTables (#…
bartekn Aug 27, 2021
c53d36f
services/horizon: Add /liquidity_pools endpoint (#3860)
bartekn Aug 27, 2021
eab8c35
bump check_release_hash to 1.16 to match circle and jenkins
Aug 27, 2021
22c2ec8
Merge pull request #3854 from stellar/amm-sdk
Aug 27, 2021
199502e
Merge branch 'master' into check_release_hash_go_version
Aug 27, 2021
b3b9547
Merge pull request #3864 from stellar/check_release_hash_go_version
Aug 27, 2021
bd8efca
services/horizon: Update TestProtocol14StateVerifier to test Liquidit…
bartekn Aug 30, 2021
ff3b615
services/horizon: Implement /liquidity_pools/{id}/{transactions,opera…
2opremio Aug 30, 2021
78a99bd
services/horizon: Add missing tx result codes (#3866)
bartekn Aug 30, 2021
8cb8106
services/horizon: Ingest Liquidity Pool Trades (#3857)
tamirms Aug 30, 2021
446120e
services/horizon: Fix operation ordering when querying by claimable b…
2opremio Aug 31, 2021
247b8e5
services/horizon: Implement /liquidity_pools/{id}/effects endpoint (#…
2opremio Aug 31, 2021
dced1cc
Combine all AMM DB migrations into a single file (#3876)
2opremio Aug 31, 2021
538ea4d
services/horizon: Allow filtering accounts by liquidity pool particip…
Shaptic Aug 31, 2021
b98ce0b
Add trustline liquidity pool id to RevokeSponsosrhip operation (#3877)
2opremio Aug 31, 2021
16850bc
Merge branch 'master' into amm
bartekn Aug 31, 2021
e6223f7
services/horizon: Liquidity pool happy path integration test (#3868)
bartekn Aug 31, 2021
90bbce1
services/horizon: Omit asset details on failed liquidity pools operat…
2opremio Sep 1, 2021
bacc986
Fix typo
2opremio Sep 1, 2021
8ced08a
xdr: Enforce asset ordering in xdr pool (#3865)
Sep 2, 2021
6b990cf
services/horizon: Add Liquidity Pools to asset stats (#3887)
bartekn Sep 3, 2021
bd0e6f6
Fix trades and trade aggregations (#3885)
tamirms Sep 3, 2021
74a307c
services/horizon: Add TestLiquidityPoolRevoke integration test (#3890)
bartekn Sep 6, 2021
873a846
services/horizon: Include liquidity pool trades in trades endpoint (#…
tamirms Sep 6, 2021
becf4b6
services/horizon: Add /liquidity_pools to root resource _links (#3895)
bartekn Sep 7, 2021
1fae4ee
txnbuild: implement encoding.Text{M,Unm}arshaler in Transaction, FeeB…
leighmcculloch Sep 7, 2021
4576cee
clients/horizonclient: allow Fund on other networks and rely on serve…
leighmcculloch Sep 7, 2021
d04fe84
services/horizon: Soft delete liquidity pools (#3898)
tamirms Sep 8, 2021
ad186c8
services/horizon: Split App.UpdateLedgerState into Core and Horizon l…
bartekn Sep 9, 2021
fca6ff5
Merge pull request #3901 from stellar/master
bartekn Sep 9, 2021
6f9372d
services/horizon: Fix verify-image issues (#3902)
bartekn Sep 9, 2021
2a43a5d
Update docker image build timestamp format
jacekn Sep 10, 2021
56c8c74
.vscode: fix tasks file (#3908)
leighmcculloch Sep 13, 2021
0cd93ab
Include liquidity pools in OrderBookStream (#3909)
tamirms Sep 13, 2021
2648390
services/horizon/internal/actions: Fix liquidity_pool filter for acco…
tamirms Sep 13, 2021
78532aa
Add integration test for failed liquidity pool operations (#3911)
2opremio Sep 14, 2021
a593fc3
Avoid docker-compose side-effects during integration tests cleanup (#…
2opremio Sep 14, 2021
c8c34e6
Fix error in one Makefile
jacekn Sep 14, 2021
62c4edf
Merge branch 'master' into timestamps
jacekn Sep 14, 2021
7f731ce
Merge pull request #3904 from jacekn/timestamps
jacekn Sep 14, 2021
31f51e3
Fix trade prices (#3915)
tamirms Sep 14, 2021
a6081ce
services/horizon/ingest: Batch Offers insert/updates (#3917)
bartekn Sep 14, 2021
7eee5d8
services/horizon/ingest: Batch Offers insert/updates (#3917)
bartekn Sep 14, 2021
d1876ff
services/horizon: Horizon 2.8.2 CHANGELOG (#3920)
bartekn Sep 15, 2021
e05d29b
Added 5 second grace period to ReadChallengeTx MinTime constraint
JakeUrban Aug 12, 2021
695ffca
correct grace period from 5 seconds to 5 minutes
JakeUrban Aug 12, 2021
9905918
fix formatting
JakeUrban Aug 12, 2021
32b9ab7
Merge pull request #3824 from JakeUrban/txnbuild-add-grace-period-to-…
JakeUrban Sep 15, 2021
e679f8c
Add additional test cases for parallel reingestion (#3916)
2opremio Sep 16, 2021
d4dd2be
`check_release_hash` improvements (#3925)
bartekn Sep 16, 2021
99a1533
keypair: add test coverage for hint functions (#3922)
leighmcculloch Sep 16, 2021
839c473
crc16: add tests (#3924)
leighmcculloch Sep 16, 2021
e00a568
crc16: move crc16 inside strkey/internal (#3932)
leighmcculloch Sep 17, 2021
664e661
Speed up path finding dfs by avoiding edges leading to visited nodes …
tamirms Sep 17, 2021
7128797
Merge branch 'master' into release-horizon-v2.8.2
bartekn Sep 17, 2021
9bd04be
Merge branch 'release-horizon-v2.8.2' of github.com:stellar/go into r…
bartekn Sep 17, 2021
1b74d9f
services/horizon: Change `ProcessorsRunDuration` metric type from cou…
bartekn Sep 20, 2021
7f78513
Merge branch 'master' into amm
2opremio Sep 20, 2021
a3b6d55
Merge pull request #3941 from 2opremio/merge-master
2opremio Sep 20, 2021
289706f
Sort liquidity pool revocation claimable balance effects (#3943)
2opremio Sep 20, 2021
491f1f6
keypair: improve performance of FromAddress/Full Hint, Sign, Verify (…
leighmcculloch Sep 20, 2021
abf5877
Merge branch 'master' into release-horizon-v2.8.2
bartekn Sep 21, 2021
649d53f
Merge pull request #3923 from stellar/release-horizon-v2.8.2
bartekn Sep 21, 2021
49c60a2
services/horizon: Batch remove accounts, move ingestion to `AccountsP…
bartekn Sep 21, 2021
bf3f1ea
services/horizon: Batch liquidity pool updates/removals (#3944)
2opremio Sep 21, 2021
72dd2c6
ingest: Return progress in CheckpointChangeReader (#3946)
bartekn Sep 21, 2021
90b5e69
services/horizon: Add a brin index on history_trades_60000(timestamp)…
Sep 22, 2021
3d11be3
services/horizon: Add a brin index on history_trades_60000(timestamp)…
Sep 22, 2021
371fe5f
services/horizon/internal/db2/history: Fix fee source muxed account i…
tamirms Sep 22, 2021
124e6aa
Remove migration to fix fee account transactions (#3949)
tamirms Sep 22, 2021
de5460c
services/horizon/internal/db2/history: Fix fee source muxed account i…
tamirms Sep 22, 2021
fc56e16
Remove migration to fix fee account transactions (#3949)
tamirms Sep 22, 2021
8ba190d
Horizon 2.8.3 CHANGELOG (#3950)
bartekn Sep 22, 2021
921c3be
Merge branch 'master' into release-horizon-v2.8.3
bartekn Sep 22, 2021
4572291
Merge pull request #3951 from stellar/release-horizon-v2.8.3
bartekn Sep 22, 2021
9d3fa7b
Horizon 2.8.3 CHANGELOG fix (#3952)
bartekn Sep 22, 2021
38d0590
protocols/horizon: Represent liquidity pool total_shares field as amo…
tamirms Sep 23, 2021
076feff
Include omitempty tag for counter_account field in trade json (#3955)
tamirms Sep 23, 2021
b5fd550
services/horizon: Batch claimable balance creation, updates and delet…
2opremio Sep 23, 2021
262d7a1
Updating old docs url to new docs urls (liquidity pools)
Sep 27, 2021
e9c3538
exp/orderbook: Expand path payment algorithm to search liquidity pool…
Shaptic Sep 27, 2021
0d201e5
Batch account data creation, updates and deletions (#3956)
2opremio Sep 28, 2021
cb9726e
Updating old docs url to new docs urls (#3959)
Sep 28, 2021
d817357
Merge branch 'master' into amm-bartek
bartekn Sep 28, 2021
cfd93ac
gofmt
bartekn Sep 28, 2021
9aef802
Batch deletion of offers with offer upserting (#3960)
2opremio Sep 28, 2021
c45c9eb
Merge branch 'master' into amm-bartek
2opremio Sep 28, 2021
06d4fb8
Merge pull request #3962 from bartekn/amm-bartek
2opremio Sep 28, 2021
a1f6d23
services/horizon: Upgrade Stellar-Core to 18.0.0 in integration tests…
bartekn Sep 28, 2021
a39f46d
services/horizon: Batch trustline removals (#3963)
2opremio Sep 28, 2021
5fa9e14
exp/orderbook: Adds tests for liquidity pool exchange math. (#3964)
Shaptic Sep 28, 2021
4aa3936
verify-range: Remove offer_id from DB compare checks (#3971)
bartekn Sep 29, 2021
521eec9
services/horizon: Ingest base and counter offer id when offer consume…
bartekn Sep 29, 2021
eb80a85
.vscode: fix tasks files again (#3967)
leighmcculloch Sep 29, 2021
69719b8
Fix uint32 argument parsing type mismatch (#3969)
2opremio Sep 29, 2021
beecf38
exp/orderbook: Speed up path finding by changing edgeSet data structu…
tamirms Sep 29, 2021
da498ac
services/horizon: Update `asset_type` for trust line sponsorship effe…
bartekn Sep 30, 2021
320e53c
Set default for ingest flag to be true (#3976)
tamirms Sep 30, 2021
4f10dce
exp/orderbook: Add parameter to exclude pools from path finding searc…
tamirms Oct 1, 2021
bec85cd
services/horizon: Use operation body to generate "claimant created" e…
bartekn Oct 1, 2021
2a9a24d
services/horizon: Horizon 2.9.0 CHANGELOG (#3978)
bartekn Oct 4, 2021
3bdaba4
txnbuild, clients/horizonclient: Release v8.0.0-beta.0. (#3981)
Shaptic Oct 4, 2021
9d98d38
services/horizon: Upgrade Stellar-Core to 18.0.1 (#3984)
bartekn Oct 5, 2021
489cd85
Handle negative transaction sequences gracefully (#3983)
2opremio Oct 5, 2021
1ce3707
services/horizon: Add link to LP API in CHANGELOG (#3985)
bartekn Oct 6, 2021
2f8891f
services/horizon/cmd: Fix db reingest range cmdlineparsing (#3996)
2opremio Oct 7, 2021
dd5852a
Remove unintentioned print statement
2opremio Oct 7, 2021
68b955d
services/horizon: Add more info about breaking changes in Horizon 2.9…
bartekn Oct 7, 2021
55e7c3f
services/horizon: expose the Latest-Ledger header to browser web page…
leighmcculloch Oct 7, 2021
f3fbe45
protocol/horizon/operations: fix AllowTrust comment to use SetTrustLi…
howardtw Oct 7, 2021
883f41f
services/horizon: Activate breaking change in /trades on protocol upg…
bartekn Oct 8, 2021
f202068
services/horizon: Fix empty offer_id in /trades before Protocol 18 (#…
bartekn Oct 8, 2021
c23ef29
historyarchive: Emit less log noise for missing fs paths. (#3903)
graydon Oct 8, 2021
360c7a7
services/horizon: Final 2.9.0 fixes (#4002)
bartekn Oct 11, 2021
fa2f19f
Remove `stellar-core new-db` invocation when reingesting (#4003)
2opremio Oct 11, 2021
4c48c96
Merge pull request #4004 from stellar/release-horizon-v2.9.0
bartekn Oct 12, 2021
7a349d5
Correct reingest-range output command name when invoking detect-gaps …
2opremio Oct 13, 2021
e325e29
support/log: Remove Entry.Logger from public API (#4012)
bartekn Oct 18, 2021
3646635
support/db: Add round_trip_time_seconds metric (#4009)
bartekn Oct 18, 2021
e542527
protocols/horizon: dont cast untyped const (#3988)
leighmcculloch Oct 18, 2021
1672dab
exp/orderbook: Use correct assets when calculating liquidity pool dis…
Shaptic Oct 20, 2021
55eea5f
services/horizon: Add state_verify_ledger_entries_count metric (#4015)
bartekn Oct 21, 2021
1f6d54b
services/horizon: Add ledger_fetch_duration_seconds metric (#4016)
bartekn Oct 21, 2021
e919a75
services/horizon: Add feature flag for extended ledger stats log (#4017)
bartekn Oct 21, 2021
e0c6753
ingest/ledgerbackend: Make sure Stellar-Core is not started before pr…
bartekn Oct 21, 2021
3c7b335
Sync XDR files with Core v18.1.0rc1 (#4021)
2opremio Oct 21, 2021
0aecbe6
Add CHANGELOG entry for Horizon 2.10.0 release (#4024)
2opremio Oct 22, 2021
71896cd
services/horizon: Add record-metrics command (#4023)
bartekn Oct 22, 2021
a97cd70
services/horizon: Increase http client timeout in record-metrics (#4025)
bartekn Oct 22, 2021
2606b8d
services/horizon: Register new ingestion metrics (#4026)
bartekn Oct 22, 2021
f825e6a
Make dump-ledger-state compatible with protocol 18 (#4028)
tamirms Oct 22, 2021
d4454ce
Add stellar-core to stellar/stellar-horizon image (#4019)
2opremio Oct 25, 2021
d936645
Bump go version of check_release_hash (#4032)
2opremio Oct 26, 2021
2c59426
Merge branch 'master' into release-horizon-v2.10.0
2opremio Oct 27, 2021
f93b8bf
Merge pull request #4034 from stellar/release-horizon-v2.10.0
2opremio Oct 27, 2021
c92d411
services/horizon/internal/ingest: Include Accounts and Account Data l…
tamirms Oct 28, 2021
862154b
services/ticker: move v1 to master (#4039)
stfung77 Nov 1, 2021
b36711b
Add stellar/go-xdr#16 (reduction on XDR encoding/decoding allocations…
2opremio Nov 3, 2021
f97d458
Add (failing) test to find pools by participating account
Shaptic Nov 3, 2021
7e733b6
Refactor the other tests to use the new helper
Shaptic Nov 3, 2021
6252779
Rename test
Shaptic Nov 3, 2021
953e0a3
Refactor test cases together
Shaptic Nov 3, 2021
01f2bce
Query liquidity pools by account id
erika-sdf Nov 3, 2021
9503efb
Merge unit tests
erika-sdf Nov 3, 2021
5870a83
Merge branch 'liquidity' of github.com:erika-sdf/go into liquidity
Shaptic Nov 3, 2021
721d350
Fix join sql syntax
Shaptic Nov 3, 2021
ffa4905
Drop redundant test
Shaptic Nov 3, 2021
37c6998
Minor refactors to test cases
Shaptic Nov 3, 2021
6c1bf6a
Remove slack notification from dump ledger state job (#4044)
tamirms Nov 4, 2021
c49e586
Clean up print statements
erika-sdf Nov 4, 2021
0acce98
Update parameter name: accountID -> account.
erika-sdf Nov 4, 2021
cad05d3
Update services/horizon/internal/actions/liquidity_pool.go
erika-sdf Nov 4, 2021
7d8d879
Add new liqudity pool test to db2/history
erika-sdf Nov 4, 2021
e4b1655
Merge branch 'liquidity' of https://github.com/erika-sdf/go into liqu…
erika-sdf Nov 4, 2021
f739c4f
Merge branch 'liquidity' of github.com:erika-sdf/go into liquidity
Shaptic Nov 4, 2021
e056dcf
Merge branch 'stellar:master' into liquidity
erika-sdf Nov 4, 2021
0bde629
Fix trustline creation helper to always have unique ledgerkey
Shaptic Nov 4, 2021
f5a25a2
Merge branch 'liquidity' of github.com:erika-sdf/go into liquidity
Shaptic Nov 4, 2021
5b5e84b
Update test to upsert correctly now, and change assertion order
Shaptic Nov 4, 2021
35a3229
Merge pull request #4043 from erika-sdf/liquidity
erika-sdf Nov 4, 2021
2c6a71e
Add `is_source` query parameter to transaction endpoints (#4035)
2opremio Nov 5, 2021
48bec1b
Remove --captive-core-reuse-storage-dir flag (#4048)
erika-sdf Nov 5, 2021
44db6f5
Add --max_assets_per_path_request flag (#4046)
erika-sdf Nov 5, 2021
beacb98
Revert "Add `is_source` query parameter to transaction endpoints (#40…
2opremio Nov 5, 2021
4ab2173
ingest: Remove base64 encoding when storing LedgerKey in memory (#4047)
bartekn Nov 6, 2021
518191f
xdr: update using xdrgen#65 (#3957)
leighmcculloch Nov 8, 2021
795ee0c
Let filewatcher check for binary hash to detect captive core updates.…
erika-sdf Nov 8, 2021
38f72ec
strkey: add muxed accounts support (#4053)
marcelosalloum Nov 9, 2021
059cf74
services/friendbot: removed friendbot is disabled error messages that…
leighmcculloch Nov 9, 2021
d18e30e
services/horizon: Change horizon_ingest_state_verify_ledger_entries m…
bartekn Nov 9, 2021
e302868
services/horizon: Implement horizon db fill-gaps command (#4060)
tamirms Nov 10, 2021
d72ab27
Create xdr.EncodingBuffer, which reduces buffer allocations (#4056)
2opremio Nov 10, 2021
04147cf
keypair: add BinaryMarsheler support to FromAddress (#4040)
leighmcculloch Nov 10, 2021
d877832
Add stellar/xdrgen#68 (#4063)
2opremio Nov 11, 2021
1a7b29b
Fix stream parsing error printing in captive core (#4066)
2opremio Nov 12, 2021
3e6867a
services/horizon/internal/db2/history: Improve effects for liquidity …
tamirms Nov 12, 2021
391f212
Use encoding buffer in state verifier (#4069)
2opremio Nov 13, 2021
459b836
Update horizon v2.11.0 changelog (#4072)
tamirms Nov 15, 2021
43cfa84
xdr: update using xdrgen#72 (#4068)
2opremio Nov 15, 2021
09ccf2c
xdr: update using xdrgen#73 (#4073)
2opremio Nov 15, 2021
6ac3ea9
xdr: update using xdrgen#71 (#4064)
2opremio Nov 15, 2021
26001c1
xdr and horizon: Optimize compress-marshaling of ledger keys (#4071)
2opremio Nov 16, 2021
835e2e7
Remove XDR decoder allocations from XdrStream (#4075)
2opremio Nov 16, 2021
3dc17b8
Merge pull request #4074 from stellar/release-horizon-v2.11.0
jacekn Nov 16, 2021
464b0ea
doc: use pull request title pattern for issues too (#4057)
leighmcculloch Nov 16, 2021
0a3ecee
Optimize captive-core pipe decoding (#4077)
2opremio Nov 17, 2021
2e826ca
Add marshal binary compress test (#4076)
2opremio Nov 17, 2021
4a3d850
services/horizon: Move a couple services metrics registration to be c…
Nov 17, 2021
3b18b94
Remove unused indexes from history_transactions (#4085)
2opremio Nov 17, 2021
3a52625
Return inner and outer result codes for fee bump transactions. (#4081)
erika-sdf Nov 18, 2021
ad88548
services/horizon: Update 51_remove_ht_unused_indexes.sql to drop inde…
bartekn Nov 19, 2021
e1e1ee3
services/horizon/docker: enable docker-compose tooling to work with u…
sreuland Nov 19, 2021
b359914
Add benchmark for BatchInsertBuilder (#4086)
2opremio Nov 22, 2021
0ba3e65
exp/orderbook: Update orderbook benchmark to use liquidity pools from…
tamirms Nov 23, 2021
582e305
xdr and support/db: Minor benchmark code quality improvements (#4093)
2opremio Nov 23, 2021
9e89c2d
exp/orderbook: Improve path finding algorithm (#4096)
tamirms Nov 24, 2021
83ec0b7
exp/orderbook: Use uint128s for pool calculations to improve performa…
2opremio Nov 29, 2021
8997a53
exp/orderbook: Represent assets in orderbook graph as int32 instead o…
tamirms Nov 30, 2021
2020829
Revert "exp/orderbook: Use uint128s for pool calculations to improve …
tamirms Nov 30, 2021
9a52e65
services/horizon: map client disconnects as Status 499 instead of 503…
sreuland Nov 30, 2021
b4d6344
Exclude arbitrage paths from path finding search (#4109)
tamirms Nov 30, 2021
d738423
strkey: reduce allocs in Encode (#4107)
leighmcculloch Dec 1, 2021
aa4c201
xdr and exp/orderbook: Reduce path search allocations (#4105)
2opremio Dec 1, 2021
d4871e9
strkey: reduce allocs in Encode (#4107)
leighmcculloch Dec 1, 2021
b226633
exp/orderbook: Use uint256 for pool calculations to improve performan…
2opremio Dec 2, 2021
5b5b452
service/ingest: repeated reingest of overlapping range causes duplica…
sreuland Dec 2, 2021
6fc6e9e
services/horizon: Update Horizon changelog for v2.12.0 release (#4119)
tamirms Dec 3, 2021
1d63eee
Micro-optimization for paths search (#4125)
2opremio Dec 4, 2021
434cad1
exp/orderbook: Periodically verify internal consistency of in memory …
tamirms Dec 5, 2021
e45f426
services/horizon/cmd: Fix parsing of reingest range flags like --para…
tamirms Dec 6, 2021
147558f
exp/orderbook: (#4128)
tamirms Dec 6, 2021
0c17ee5
Merge branch 'master' into release-horizon-v2.12.0
tamirms Dec 6, 2021
6f5f8c5
all: update golang.org/x/crypto and golang.org/x/net (#4130)
leighmcculloch Dec 7, 2021
a3ad250
Merge branch 'master' into release-horizon-v2.12.0
tamirms Dec 7, 2021
8aa0524
Merge pull request #4129 from stellar/release-horizon-v2.12.0
tamirms Dec 7, 2021
0985cff
ingest/ledgerbackend: Differentiate between isPrepared and isClosed i…
erika-sdf Dec 8, 2021
f25d8ce
gxdr: generate using go:generate (#4135)
leighmcculloch Dec 8, 2021
9008ecb
Add trades metrics to ingestion. (#4123)
erika-sdf Dec 8, 2021
88d2b8d
services/horizon: Horizon 2.12.1 CHANGELOG (#4138)
bartekn Dec 9, 2021
abcc975
all: make docker images consistently build from go 1.17 (#4139)
leighmcculloch Dec 9, 2021
f605f0d
all: use go run for go-bindata go generate commands (#4134)
leighmcculloch Dec 9, 2021
b8d04b5
Include empty paths in path finding response (#4137)
tamirms Dec 10, 2021
aaaf327
Optimize asset string conversion (#4140)
2opremio Dec 11, 2021
783df89
added example of feestats query in gojet
sreuland Dec 13, 2021
0e93ac6
run gofmt for cleanup
sreuland Dec 14, 2021
783b334
exp/services/market-tracker: fix go mod errors (#4136)
leighmcculloch Dec 15, 2021
bc2891f
Refactor HAL template generation in root endpoint (#4145)
2opremio Dec 17, 2021
8aa0ea4
support/log: decouple default logger and horizon unit tests (#4142)
sreuland Dec 17, 2021
58c51e4
xdr: Add AbsBeforeEpoch to ClaimableBalance:Predicate API resource (…
sreuland Dec 20, 2021
f559a0e
services/horizon/internal/db2: Add column and index to improve trade_…
tamirms Dec 21, 2021
3d396bd
Revert transient protocol18 trades changes (#4153)
2opremio Jan 3, 2022
194ecf4
Remove mutex from stream operations and count 'hello' event (#4151)
2opremio Jan 4, 2022
c402af9
horizon/ingest: refactor initial load of orderbook offers/pools to us…
sreuland Jan 4, 2022
9263a80
Refactor integration tests (#4159)
2opremio Jan 4, 2022
3a4318c
horizon/ingest: Add method to ingest.Change that returns liquidity po…
erika-sdf Jan 4, 2022
883cc76
Add more verbose logging to horizon start up (#4161)
erika-sdf Jan 6, 2022
34d9a48
services/horizon/internal/httpx: Exempt SSE requests from rate limiti…
tamirms Jan 6, 2022
e6e4a97
Specify ghr version to be v0.14.0 (#4157)
erika-sdf Jan 7, 2022
59839ad
Remove network call from history_archive_adapter unit test (#4166)
erika-sdf Jan 7, 2022
9a871d0
txnbuild: add ToXDR, Hash, HashHex to GenericTransaction (#4144)
leighmcculloch Jan 7, 2022
50f7a31
txnbuild: Enable Muxed accounts by default (#4169)
2opremio Jan 10, 2022
4250b29
txnbuild: Use xdr.Price to represent prices in txnbuild instead of st…
tamirms Jan 10, 2022
425b41a
services/horizon: Update cursor when bumping ledger to latest (#4150)
bartekn Jan 10, 2022
ba63c3d
txnbuild/horizonclient: Update changelog for v9 txnbuild/horizonclien…
2opremio Jan 11, 2022
182b86d
txnbuild: add ClearSignatures functions to Transaction and FeeBumpTra…
leighmcculloch Jan 11, 2022
063a59d
txnbuild: add AddSignatureDecorate to FeeBumpTransaction (#4174)
leighmcculloch Jan 12, 2022
d91b749
services/horizon/../sse: Stop counting the SSE preamble as an event …
2opremio Jan 13, 2022
5da91eb
Set docker platform flag so core works on ARM Macs (#4177)
Jan 13, 2022
161a0e9
Merge remote-tracking branch 'origin/master' into generated_db_sql_poc
sreuland Jan 14, 2022
da37c36
updated go.sum after merge
sreuland Jan 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion services/horizon/internal/actions/trade.go
Original file line number Diff line number Diff line change
@@ -165,7 +165,7 @@ func (handler GetTradesHandler) GetResourcePage(w HeaderWriter, r *http.Request)
}

var records []history.Trade
if err = trades.Page(ctx, pq).Select(ctx, &records); err != nil {
if records, err = trades.Page(ctx, pq).Select(ctx); err != nil {
return nil, err
}
var response []hal.Pageable
158 changes: 133 additions & 25 deletions services/horizon/internal/actions_trade_test.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
//lint:file-ignore U1001 Ignore all unused code, thinks the code is unused because of the test skips
package horizon

import (
"context"
"encoding/json"
"fmt"
"github.com/guregu/null"
"github.com/stellar/go/keypair"
"net/url"
"strconv"
"strings"
@@ -12,13 +16,14 @@ import (
"github.com/stellar/go/protocols/horizon"
"github.com/stellar/go/services/horizon/internal/db2/history"
. "github.com/stellar/go/services/horizon/internal/db2/history"
. "github.com/stellar/go/services/horizon/internal/test/trades"
"github.com/stellar/go/support/render/hal"
stellarTime "github.com/stellar/go/support/time"
"github.com/stellar/go/xdr"
)

func TestTradeActions_Index(t *testing.T) {
// TODO fix in https://github.com/stellar/go/issues/3835
t.Skip()
ht := StartHTTPTest(t, "trades")
defer ht.Finish()
var records []horizon.Trade
@@ -200,6 +205,9 @@ const week = int64(7 * 24 * time.Hour / time.Millisecond)
const aggregationPath = "/trade_aggregations"

func TestTradeActions_Aggregation(t *testing.T) {
// TODO fix in https://github.com/stellar/go/issues/3835
t.Skip()

ht := StartHTTPTest(t, "base")
defer ht.Finish()

@@ -376,6 +384,9 @@ func TestTradeActions_Aggregation(t *testing.T) {
}

func TestTradeActions_AmountsExceedInt64(t *testing.T) {
// TODO fix in https://github.com/stellar/go/issues/3835
t.Skip()

ht := StartHTTPTest(t, "base")
defer ht.Finish()
dbQ := &Q{ht.HorizonSession()}
@@ -414,6 +425,9 @@ func TestTradeActions_AmountsExceedInt64(t *testing.T) {
}

func TestTradeActions_IndexRegressions(t *testing.T) {
// TODO fix in https://github.com/stellar/go/issues/3835
t.Skip()

t.Run("Regression: https://github.com/stellar/go/services/horizon/internal/issues/318", func(t *testing.T) {
ht := StartHTTPTest(t, "trades")
defer ht.Finish()
@@ -446,6 +460,8 @@ func TestTradeActions_IndexRegressions(t *testing.T) {
// fields are correct for multiple trades that occur in the same ledger
// https://github.com/stellar/go/issues/215
func TestTradeActions_AggregationOrdering(t *testing.T) {
// TODO fix in https://github.com/stellar/go/issues/3835
t.Skip()

ht := StartHTTPTest(t, "base")
defer ht.Finish()
@@ -479,30 +495,6 @@ func TestTradeActions_AggregationOrdering(t *testing.T) {
}
}

func assertOfferType(ht *HTTPT, offerId string, idType OfferIDType) {
offerIdInt64, _ := strconv.ParseInt(offerId, 10, 64)
_, offerType := DecodeOfferID(offerIdInt64)
ht.Assert.Equal(offerType, idType)
}

// TestTradeActions_SyntheticOfferIds loads the offer_ids scenario and ensures that synthetic offer
// ids are created when necessary and not when unnecessary
func TestTradeActions_SyntheticOfferIds(t *testing.T) {
ht := StartHTTPTest(t, "offer_ids")
defer ht.Finish()
var records []horizon.Trade
w := ht.Get("/trades")
if ht.Assert.Equal(200, w.Code) {
if ht.Assert.PageOf(4, w.Body) {
ht.UnmarshalPage(w.Body, &records)
assertOfferType(ht, records[0].BaseOfferID, TOIDType)
assertOfferType(ht, records[1].BaseOfferID, TOIDType)
assertOfferType(ht, records[2].BaseOfferID, CoreOfferIDType)
assertOfferType(ht, records[3].BaseOfferID, CoreOfferIDType)
}
}
}

func TestTradeActions_AssetValidation(t *testing.T) {
ht := StartHTTPTest(t, "trades")
defer ht.Finish()
@@ -519,6 +511,9 @@ func TestTradeActions_AssetValidation(t *testing.T) {
}

func TestTradeActions_AggregationInvalidOffset(t *testing.T) {
// TODO fix in https://github.com/stellar/go/issues/3835
t.Skip()

ht := StartHTTPTest(t, "base")
defer ht.Finish()
dbQ := &Q{ht.HorizonSession()}
@@ -561,6 +556,9 @@ func TestTradeActions_AggregationInvalidOffset(t *testing.T) {
}

func TestTradeActions_AggregationOffset(t *testing.T) {
// TODO fix in https://github.com/stellar/go/issues/3835
t.Skip()

ht := StartHTTPTest(t, "base")
defer ht.Finish()
dbQ := &Q{ht.HorizonSession()}
@@ -614,3 +612,113 @@ func TestTradeActions_AggregationOffset(t *testing.T) {
})
}
}

//GetTestAsset generates an issuer on the fly and creates a CreditAlphanum4 Asset with given code
func GetTestAsset(code string) xdr.Asset {
var codeBytes [4]byte
copy(codeBytes[:], []byte(code))
ca4 := xdr.AlphaNum4{Issuer: GetTestAccount(), AssetCode: codeBytes}
return xdr.Asset{Type: xdr.AssetTypeAssetTypeCreditAlphanum4, AlphaNum4: &ca4, AlphaNum12: nil}
}

//Get generates and returns an account on the fly
func GetTestAccount() xdr.AccountId {
var key xdr.Uint256
kp, _ := keypair.Random()
copy(key[:], kp.Address())
acc, _ := xdr.NewAccountId(xdr.PublicKeyTypePublicKeyTypeEd25519, key)
return acc
}

//IngestTestTrade mock ingests a trade
func IngestTestTrade(
q *Q,
assetSold xdr.Asset,
assetBought xdr.Asset,
seller xdr.AccountId,
buyer xdr.AccountId,
amountSold int64,
amountBought int64,
timestamp stellarTime.Millis,
opCounter int64) error {

trade := xdr.ClaimAtom{
Type: xdr.ClaimAtomTypeClaimAtomTypeOrderBook,
OrderBook: &xdr.ClaimOfferAtom{
AmountBought: xdr.Int64(amountBought),
SellerId: seller,
AmountSold: xdr.Int64(amountSold),
AssetBought: assetBought,
AssetSold: assetSold,
OfferId: 100,
},
}

price := xdr.Price{
N: xdr.Int32(amountBought),
D: xdr.Int32(amountSold),
}

ctx := context.Background()
accounts, err := q.CreateAccounts(ctx, []string{seller.Address(), buyer.Address()}, 2)
if err != nil {
return err
}
assets, err := q.CreateAssets(ctx, []xdr.Asset{assetBought, assetSold}, 2)
if err != nil {
return err
}

batch := q.NewTradeBatchInsertBuilder(0)
batch.Add(ctx, InsertTrade{
HistoryOperationID: opCounter,
Order: 0,
CounterAssetID: assets[assetBought.String()].ID,
CounterAccountID: null.IntFrom(accounts[buyer.Address()]),
CounterAmount: amountBought,

BaseAssetID: assets[assetSold.String()].ID,
BaseAccountID: null.IntFrom(accounts[seller.Address()]),
BaseAmount: amountSold,
BaseOfferID: null.IntFrom(int64(trade.OfferId())),
BaseIsSeller: true,
PriceN: int64(price.N),
PriceD: int64(price.D),
LedgerCloseTime: timestamp.ToTime(),
})
err = batch.Exec(ctx)
if err != nil {
return err
}

err = q.RebuildTradeAggregationTimes(context.Background(), timestamp, timestamp)
if err != nil {
return err
}

return nil
}

//PopulateTestTrades generates and ingests trades between two assets according to given parameters
func PopulateTestTrades(
q *Q,
startTs int64,
numOfTrades int,
delta int64,
opStart int64) (ass1 xdr.Asset, ass2 xdr.Asset, err error) {

acc1 := GetTestAccount()
acc2 := GetTestAccount()
ass1 = GetTestAsset("usd")
ass2 = GetTestAsset("euro")
for i := 1; i <= numOfTrades; i++ {
timestamp := stellarTime.MillisFromInt64(startTs + (delta * int64(i-1)))
err = IngestTestTrade(
q, ass1, ass2, acc1, acc2, int64(i*100), int64(i*100)*int64(i), timestamp, opStart+int64(i))
//tt.Assert.NoError(err)
if err != nil {
return
}
}
return
}
18 changes: 12 additions & 6 deletions services/horizon/internal/db2/history/account.go
Original file line number Diff line number Diff line change
@@ -36,7 +36,13 @@ func (q *Q) CreateAccounts(ctx context.Context, addresses []string, batchSize in
// sort assets before inserting rows into history_assets to prevent deadlocks on acquiring a ShareLock
// https://github.com/stellar/go/issues/2370
sort.Strings(addresses)
for _, address := range addresses {
var deduped []string
for i, address := range addresses {
if i > 0 && address == addresses[i-1] {
// skip duplicates
continue
}
deduped = append(deduped, address)
err := builder.Row(ctx, map[string]interface{}{
"address": address,
})
@@ -50,17 +56,17 @@ func (q *Q) CreateAccounts(ctx context.Context, addresses []string, batchSize in
return nil, errors.Wrap(err, "could not exec asset insert builder")
}

var accounts []Account
addressToID := map[string]int64{}
const selectBatchSize = 10000

for i := 0; i < len(addresses); i += selectBatchSize {
for i := 0; i < len(deduped); i += selectBatchSize {
end := i + selectBatchSize
if end > len(addresses) {
end = len(addresses)
if end > len(deduped) {
end = len(deduped)
}
subset := addresses[i:end]
subset := deduped[i:end]

var accounts []Account
if err := q.AccountsByAddresses(ctx, &accounts, subset); err != nil {
return nil, errors.Wrap(err, "could not select accounts")
}
18 changes: 9 additions & 9 deletions services/horizon/internal/db2/history/asset.go
Original file line number Diff line number Diff line change
@@ -55,15 +55,6 @@ func (q *Q) CreateAssets(ctx context.Context, assets []xdr.Asset, batchSize int)
return nil, errors.Wrap(err, "could not extract asset details")
}

err = builder.Row(ctx, map[string]interface{}{
"asset_type": assetType,
"asset_code": assetCode,
"asset_issuer": assetIssuer,
})
if err != nil {
return nil, errors.Wrap(err, "could not insert history_assets row")
}

assetTuple := [3]string{
assetType,
assetCode,
@@ -72,6 +63,15 @@ func (q *Q) CreateAssets(ctx context.Context, assets []xdr.Asset, batchSize int)
if _, contains := assetToKey[assetTuple]; !contains {
searchStrings = append(searchStrings, assetType+"/"+assetCode+"/"+assetIssuer)
assetToKey[assetTuple] = asset.String()

err = builder.Row(ctx, map[string]interface{}{
"asset_type": assetType,
"asset_code": assetCode,
"asset_issuer": assetIssuer,
})
if err != nil {
return nil, errors.Wrap(err, "could not insert history_assets row")
}
}
}

20 changes: 15 additions & 5 deletions services/horizon/internal/db2/history/history_liquidity_pools.go
Original file line number Diff line number Diff line change
@@ -19,6 +19,10 @@ type QHistoryLiquidityPools interface {
// CreateHistoryLiquidityPools creates rows in the history_liquidity_pools table for a given list of ids.
// CreateHistoryLiquidityPools returns a mapping of id to its corresponding internal id in the history_liquidity_pools table
func (q *Q) CreateHistoryLiquidityPools(ctx context.Context, poolIDs []string, batchSize int) (map[string]int64, error) {
if len(poolIDs) == 0 {
return nil, nil
}

builder := &db.BatchInsertBuilder{
Table: q.GetTable("history_liquidity_pools"),
MaxBatchSize: batchSize,
@@ -28,7 +32,13 @@ func (q *Q) CreateHistoryLiquidityPools(ctx context.Context, poolIDs []string, b
// sort before inserting to prevent deadlocks on acquiring a ShareLock
// https://github.com/stellar/go/issues/2370
sort.Strings(poolIDs)
for _, id := range poolIDs {
var deduped []string
for i, id := range poolIDs {
if i > 0 && id == poolIDs[i-1] {
// skip duplicates
continue
}
deduped = append(deduped, id)
err := builder.Row(ctx, map[string]interface{}{
"liquidity_pool_id": id,
})
@@ -46,12 +56,12 @@ func (q *Q) CreateHistoryLiquidityPools(ctx context.Context, poolIDs []string, b
toInternalID := map[string]int64{}
const selectBatchSize = 10000

for i := 0; i < len(poolIDs); i += selectBatchSize {
for i := 0; i < len(deduped); i += selectBatchSize {
end := i + selectBatchSize
if end > len(poolIDs) {
end = len(poolIDs)
if end > len(deduped) {
end = len(deduped)
}
subset := poolIDs[i:end]
subset := deduped[i:end]

lps, err = q.LiquidityPoolsByIDs(ctx, subset)
if err != nil {
Loading