Merge release-horizon v0.23.0 into master #1946
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merge `master` into `release-horizon-v0.22.0`
…#1794) This commit adds trust lines to ingestion pipeline. In #1765, #1766 and #442 we need trust lines in Horizon DB to serve the requests. This commit is adding a code needed by all of the issues above. List of changes: * Adds a new migration file adding a new `trust_lines` table. * Updates `db2/history` package with trust lines related interfaces and structs. * Updates `DatabaseProcessor` to process trust lines in state and ledger pipelines.
…1828) * Remove deprecated fee_paid field from Transaction * Update services/horizon/CHANGELOG.md Co-Authored-By: Bartek Nowotarski <bartek@nowotarski.info>
…ns system. (#1827) * Add new action handler for accounts. * Use GetAccountsHandler and remove old actions code. * Remove unused functions. * DisableCursorValidation. * Disable cursor validation when building a new page. * Fix call to makeRequest. * Remove newline between err and if.
…in experimental ingestion endpoints (#1830) Include a Latest-Ledger header with the sequence number of the last processed ledger by the experimental ingestion system. Also, ensure the endpoint responses contains only data consistent with the sequence number included in the header
…#1819) This commit adds accounts and account data to ingestion pipeline. Summary of changes: * Adds a new migration file adding a new `accounts` and `accounts_data` tables. * Updates `db2/history` package with accounts and data related interfaces and structs. * Updates `DatabaseProcessor` to process accounts and data in state and ledger pipelines. * Changes `/exp/ingest/io.Change` to contain `LedgerEntry` instead of `LedgerEntryData` to support cases where stellar-core generates change in the meta but the entry does not actually change.
…sset stats endpoint (#1824)
This commit fixes fee stats query to use Postgres' percentile_disc instead of percentile_cont.
…tion readiness check (#1847)
There is a bug in `consumeOffersForBuyingAsset()` which manifests when `currentAssetAmount` is not high enough to purchase the selling asset. In such cases `consumeOffersForBuyingAsset()` should return -1, nil but because of the bug the function returns a positive amount with no error
Use https://www.gorillatoolkit.org/pkg/schema and https://github.com/asaskevich/govalidator to automate parsing and validation of query parameters.
This commit sets `referer` value to `undefined` if `Referer` HTTP header is empty. This is to help support logging in system that have problems parsing empty values.
To prevent WriteHeader from being called multiple times we have created a timeout middleware which checks that the response status hasn't already been written before setting the status to http.StatusGatewayTimeout
… to an asset. (#1835) * Extend /accounts to return accounts who trust a given asset. * Return account entries. * Add failing test for relationship data. * Add GetAccountDataByAccountsID. * Add GetTrustLinesByAccountsID. * Load account_data for records. * Load trustlines for records. * Load signers for records. * Return early if there are no records. * Add test for AccountsForAsset. * Fix test for trustlines. * Fix comment. * Flip if else. * Get historyQ from context. * Rename var to accountIDs. * Move loaders to GetAccountsHandler. * Rely on zero value for maps. * Wrap errors. * Use tt.Assert. * Add test for cursor. * Add test for trustlines results. * Use query structs for accounts. * Use query struct for asset. * Remove Q from handler. * Remove unnecessary check. * Update CHANGELOG.
…igners" end-point. (#1876)
This commits adds support for getting ledger upgrade changes from `exp/ingest/io.LedgerReader` and updates ledger processors in Horizon to process ledger upgrade meta. As described in Integration [1] doc in stellar-core the ledger meta changes need to be processed in the following order: * transaction set "meta" (fee, txmeta) * ledger upgrades "meta". This commit adds missing ledger upgrade processing by adding a new method on `LedgerReader` that streams `io.Change` objects connected to a given ledger upgrades. Because processing ledger upgrade meta is essential to update the state correctly, `LedgerReader.Close()` will return an error in case some ledger upgrades have not been read. ### Summary of changes * Added `ReadUpgradeChange` to `io.LedgerReader`. * Updated pipeline wrappers to pass reference to upgrades to the next reader in the pipeline. * Updated Horizon processors to process ledger upgrade meta. ### Known limitations & issues `LedgerEntryChanges` connected to ledger upgrade can be of arbitrary size. Currently all ledger upgrade changes are read fully into memory because they are stored in a single field in stellar-core DB (streaming them would require an awkward code that calls `substr` on an upgrade DB row). This should be solved when stellar-core ledger protocol is finished. `LedgerReader` API is designed to support streaming upgrade changes. [1] https://github.com/stellar/stellar-core/blob/master/docs/integration.md#applying-changes-in-the-high-resolution-custom-view
…ructs. (#1883) * Automate URI template generation with query params. * Use accounts URITemplate in root. * Add URITemplate to OffersQuery. * Use URITemplate for offers root URL.
…1868) This commit adds temporary code (to be removed in one of the future versions of Horizon) that compares account state in Stellar-Core and Horizon DB. If a difference is found it's logged with `WARNING` level.
This commit is changing column names in tables used by expingest package to be consistent.
…1891) * Add validator for canonical representation for assets. * Update asset filter on accounts. * Update docs.
…use query structs. (#1897) * Add validator for amount. * Export actions.ValidateAssetParams. * Add query struct for FindFixedPathsHandler. * Add asset.BuildAssets * Use xdr.BuildAssets * use xdr.BuildAssets. * Export getURIParams. * Add URITemplate tp FindFixedPathsQuery. * Add strict send paths to root url. * Mark destination assets as optional * Create invalid field error.
…to use query struct. (#1901) * Use query-struct in strict receive paths search. * Use strict receive query params to generate root url.
…g in /assets endpoint (#1930)
Remove the extra serve mux http handler layer between the http.Server and the chi.Router, which itself is a http.Handler. The chi.Router is a handler, and we don't need to wrap it in another serve mux, whether it be the default or our own one. I noticed in #1935 when we replaced the default serve mux with our own one that both are unnecessary.
…ed trustlines when updating asset stats (#1936) The experimental ingestion pipeline has a bug in that it counts unauthorized trustlines when computing asset stats. Only authorized trustlines should be included. Another issue fixed in this commit is that the cursor for the new asset stats endpoint diverges from the existing asset stats endpoint
Bumps `expingest.CurrentVersion` to recreate state due to a bug fixed in #1936. We need to reingest existing state due to a rules change connected to trust lines authorization.
…te (#1940) Updates `ExperimentalIngestionTables` with `exp_asset_stats` that contains asset stats populated by experimental ingestion pipeline. When `expingest` rebuilds state it truncates state tables to start from scratch. One of the new tables introduced in 0.23.0 were not truncated what resulted in: `duplicate key value violates unique constraint` error.
We should be sending a bye bye event before closing an SSE request. But we neglect to do so when the StreamHandler context terminates the request.
|
Merging without a review as all of the PRs here have been reviewed and Horizon release has been tested. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.