Merge Captive core into 1.5.0 release branch #2695
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is an initial sketch of support for an ingestion backend driven by a captive stellar-core subprocess replaying ledgers in memory. It's going to need a bunch of iteration before it's viable, but this PR should serve as a place to examine & discuss the approach, fix bugs, etc.
…2609) * Add configuration for captive core ingestion * Use captive core path configuration when initializing ingestion system
This commit refactors `captiveStellarCore` ledger backend and adds tests. `stellarCoreRunner` has been extracted from `captiveStellarCore` to allow mocking stellar-core runs. This makes it possible to write tests that do not execute `stellar-core` binary. Functions marshalling and unmarshalling "framed" XDR have been moved to `xdr` package.
…g them (#2672) In macOS, captive-core blocks when writing to a pipe which isn't read from.
Also, upgrade the container to ubuntu 18.04 (16.04 didn't have a compatible libstdc++ library)
* Rename `DBLedgerReader` to `LedgerTransactionReader` because it works with any ledger backend. * `LedgerTransactionReader` gets `LedgerCloseMeta` from a provided ledger backend. * `LedgerChangeReader` has been refactored and now embeds `LedgerTransactionReader`. * Updated `GetChangesFromLedgerEntryChanges` to be a public function. * Removed `ctx.Context` from readers because it's not idiomatic. * Removed unused interfaces and mocks. Part of #2187.
tamirms
approved these changes
Jun 16, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add experimental support for database reingestion using a Stellar Core subprocess instead of a persistent Stellar Core database (#2695).
Stellar Core v12.3.0 added an experimental feature which allows replaying legder's metadata in-memory. This feature allow us to speed up reingestion and removed the dependency between Stellar Core's database and Horizon.
For now this is only supported while running
horizon db reingest. To try out this new experimental feature you need to specify the following parameters:--enable-captive-core-ingestionorENABLE_CAPTIVE_CORE_INGESTION=true.--stellar-core-binary-pathorSTELLAR_CORE_BINARY_PATH.