Bug 2226 fast replay streaming txmeta #2343

graydon · 2019-11-15T02:45:29Z

Description

First cut at #2226. This has 3 main pieces:

A new XDR type of high-resolution metadata called LedgerCloseMeta that captures a per-txn, per-ledger-entry change stream that's equivalent to / a superset of the history tables currently being written into the core database for horizon's benefit.
A new "application mode" field in ApplicationImpl that lets us switch-off subsystems and/or replace them with in-memory stubs when running in modes that don't use them.
A new catchup option --replay-in-memory that runs catchup in application mode REPLAY_HISTORY_FOR_METADATA, that has no overlay or database, and uses a "stub" LedgerTxnRoot that returns empty for any query, and a "never committed" pseudo-root LedgerTxn connected to it, that we use as the in-memory ledger.

The idea here is for horizon to write a config file that mentions a local stream and then runs stellar-core as a subprocess, in either bulk replay mode (catchup --replay-in-memory) during initial ingestion or in run mode to track the ongoing changes in the network.

(cc @ire-and-curses and @bartekn)

marta-lokhova · 2019-11-15T21:58:41Z

src/main/Application.h

+ RUN_LIVE_NODE,
+
+ // Application will connect to networks but _not_ establish a database
+ // of any sort, nor participate in consensus, apply transactions or


I believe we want to participate in consensus

I think consensus, or at least voting in it, is an optional aspect of this mode -- in the sense that we could also set the NODE_IS_VALIDATOR flag to false -- but I'll change the comment to reflect that.

marta-lokhova · 2019-11-17T23:46:36Z

src/main/Maintainer.cpp

@@ -28,6 +28,10 @@ Maintainer::start()
 void
 Maintainer::scheduleMaintenance()


Looks like maintenance is fully controlled by Application, would be it cleaner to not initialize it there based on the mode, and assert in Maintainer's constructor that the mode is correct?

Good catch. fixed.

marta-lokhova · 2019-11-18T00:14:55Z

src/ledger/InMemoryLedgerTxnRoot.cpp

+std::shared_ptr<LedgerEntry const>
+InMemoryLedgerTxnRoot::getNewestVersion(LedgerKey const& key) const
+{
+ return nullptr;


Can we add a comment to explain that using the never-committing ledger txn implies that we must have the entire ledger state in memory (unless we disable ledger state loads for certain modes)?

There's some mention in ApplicationImpl.h but I'll expand it a bit.

src/main/Application.cpp

marta-lokhova · 2019-11-18T16:04:42Z

src/main/Application.h

+ // incorrectly.
+ static bool modeHasOverlay(AppMode m);
+ static bool modeHasDatabase(AppMode m);
+ static bool modePublishesHistory(AppMode m);


Do we also need modeWritesToBucketList? I'm happy to add it to my implementation (i disable the bucketlist anyway), but want to make sure we agree on it.

I expect we'll need a few more such mode judgements but I didn't have a clear enough idea of which ones we'd need. I'm going to leave this as-is for now since this PR isn't switching bucket behaviour (yet).

src/main/ApplicationImpl.h

src/main/Application.h

marta-lokhova · 2019-11-18T16:50:12Z

src/history/HistoryManagerImpl.cpp

@@ -222,6 +229,11 @@ HistoryManagerImpl::getMaxLedgerQueuedToPublish()
 bool
 HistoryManagerImpl::maybeQueueHistoryCheckpoint()
 {
+ if (!Application::modePublishesHistory(mApp.getMode()))


what about getMinLedgerQueuedToPublish/getMaxLedgerQueuedToPublish functions? They are in the public interface, and will break if called when DB is not configured.

Good catch, fixed.

src/main/Application.cpp

jonjove · 2019-11-18T16:30:02Z

src/main/Application.h

+ // Gets the current Application mode. This is a constant established
+ // when the Application is constructed and will not change while it
+ // runs.
+ virtual AppMode getMode() const = 0;


From the perspective of designing an interface that is easy to use correctly and hard to use incorrectly, I would prefer an interface with functions like "hasOverlay" over an interface with functions like "getMode". If we can remove this, that would be a good step toward avoiding errors in the future.

hm, we have such interfaces as static helpers (Application::modeHasFoo(AppMode)); I figured it might be helpful to have the flexibility to switch or use the helpers in different contexts (and/or reify the mode as a value you might want to store / pass around). I guess these are hypotheticals though; can change if you feel strongly!

Yeah I agree, the code is littered with Application::modeHasDatabase(mApp.getMode()) .
Also, something like mApp.hasDatabase() is easier to read

fixed in all cases (though delegating to the static methods, which are still there for querying a mode before the application is built).

jonjove · 2019-11-18T16:52:28Z

src/ledger/LedgerTxn.h

@@ -391,6 +391,50 @@ class AbstractLedgerTxnParent
 // or if the corresponding LedgerEntry has been erased.
 virtual std::shared_ptr<LedgerEntry const>
 getNewestVersion(LedgerKey const& key) const = 0;
+
+ virtual uint64_t


These should all be pure-virtual. Given the critical nature of LedgerTxn-type objects, we should make it as hard as possible for anyone implementing a subclass to forget to override functionality. For example, you didn't provide overrides for LedgerTxn even though the default implementations do not apply.

The reason these functions were originally excluded from the AbstractLedgerTxnParent interface is that they weren't needed for LedgerTxn. It should be possible to remove deleteObjectsModifiedOnOrAfterLedger entirely, and I don't think it would be too hard to implement LedgerTxn::countObjects. The drop* functions would probably be tricky to implement. Maybe we can find a way to avoid needing to implement any of these functions?

I'm reasonably comfortable changing them to "implemented as throw on anything other than a root LedgerTxn". Is that ok? They're never used outside of that context, and they seem pointlessly tricky to try to implement "for completeness" on other LedgerTxn subclasses.

jonjove · 2019-11-18T17:44:29Z

src/main/ApplicationImpl.h

+ // commits and answers null to all queries; then a "never-committing"
+ // sub-LedgerTxn is constructed beneath it that serves as the "effective"
+ // in-memory root transaction.
+ std::unique_ptr<InMemoryLedgerTxnRoot> mInMemoryLedgerTxnRoot;


Any reason not to use mLedgerTxnRoot? Looks like you could just change the type of that to std::unique_ptr<AbstractLedgerTxnParent>.

Oh! Wow, I literally did not know unique_ptr would accept an abstract class as its type parameter. Learn something every day. Done!

jonjove · 2019-11-18T19:24:53Z

src/ledger/LedgerManagerImpl.cpp

@@ -888,12 +896,16 @@ LedgerManagerImpl::closeLedger(LedgerCloseData const& ledgerData)
 LedgerTxn ltxUpgrade(ltx);
 Upgrades::applyTo(lupgrade, ltxUpgrade);

- auto ledgerSeq = ltxUpgrade.loadHeader().current().ledgerSeq;
+ LedgerEntryChanges changes = ltxUpgrade.getChanges();


Why even compute this unless there is a database?

see comment below.

jonjove · 2019-11-18T19:26:23Z

src/ledger/LedgerManagerImpl.cpp

-  static_cast<int>(i + 1));
+ if (Application::modeHasDatabase(mApp.getMode()))
+ {
+ auto ledgerSeq = ltxUpgrade.loadHeader().current().ledgerSeq;


This doesn't look right to me. You can't call ltxUpgrade.loadHeader() after calling ltxUpgrade.getChanges() because getChanges seals the LedgerTxn. What motivated you to change the way this was written in the first place?

Motivation shows up in later patch in the PR (00e8866b73abc478a6b3d78cdd09edbde161b551) where we need to have the changes pulled out early (and separately) in order to optionally send them to either or both of the stream and the database.

(That said, you're right it doesn't quite work as written, will fix)

jonjove · 2019-11-18T21:31:55Z

src/ledger/LedgerManagerImpl.cpp

@@ -970,6 +997,45 @@ LedgerManagerImpl::deleteOldEntries(Database& db, uint32_t ledgerSeq,
 txscope.commit();
 }

+void
+LedgerManagerImpl::setupLedgerCloseMetaStream()


I think this is unused.

Oh wow! I don't know how calling that got dropped from the diff, added back!

I guess I need to, uh, make some tests :)

jonjove · 2019-11-18T22:18:57Z

src/ledger/LedgerManagerImpl.h

@@ -67,11 +71,13 @@ class LedgerManagerImpl : public LedgerManager
 CatchupConfiguration::Mode catchupMode);

 void processFeesSeqNums(std::vector<TransactionFramePtr>& txs,
- AbstractLedgerTxn& ltxOuter, int64_t baseFee);
+ AbstractLedgerTxn& ltxOuter, int64_t baseFee,
+ LedgerCloseMeta* ledgerCloseMeta);


Why take the raw-pointer as argument instead of std::unique_ptr<LedgerCloseMeta> const&? Passing the smart-pointer makes ownership questions (eg. do I have to delete this?) much clearer.

Ah, good point. Fixed.

marta-lokhova · 2019-11-19T21:18:32Z

src/herder/TxSetFrame.cpp

@@ -512,6 +513,8 @@ TxSetFrame::getTotalFees(LedgerHeader const& lh) const
 void
 TxSetFrame::toXDR(TransactionSet& txSet)
 {
+ // Should only call toXDR when in sorted-for-hash state.
+ releaseAssertOrThrow(mHashIsValid);


I don't think we can guarantee this currently: some places like StateSnapshot::writeHistoryBlocks, or HerderImpl::persistSCPState convert to XDR without necessarily calculating the hash (which sets mHashIsValid to true)

Mm, ok. I'll call sortForHash explicitly. Sigh. This class is .. kinda a mess.

graydon · 2019-11-27T09:27:45Z

Rebased, addressed all review comments, added test.

graydon · 2019-11-28T01:10:40Z

Repushed with new test that runs replay mode and fixes a couple of the throwing methods on ledgertxn to call through to their parent so they can be used in that mode.

marta-lokhova · 2019-11-27T19:03:13Z

src/main/ExternalQueue.cpp

@@ -25,6 +25,7 @@ string ExternalQueue::kSQLCreateStatement =

 ExternalQueue::ExternalQueue(Application& app) : mApp(app)
 {
+ releaseAssertOrThrow(Application::modeHasDatabase(mApp.getMode()));


Still need to remove checks for database existence from member functions

marta-lokhova · 2019-12-02T17:15:19Z

src/herder/TxSetFrame.cpp

@@ -512,6 +513,7 @@ TxSetFrame::getTotalFees(LedgerHeader const& lh) const
 void
 TxSetFrame::toXDR(TransactionSet& txSet)


It'd be nice to have this function as const (implied by its name and usage). can we explicitly call sortForHash() at the call sites?

Not .. entirely. Depends how much mess I make / clean up in the process. See comment below re: nicolas' question about the same thing.

marta-lokhova · 2019-12-02T17:29:15Z

src/main/ApplicationImpl.cpp

@@ -333,6 +360,15 @@ ApplicationImpl::~ApplicationImpl()
 {
 mProcessManager->shutdown();
 }
+ if (!modeHasDatabase(mAppMode))


Why do we need this?

Because at least in this PR we're treating "no-database" mode as meaning "form buckets while you're running to check hashes, but don't keep buckets around when you shut down". This is debatable.

marta-lokhova · 2019-12-02T18:01:11Z

src/history/HistoryManagerImpl.cpp

@@ -147,6 +148,12 @@ HistoryManagerImpl::logAndUpdatePublishStatus()
 size_t
 HistoryManagerImpl::publishQueueLength() const
 {
+ if (!Application::modePublishesHistory(mApp.getMode()) ||


Only checking Application::modePublishesHistoryis sufficient here.

Same applies to getPublishQueueStates and publishQueuedHistory

switched to !modeHasDatabase() in all cases, removed modePublishesHistory()

marta-lokhova · 2019-12-02T19:06:28Z

src/history/HistoryManagerImpl.cpp

@@ -147,6 +148,12 @@ HistoryManagerImpl::logAndUpdatePublishStatus()
 size_t
 HistoryManagerImpl::publishQueueLength() const


General question on this class: it looks like right now the choice between throwing and simulating function behavior is sort of arbitrary. For example, we allow to "publish" queued history, or to check publish queue length, but throw if we're trying to queue something up. It'd be nice to have consistent behavior across functions (note for the future: maybe at some point we can even not spin up HistoryManager at all when in "publishing disabled" mode, i think it's centrally controlled in LedgerManager anyway)

Unfortunately HistoryManager provides also the interface we use to do ledger-range calculations (based on whether or not we're in accelerated-time mode) and a few other query-history helpers, so there are several dozen call sites even in not-publishing mode. Again, I'm not strictly opposed to eliminating these / trying to move to a no-history-manager mode, but it involves yet more code motion and the PR is already pretty big.

MonsieurNicolas · 2019-12-03T00:34:36Z

src/history/HistoryManagerImpl.cpp

@@ -147,6 +148,12 @@ HistoryManagerImpl::logAndUpdatePublishStatus()
 size_t
 HistoryManagerImpl::publishQueueLength() const
 {
+ if (!Application::modePublishesHistory(mApp.getMode()) ||


Why do we need modePublishesHistory ? If you don't want to publish, just don't add a writeable archive in the first place... Less confusing than having arbitrary settings from the config getting ignored.

As it happens we won't be able to publish history if we don't have a database -- the checkpoints all bounce off a database table that's not replicated in the no-database case, we just don't even store it.

So I could collapse this into modeHasDatabase, but we need to not-even-try to publish history when we're in a no-database mode.

so maybe bomb out on startup if there is a writeable archive but we're configured with no database?

MonsieurNicolas · 2019-12-03T00:42:51Z

src/herder/TxSetFrame.cpp

@@ -512,6 +513,7 @@ TxSetFrame::getTotalFees(LedgerHeader const& lh) const
 void
 TxSetFrame::toXDR(TransactionSet& txSet)
 {
+ sortForHash();


why do we need to sort here? Is it out of paranoia? Call sites seem to call sortForHash already

It is because TxSetFrame and friends are a mess. AbstractTxSetFrame (which is all we have a pointer to in LedgerManagerImpl) does not have a sortForHash method, and shouldn't because SimulationTxSetFrame has only const members (this is a good design I want to keep).

In contrast, SimulationTxSetFrame::toXDR actually constructs, sorts-for-hash, and calls toXDR on a temporary TxSet, which it then throws out. I could theoretically do something like this in TxSetFrame::toXDR as well -- at least the construction-and-sorting part, the delegation has to bottom out with an xdrpp call somewhere! -- and remove or change the other call sites. Not sure what you prefer.

I should also point out that the not-very-well-named TxSetFrame::sortForApply doesn't actually sort-in-place either: it produces a new sorted vector for application. I'd be happy to try moving the sortForHash interface to this sort of shape too, but I wasn't sure how much deeper the pile of yak hair should get in this PR. Happy to go deeper if you like.

Basically TxSetFrame is a mutable/stateful type that's half way through migrating to an immutable/stateless type. I was trying to fit into that existing situation without knocking too many other things over.

What I was asking is: where in the code are you calling toXdr on a tx set that is not already in hash order? I can't find any.

fixed to just assert that it's sorted

MonsieurNicolas · 2019-12-03T01:00:19Z

src/ledger/LedgerCloseMetaStream.cpp

+namespace stellar
+{
+
+LedgerCloseMetaStream::LedgerCloseMetaStream(asio::io_context& ctx,


Do we gain anything from the extra level of buffering to write to the pipe?

For the "rebuild the meta" scenario, the "dumb" implementation would just synchronously write on the main thread (blocking). This would not require any throttling logic, would not do an extra copy of the data into buffers, and the only assumption would be that the reader would consume data as it gets pushed onto the pipe (and as we copy less, we may have a net perf increase).

If it's to make the "online" version work, it's probably premature as this design doesn't handle failures anyways (and this PR doesn't even deal with shutdown properly if not run with catchup).

MonsieurNicolas · 2019-12-03T01:13:20Z

src/ledger/test/LedgerCloseMetaStreamTests.cpp

+
+using namespace stellar;
+
+#ifndef _WIN32


Is there a reason that those tests are not written cross platform?

I initially thought I'd be using anonymous pipe() rather than a file-backed fd. I suppose the file-backed fd should work on windows too. I'll give it a try.

windows tests work now too. and confirmed named pipes still work.

MonsieurNicolas · 2019-12-03T01:15:50Z

src/ledger/test/LedgerCloseMetaStreamTests.cpp

+ clock.crank(true);
+ }
+ Hash hash = app->getLedgerManager().getLastClosedLedgerHeader().hash;
+ while (!app->getLedgerManager().metadataBufferEmpty())


you should not need to wait for the meta buffer to be empty (that seems to be missing logic in the app shutdown code: the "wrapper process" would not be able to do this) - regardless, I don't think we should support LIVE_NODE in this PR as there are too many unknowns around it

Fixed in process of making it synchronous.

MonsieurNicolas · 2019-12-03T01:20:26Z

src/main/CommandLine.cpp

@@ -1024,6 +1050,9 @@ handleCommandLine(int argc, char* const* argv)
 runSignTransaction},
 {"upgrade-db", "upgade database schema to current version",
 runUpgradeDB},
+ {"replay-history-for-metadata",


this should be an option for the catchup sub-command instead of being exposed at the top level like this

Heh. The comment I put in for modes specifically says it's for when cases where it does something meaningfully different than an existing subcommand. I think in this case it really does! catchup modifies local state; replay-history-for-metadata is stateless, idempotent. From the user's perspective it's quite different than catchup; it just happens to do catchup internally. No?

fixed, made into a catchup argument --replay-in-memory.

MonsieurNicolas · 2019-12-03T01:22:07Z

src/main/Config.cpp

+ if (METADATA_OUTPUT_FILE_DESCRIPTOR != -1 &&
+ METADATA_OUTPUT_NAMED_PIPE != "")
+ {
+ LOG(FATAL) << "Can only set one of METADATA_OUTPUT_NAMED_PIPE and"


we could also only have one setting that can be either a path or an integer?

I think this is not ideal: it's weird to the user (precludes integer-valued filenames, granted, unlikely) and requires weird implementation (a config-value visitor I guess? something that precludes accessing the string as a string when it's "really" an integer..)

Fixed: went with "pathname which can be a pipe or a file, or 'fd:N' which opens the N file descriptor". User can deal with living without files named "fd:N".

graydon · 2019-12-05T06:15:55Z

Ok I believe this now addresses all review comments so far. Need to squash & rebase but it's (thankfully) getting quite small!

graydon · 2019-12-05T21:21:15Z

Squashed and formatted. Running tests now. Ought to be nearly ready.

MonsieurNicolas · 2019-12-05T21:48:17Z

docs/stellar-core_example.cfg

+# existing open file descriptor N inherited by the process (for
+# example to write to an anonymous pipe). Anonymous file descriptor
+# streams are not supported on Windows (anonymous pipes do not work
+# with the ASIO library used in stellar-core).


I think the asio comment is not accurate anymore

MonsieurNicolas · 2019-12-05T21:53:18Z

src/herder/TxSetFrame.cpp

@@ -512,6 +513,7 @@ TxSetFrame::getTotalFees(LedgerHeader const& lh) const
 void
 TxSetFrame::toXDR(TransactionSet& txSet)
 {
+ sortForHash();


What I was asking is: where in the code are you calling toXdr on a tx set that is not already in hash order? I can't find any.

MonsieurNicolas · 2019-12-05T21:55:37Z

src/history/HistoryManagerImpl.cpp

@@ -222,6 +230,11 @@ HistoryManagerImpl::getMaxLedgerQueuedToPublish()
 bool
 HistoryManagerImpl::maybeQueueHistoryCheckpoint()
 {
+ if (!mApp.modeHasDatabase())


there is already the check below that looks at hasAnyWritableHistoryArchive

MonsieurNicolas · 2019-12-05T22:18:11Z

src/history/HistoryManagerImpl.cpp

@@ -317,6 +333,11 @@ HistoryManagerImpl::publishQueuedHistory()
 }
 #endif

+ if (!mApp.modeHasDatabase())


meta question here...

I think we could simplify (and remove more of those if (!mApp.modeHasDatabase())) calls:
what if we were instead setting up an in memory sqlite database, that way code that deals with tables that are low churn like PersistentState would not have to be changed.
We could also not create the expensive tables at all, but we can also just check in tests that we don't populate them (and a bug in that front would be trivial to spot as we would have large memory usage).

Possible! But will defer this question to a later PR as discussed in chat.

MonsieurNicolas · 2019-12-05T22:20:19Z

src/main/ApplicationImpl.cpp

+ // Note: BucketManager::dropAll will delete the $BUCKETDIR/tmp which is
+ // where process and history manager write _their_ temp files to, so we
+ // make sure to tear them down first here.
+ mProcessManager = nullptr;


should this be done after joinAllThreads below?

oh, good catch. sure, fixed.

…ble archives.

MonsieurNicolas · 2019-12-06T19:36:20Z

r+ 6eff08a

…txmeta Bug 2226 fast replay streaming txmeta Reviewed-by: MonsieurNicolas

graydon requested review from MonsieurNicolas, marta-lokhova and jonjove November 15, 2019 02:45

marta-lokhova reviewed Nov 18, 2019

View reviewed changes

jonjove reviewed Nov 18, 2019

View reviewed changes

marta-lokhova reviewed Nov 19, 2019

View reviewed changes

marta-lokhova mentioned this pull request Nov 19, 2019

Skip tx application and validation for simulation purposes #2348

Merged

graydon force-pushed the bug-2226-fast-replay-streaming-txmeta branch 2 times, most recently from 5f4954e to 2fff91e Compare November 27, 2019 09:15

graydon force-pushed the bug-2226-fast-replay-streaming-txmeta branch from 2fff91e to f44257d Compare November 28, 2019 00:30

marta-lokhova reviewed Dec 2, 2019

View reviewed changes

MonsieurNicolas requested changes Dec 3, 2019

View reviewed changes

graydon force-pushed the bug-2226-fast-replay-streaming-txmeta branch from 8352181 to d107e08 Compare December 5, 2019 21:19

graydon force-pushed the bug-2226-fast-replay-streaming-txmeta branch from d107e08 to b41b6cd Compare December 5, 2019 21:51

MonsieurNicolas requested changes Dec 5, 2019

View reviewed changes

graydon added 11 commits December 6, 2019 11:19

Make some static constants 'static const'.

ea1f77b

Add releaseAssertOr{Abort,Throw} release-assert helpers.

460919c

Add Application::AppMode and use in Application implementations.

641b4d7

Allow use of different AbstractLedgerTxnParent types as LedgerTxnRoot.

b574704

Add InMemoryLedgerTxnRoot.{cpp,h}

53ba379

Use InMemoryLedgerTxnRoot + never-committed LedgerTxn for replay.

f943ed7

Delete buckets on shutdown when !modeHasDatabase

2bdc933

Inhibit storing transactions and ledgers when !modeHasDatabase.

0092487

Virtualize AbstractTxSetFrame::toXDR, assert txs sorted.

aa13445

Add XDROutputFileStream::{fdopen,flush}.

d629224

Stream LedgerCloseMeta to XDR output stream.

acb4844

graydon added 9 commits December 6, 2019 11:20

Inhibit publishing when !modeHasDatabase.

f2d04d7

Store persistentState in memory when !modeHasDatabase.

a826dab

Inhibit ExternalQueue and Maintenance when !modeHasDatabase

fa8c564

Inhibit constructing database entirely when !modeHasDatabase.

5d37c56

Add '--replay-in-memory' option to catchup subcommand.

805e0ba

Add files to VS project files.

145a9a7

Guard !modeHasDatabase startup against NODE_IS_VALIDATOR or any writa…

fb573d0

…ble archives.

Guard call to deleteObjectsModifiedOnOrAfter by modeHasDatabase()

3e42339

Add basic round trip test of LedgerCloseMetaStream

6eff08a

graydon force-pushed the bug-2226-fast-replay-streaming-txmeta branch from b41b6cd to 6eff08a Compare December 6, 2019 19:21

latobarita added a commit that referenced this pull request Dec 6, 2019

Merge pull request #2343 from graydon/bug-2226-fast-replay-streaming-…

cd66d85

…txmeta Bug 2226 fast replay streaming txmeta Reviewed-by: MonsieurNicolas

latobarita merged commit 6eff08a into stellar:master Dec 6, 2019

graydon deleted the bug-2226-fast-replay-streaming-txmeta branch January 3, 2020 01:29

nebolsin mentioned this pull request Feb 2, 2020

Switch to fast replay mode introduced in stellar-core 12.3.0 astroband/astrologer#20

Open

		@@ -28,6 +28,10 @@ Maintainer::start()
		void
		Maintainer::scheduleMaintenance()

		@@ -512,6 +513,7 @@ TxSetFrame::getTotalFees(LedgerHeader const& lh) const
		void
		TxSetFrame::toXDR(TransactionSet& txSet)

		@@ -147,6 +148,12 @@ HistoryManagerImpl::logAndUpdatePublishStatus()
		size_t
		HistoryManagerImpl::publishQueueLength() const

Bug 2226 fast replay streaming txmeta #2343

Bug 2226 fast replay streaming txmeta #2343

Conversation

graydon commented Nov 15, 2019 • edited Loading

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

graydon commented Nov 27, 2019

graydon commented Nov 28, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

graydon commented Dec 5, 2019

graydon commented Dec 5, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

graydon commented Nov 15, 2019 •

edited

Loading