You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Most of the Horizon ingestion uses a pattern where a bunch of rows are inserted into a table using a BatchInsertBuilder. For example, the TransactionProcessor uses a TransactionBatchInsertBuilder to record all the transactions for a given ledger in the history_transactions table in the Horizon DB.
TransactionBatchInsertBuilder is a wrapper around BatchInsertBuilder which ultimately calls Row() to append a transaction to the current batch:
// Row adds a new row to the batch. All rows must have exactly the same columns// (map keys). Otherwise, error will be returned. Please note that rows are not// added one by one but in batches when `Exec` is called (or `MaxBatchSize` is// reached).func (b*BatchInsertBuilder) Row(rowmap[string]interface{}) error
map[string]interface{} isn't an ideal representation for a row because there aren't any guarantees provided by the type system that two map[string]interface{} values have the same length and the same set of keys.
In #2562 , we had to fix an issue where a TransactionBatchInsertBuilder was calling Row() on two map[string]interface{} values which were not consistent with each other. This manifested in a runtime error which blocked ingestion.
If we can pass history.Transaction instances to Row() instead of a map[string]interface{} values, we can know that all row values will have the same number of columns and the same set of column names. This would make ingestion more robust to the kind of bug we had to fix in #2562 .
If we use the following signature func (b *BatchInsertBuilder) Row(row interface{}) error we can use reflection to extract all the columns by looking at all struct fields with the db tag.
The text was updated successfully, but these errors were encountered:
Most of the Horizon ingestion uses a pattern where a bunch of rows are inserted into a table using a
BatchInsertBuilder
. For example, theTransactionProcessor
uses aTransactionBatchInsertBuilder
to record all the transactions for a given ledger in thehistory_transactions
table in the Horizon DB.TransactionBatchInsertBuilder
is a wrapper aroundBatchInsertBuilder
which ultimately callsRow()
to append a transaction to the current batch:map[string]interface{}
isn't an ideal representation for a row because there aren't any guarantees provided by the type system that twomap[string]interface{}
values have the same length and the same set of keys.In #2562 , we had to fix an issue where a
TransactionBatchInsertBuilder
was callingRow()
on twomap[string]interface{}
values which were not consistent with each other. This manifested in a runtime error which blocked ingestion.If we can pass
history.Transaction
instances toRow()
instead of amap[string]interface{}
values, we can know that all row values will have the same number of columns and the same set of column names. This would make ingestion more robust to the kind of bug we had to fix in #2562 .If we use the following signature
func (b *BatchInsertBuilder) Row(row interface{}) error
we can use reflection to extract all the columns by looking at all struct fields with thedb
tag.The text was updated successfully, but these errors were encountered: