Code

E2 snapshot uploading#9056

Code

Merged

mh0lt merged 78 commits intodevelerigontech/erigon:develfrom e2_snapshot_uploadingerigontech/erigon:e2_snapshot_uploadingCopy head branch name to clipboardDec 27, 2023

+6,593-673

Commits

Commits on Oct 19, 2023

• working upload of mumbai history

  mh0lt

  committed
  4143702

  Copy full SHA for 4143702

Commits on Oct 21, 2023

• added min block processing

  mh0lt

  committed
  77a8f06

  Copy full SHA for 77a8f06

Commits on Oct 22, 2023

• retrieve initial torrent hashes from remote

  mh0lt

  committed
  cbd4625

  Copy full SHA for cbd4625

• Moved setting of segment min earlier in the initialization process

  mh0lt

  committed
  21a4b72

  Copy full SHA for 21a4b72

• base progress on headers - they are alost slower than bor

  mh0lt

  committed
  af3b181

  Copy full SHA for af3b181

Commits on Oct 23, 2023

• added uplo9ad running check

  mh0lt

  committed
  c6e376b

  Copy full SHA for c6e376b

• Fix panic in isAgra check for txpool

  mh0lt

  committed
  d6e6055

  Copy full SHA for d6e6055

Commits on Oct 26, 2023

• fixes for eth based chains

  mh0lt

  committed
  7bd8278

  Copy full SHA for 7bd8278

Commits on Oct 27, 2023

• Merge branch 'devel' of https://github.com/ledgerwatch/erigon into devel

  mh0lt

  committed
  df46494

  Copy full SHA for df46494

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  e276ce9

  Copy full SHA for e276ce9

Commits on Oct 28, 2023

• apped parallel transaction parsing to tx dump

  mh0lt

  committed
  a84012a

  Copy full SHA for a84012a

Commits on Oct 29, 2023

• make service start continue on error if more to do

  mh0lt

  committed
  fef6cbc

  Copy full SHA for fef6cbc

• Fix unknown file/permission denied errors on rename

  mh0lt

  committed
  6240732

  Copy full SHA for 6240732

• add logger

  mh0lt

  committed
  aed0383

  Copy full SHA for aed0383

• warn on error

  mh0lt

  committed
  c185f74

  Copy full SHA for c185f74

• ajusted logging

  mh0lt

  committed
  3981bae

  Copy full SHA for 3981bae

• don't check missing indexes while still retiring blocks

  mh0lt

  committed
  423a7dc

  Copy full SHA for 423a7dc

• remove created log

  mh0lt

  committed
  e0de98f

  Copy full SHA for e0de98f

Commits on Oct 30, 2023

• Added pos transition processing

  mh0lt

  committed
  ec64427

  Copy full SHA for ec64427

Commits on Nov 7, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  fe8a401

  Copy full SHA for fe8a401

• merge edits

  mh0lt

  committed
  d1b49ff

  Copy full SHA for d1b49ff

Commits on Nov 17, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  dbfec78

  Copy full SHA for dbfec78

Commits on Nov 21, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  5a568c1

  Copy full SHA for 5a568c1

Commits on Nov 26, 2023

• added snapshotversion to snapshot file readers

  mh0lt

  committed
  545a235

  Copy full SHA for 545a235

Commits on Nov 29, 2023

• running snapshot cmp

  mh0lt

  committed
  8def1b6

  Copy full SHA for 8def1b6

Commits on Dec 2, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  525336a

  Copy full SHA for 525336a

• use downloader rclone for uploading

  mh0lt

  committed
  58b896b

  Copy full SHA for 58b896b

Commits on Dec 5, 2023

• added copy operation and latest block download

  mh0lt

  committed
  756dd57

  Copy full SHA for 756dd57

Commits on Dec 6, 2023

• added last file download and fixed downloding version issues

  mh0lt

  committed
  65354e0

  Copy full SHA for 65354e0

Commits on Dec 7, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  42f4fac

  Copy full SHA for 42f4fac

• added headers to pipeline

  mh0lt

  committed
  4bc2f92

  Copy full SHA for 4bc2f92

• fixed bor snapshot versioning

  mh0lt

  committed
  9c24bf6

  Copy full SHA for 9c24bf6

Commits on Dec 8, 2023

• fixes for bor snapshot processing at startup

  mh0lt

  committed
  1b8cfe8

  Copy full SHA for 1b8cfe8

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  2f76f5e

  Copy full SHA for 2f76f5e

Commits on Dec 9, 2023

• added manifest processing

  mh0lt

  committed
  9ac14ac

  Copy full SHA for 9ac14ac

Commits on Dec 11, 2023

• add update all flag

  mh0lt

  committed
  c63c00c

  Copy full SHA for c63c00c

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  ea58df4

  Copy full SHA for ea58df4

Commits on Dec 12, 2023

• added log labels for stats and fixed missing removes of bor segments on merge

  mh0lt

  committed
  77fda8c

  Copy full SHA for 77fda8c

• fix bor merge limit

  mh0lt

  committed
  c4d5791

  Copy full SHA for c4d5791

Commits on Dec 13, 2023

• added manifest update fix and worker setting for upload

  mh0lt

  committed
  8010607

  Copy full SHA for 8010607

Commits on Dec 15, 2023

• updated header check to avoid overruns

  mh0lt

  committed
  02da5be

  Copy full SHA for 02da5be

• fix locator parsing

  mh0lt

  committed
  bfd3bfe

  Copy full SHA for bfd3bfe

Commits on Dec 16, 2023

• fixed issues added verify

  mh0lt

  committed
  1bc47fe

  Copy full SHA for 1bc47fe

• remove printer

  mh0lt

  committed
  b960953

  Copy full SHA for b960953

• updated cmp & copy

  mh0lt

  committed
  d18c8d9

  Copy full SHA for d18c8d9

• added more parallel execution

  mh0lt

  committed
  986019a

  Copy full SHA for 986019a

Commits on Dec 17, 2023

• fixed logging initialization

  mh0lt

  committed
  72b4ae9

  Copy full SHA for 72b4ae9

• remove flags from main

  mh0lt

  committed
  5dd3025

  Copy full SHA for 5dd3025

• remove --snapshot.version - use KnownCfg instead

  mh0lt

  committed
  33fe185

  Copy full SHA for 33fe185

• only update pv's on override

  mh0lt

  committed
  7785008

  Copy full SHA for 7785008

Commits on Dec 18, 2023

• update experimental values to flags for stage loop control

  mh0lt

  committed
  79606b1

  Copy full SHA for 79606b1

Commits on Dec 19, 2023

• fix init snapshot configs

  mh0lt

  committed
  521880e

  Copy full SHA for 521880e

• use correct flag for prune

  mh0lt

  committed
  81fadd0

  Copy full SHA for 81fadd0

Commits on Dec 20, 2023

• ask known config for specific version

  mh0lt

  committed
  d5b883d

  Copy full SHA for d5b883d

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  e232428

  Copy full SHA for e232428

Commits on Dec 21, 2023

• post merge updates

  mh0lt

  committed
  10f8506

  Copy full SHA for 10f8506

• added readmes

  mh0lt

  committed
  eb9a2b3

  Copy full SHA for eb9a2b3

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  1c05790

  Copy full SHA for 1c05790

• fix torrent naming changes

  mh0lt

  committed
  0f9e1a3

  Copy full SHA for 0f9e1a3

• change prune/retire ordering so retire is activated during pruning

  mh0lt

  committed
  d46d6f8

  Copy full SHA for d46d6f8

Commits on Dec 22, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  966bbab

  Copy full SHA for 966bbab

• add disk options config

  mh0lt

  committed
  1adf7da

  Copy full SHA for 1adf7da

• typos

  mh0lt

  committed
  9c44689

  Copy full SHA for 9c44689

• added upload to local fs

  mh0lt

  committed
  1172e6e

  Copy full SHA for 1172e6e

• change _group to group for rclone stats

  mh0lt

  committed
  61d05b3

  Copy full SHA for 61d05b3

Commits on Dec 23, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  c97bfda

  Copy full SHA for c97bfda

• added merge fixes

  mh0lt

  committed
  39b1443

  Copy full SHA for 39b1443

• fix retire loop conditions

  mh0lt

  committed
  b01697c

  Copy full SHA for b01697c

• fix slices package dependency

  mh0lt

  committed
  89672a2

  Copy full SHA for 89672a2

• fix lint errors

  mh0lt

  committed
  3010175

  Copy full SHA for 3010175

• fix lint errors

  mh0lt

  committed
  19e0a6a

  Copy full SHA for 19e0a6a

• fix lint errors

  mh0lt

  committed
  17225f4

  Copy full SHA for 17225f4

• fix lint errors

  mh0lt

  committed
  6f7a953

  Copy full SHA for 6f7a953

Commits on Dec 24, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  cf6b1d2

  Copy full SHA for cf6b1d2

• post merge fixes

  mh0lt

  committed
  7b7dd4c

  Copy full SHA for 7b7dd4c

Commits on Dec 27, 2023

• Merge branch 'devel' into e2_snapshot_uploading

  mh0lt

  committed
  bc565e0

  Copy full SHA for bc565e0

• fixed compile errors

  mh0lt

  committed
  b29c330

  Copy full SHA for b29c330

• updated sync simulator

  mh0lt

  committed
  426e30a

  Copy full SHA for 426e30a