feat: add db migration/patch tool #1908
Conversation
JayT106
requested review from
randy-cro and
thomas-nguy
and removed request for
a team
This comment has been minimized.
This comment has been minimized.
WalkthroughAdd a top-level Changes
Sequence Diagram(s)sequenceDiagram
participant User as CLI User
participant DBCmd as database cmd
participant MigrateCLI as migrate cmd
participant MigrateEngine as dbmigrate.Migrate
participant Src as Source DB
participant Tgt as Target DB
User->>DBCmd: database migrate --source-home ... --target-home ...
DBCmd->>MigrateCLI: validate & forward flags
MigrateCLI->>MigrateEngine: Migrate(MigrateOptions)
MigrateEngine->>Src: open source DB
MigrateEngine->>Tgt: create/open target DB
rect rgb(230,245,255)
loop batched transfer
MigrateEngine->>Src: read batch
MigrateEngine->>Tgt: write batch
MigrateEngine->>MigrateEngine: update stats/log
end
end
opt verify enabled
MigrateEngine->>Tgt: reopen & verify keys
end
MigrateEngine-->>MigrateCLI: return stats
MigrateCLI-->>User: print summary
sequenceDiagram
participant User as CLI User
participant PatchCLI as patch cmd
participant Height as ParseHeightFlag
participant PatchEngine as dbmigrate.PatchDatabase
participant Src as Source DB
participant Tgt as Target DB
User->>PatchCLI: database patch --height 100-200 --database blockstore
PatchCLI->>Height: ParseHeightFlag("100-200")
Height-->>PatchCLI: HeightRange{Start:100,End:200}
PatchCLI->>PatchEngine: PatchDatabase(PatchOptions)
PatchEngine->>Src: open source DB
PatchEngine->>Tgt: open target DB
rect rgb(235,255,235)
loop for height-filtered keys
PatchEngine->>Src: iterate keys
alt key exists in target
PatchEngine->>PatchEngine: resolve conflict (prompt/strategy)
else
PatchEngine->>Tgt: write key (or dry-run)
end
end
end
PatchEngine-->>PatchCLI: return stats
PatchCLI-->>User: print summary
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Files/areas needing extra attention:
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (24)
Makefile(1 hunks)cmd/cronosd/cmd/database.go(1 hunks)cmd/cronosd/cmd/migrate_db.go(1 hunks)cmd/cronosd/cmd/migrate_db_test.go(1 hunks)cmd/cronosd/cmd/patch_db.go(1 hunks)cmd/cronosd/cmd/root.go(1 hunks)cmd/cronosd/dbmigrate/QUICKSTART.md(1 hunks)cmd/cronosd/dbmigrate/README.md(1 hunks)cmd/cronosd/dbmigrate/height_filter.go(1 hunks)cmd/cronosd/dbmigrate/height_filter_test.go(1 hunks)cmd/cronosd/dbmigrate/height_parse_test.go(1 hunks)cmd/cronosd/dbmigrate/migrate.go(1 hunks)cmd/cronosd/dbmigrate/migrate_basic_test.go(1 hunks)cmd/cronosd/dbmigrate/migrate_dbname_test.go(1 hunks)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go(1 hunks)cmd/cronosd/dbmigrate/migrate_rocksdb.go(1 hunks)cmd/cronosd/dbmigrate/migrate_rocksdb_test.go(1 hunks)cmd/cronosd/dbmigrate/migrate_test.go(1 hunks)cmd/cronosd/dbmigrate/patch.go(1 hunks)cmd/cronosd/dbmigrate/patch_test.go(1 hunks)cmd/cronosd/dbmigrate/swap-migrated-db.sh(1 hunks)x/cronos/keeper/keeper.go(3 hunks)x/cronos/rpc/api.go(2 hunks)x/e2ee/client/cli/encrypt.go(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2024-10-12T22:09:46.096Z
Learnt from: yihuang
Repo: crypto-org-chain/cronos PR: 1618
File: memiavl/db_test.go:199-199
Timestamp: 2024-10-12T22:09:46.096Z
Learning: In unit tests within `memiavl/db_test.go`, converting `int64` to `uint32` is acceptable.
Applied to files:
cmd/cronosd/dbmigrate/migrate_basic_test.gocmd/cronosd/dbmigrate/migrate_test.gocmd/cronosd/dbmigrate/migrate_dbname_test.gocmd/cronosd/dbmigrate/patch_test.gocmd/cronosd/dbmigrate/height_filter_test.gocmd/cronosd/dbmigrate/migrate_rocksdb_test.go
🧬 Code graph analysis (15)
cmd/cronosd/dbmigrate/migrate_rocksdb.go (2)
cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/opendb/opendb_rocksdb.go (1)
NewRocksdbOptions(88-138)
cmd/cronosd/cmd/database.go (2)
cmd/cronosd/cmd/migrate_db.go (1)
MigrateCmd(301-306)cmd/cronosd/cmd/patch_db.go (1)
PatchCmd(306-311)
cmd/cronosd/cmd/migrate_db_test.go (1)
cmd/cronosd/cmd/migrate_db.go (3)
DBTypeApp(28-28)DBTypeCometBFT(29-29)DBTypeAll(30-30)
cmd/cronosd/dbmigrate/height_parse_test.go (1)
cmd/cronosd/dbmigrate/height_filter.go (2)
HeightRange(20-24)ParseHeightFlag(124-151)
cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(14-17)
cmd/cronosd/dbmigrate/migrate_basic_test.go (2)
cmd/cronosd/dbmigrate/migrate_test.go (7)
TestCountKeys(60-88)TestMigrationStats(91-106)TestMigrateLargeDatabase(109-140)TestMigrateEmptyDatabase(143-167)TestMigrationWithoutVerification(170-196)TestMigrationBatchSizes(199-229)TestMigrateSpecialKeys(334-423)cmd/cronosd/dbmigrate/migrate.go (3)
MigrateOptions(23-45)Migrate(74-233)MigrationStats(48-54)
cmd/cronosd/dbmigrate/migrate_test.go (2)
cmd/cronosd/dbmigrate/migrate_basic_test.go (7)
TestCountKeys(41-69)TestMigrationStats(102-117)TestMigrateLargeDatabase(120-151)TestMigrateEmptyDatabase(154-178)TestMigrationWithoutVerification(181-207)TestMigrationBatchSizes(210-240)TestMigrateSpecialKeys(243-288)cmd/cronosd/dbmigrate/migrate.go (3)
MigrationStats(48-54)MigrateOptions(23-45)Migrate(74-233)
cmd/cronosd/cmd/patch_db.go (5)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/dbmigrate/height_filter.go (2)
ParseHeightFlag(124-151)HeightRange(20-24)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(14-17)cmd/cronosd/dbmigrate/patch.go (3)
PatchOptions(46-59)ConflictAsk(36-36)PatchDatabase(62-170)
cmd/cronosd/cmd/migrate_db.go (4)
cmd/cronosd/dbmigrate/height_filter.go (2)
ParseHeightFlag(124-151)HeightRange(20-24)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(14-17)cmd/cronosd/dbmigrate/migrate.go (4)
MigrationStats(48-54)MigrateOptions(23-45)Migrate(74-233)DefaultBatchSize(17-17)
cmd/cronosd/dbmigrate/migrate_dbname_test.go (1)
cmd/cronosd/dbmigrate/migrate.go (2)
MigrateOptions(23-45)Migrate(74-233)
cmd/cronosd/cmd/root.go (1)
cmd/cronosd/cmd/database.go (1)
DatabaseCmd(8-29)
cmd/cronosd/dbmigrate/migrate.go (2)
memiavl/types.go (2)
Logger(6-10)NewNopLogger(18-18)cmd/cronosd/dbmigrate/height_filter.go (3)
HeightRange(20-24)DBNameBlockstore(14-14)DBNameTxIndex(15-15)
cmd/cronosd/dbmigrate/height_filter_test.go (1)
cmd/cronosd/dbmigrate/height_filter.go (1)
HeightRange(20-24)
cmd/cronosd/dbmigrate/patch.go (3)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/dbmigrate/height_filter.go (3)
HeightRange(20-24)DBNameBlockstore(14-14)DBNameTxIndex(15-15)cmd/cronosd/dbmigrate/migrate.go (1)
MigrationStats(48-54)
cmd/cronosd/dbmigrate/migrate_rocksdb_test.go (2)
cmd/cronosd/dbmigrate/migrate.go (2)
MigrateOptions(23-45)Migrate(74-233)memiavl/types.go (2)
Logger(6-10)NewNopLogger(18-18)
🪛 Gitleaks (8.29.0)
cmd/cronosd/dbmigrate/README.md
[high] 941-941: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🪛 LanguageTool
cmd/cronosd/dbmigrate/README.md
[style] ~493-~493: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...xample 6: Large Database Migration For very large databases, disable verification for fas...
(EN_WEAK_ADJECTIVE)
[grammar] ~1043-~1043: Ensure spelling is correct
Context: ... Safety and Best Practices #### Always Backup First ```bash # Timestamp your backups...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[uncategorized] ~1168-~1168: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ase content and paths. 5. "Failed to open source database" ``` Error: failed to open ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
cmd/cronosd/dbmigrate/QUICKSTART.md
[grammar] ~51-~51: Ensure spelling is correct
Context: ...# or manually pkill cronosd ### 2. Backup Your Databases bash # Backup applic...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[grammar] ~494-~494: Ensure spelling is correct
Context: ...| HDD (100MB/s) | SSD (500MB/s) | NVMe (3GB/s) | |--------------|---------------|--...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🪛 markdownlint-cli2 (0.18.1)
cmd/cronosd/dbmigrate/README.md
268-268: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
583-583: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
663-663: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
677-677: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
704-704: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
714-714: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
729-729: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
737-737: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
744-744: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
759-759: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
765-765: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
769-769: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
781-781: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
929-929: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
935-935: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1088-1088: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1114-1114: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1116-1116: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1130-1130: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1132-1132: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1142-1142: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1144-1144: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1155-1155: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1157-1157: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1168-1168: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1170-1170: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1218-1218: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1235-1235: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1277-1277: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1298-1298: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
cmd/cronosd/dbmigrate/QUICKSTART.md
126-126: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
138-138: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
355-355: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
360-360: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
750-750: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
756-756: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
762-762: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
768-768: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
794-794: Bare URL used
(MD034, no-bare-urls)
🪛 Shellcheck (0.11.0)
cmd/cronosd/dbmigrate/swap-migrated-db.sh
[warning] 212-212: FAILED_COUNT appears unused. Verify use (or export if used externally).
(SC2034)
[warning] 345-345: Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.
(SC2010)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
- GitHub Check: integration_tests (ibc)
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (gas)
- GitHub Check: integration_tests (slow)
- GitHub Check: Run golangci-lint
- GitHub Check: integration_tests (unmarked)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (ibc_rly_gas)
- GitHub Check: Analyze (go)
- GitHub Check: unittest
- GitHub Check: build (macos-latest)
- GitHub Check: build (macos-14)
- GitHub Check: build (ubuntu-latest)
- GitHub Check: semgrep-cloud-platform/scan
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
just wanted to clarify, how are we patching application.db if there are corrupted missing values in the KV store for a historical block? do we replay based on the blockstore? or is this feature not supported at the moment? |
This tooling assumes the source db has correct data when patching the blockstore and tx_index db (mainly for querying purpose). So the application.db doesn't support db patch. Therefore, the user should migrate the db. |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (6)
Makefile (1)
112-122: Includingtest-versiondbin defaulttesttarget may require RocksDB toolchain everywhereHooking
test-versiondbinto the maintesttarget meansmake testwill now always run RocksDB-tagged tests (-tags=objstore,rocksdb). That’s fine if CI and dev environments are guaranteed to have RocksDB headers/libs installed, but it’s a behavior change compared to only running generic tests.If the intent is to keep
make testusable on machines without RocksDB, consider either documenting this requirement clearly or gatingtest-versiondbbehind an env flag/separate target.cmd/cronosd/dbmigrate/README.md (1)
470-483: Optional: tighten Markdown formatting (fence languages, headings)The content is thorough; a few small formatting tweaks would make it friendlier to linters and renderers:
- Add explicit languages to generic fenced blocks that currently use just ```—for example, key-format/code examples could use
textor `bash`.- Where you use bold “Note/Important/Debug Output Includes” as pseudo-headings, consider converting to proper
###/####headings for better structure.These are purely cosmetic and can be done opportunistically.
Also applies to: 515-544, 574-607, 862-879, 1021-1105, 1179-1214
cmd/cronosd/dbmigrate/swap-migrated-db.sh (2)
305-337: UseHOME_DIRinstead of hard-coded~/.cronosin guidanceIn the “Next steps” section, the script always prints paths under
~/.cronos/config/...even if the user passed a custom--homedirectory. Since you already trackHOME_DIR, it would be more accurate to reference$HOME_DIR/config/app.tomland$HOME_DIR/config/config.tomlhere so the instructions match the actual node home.This is just a UX improvement; the swap behavior itself is correct.
345-351: Replacels | grep | awkwith direct directory iterationThe final summary uses:
ls -lh "$DATA_DIR" | grep -E "^d" | awk '{print " " $9 " (" $5 ")"}'This pattern is fragile (depends on
lsformatting, breaks on unusual filenames) and is flagged by ShellCheck (SC2010). You can achieve the same effect more robustly with a simple loop:for d in "$DATA_DIR"/*/; do [ -d "$d" ] || continue size=$(du -sh "$d" 2>/dev/null | awk '{print $1}') printf ' %s (%s)\n' "$(basename "$d")" "$size" doneThis avoids parsing
lsoutput while still giving a nice directory/size summary.cmd/cronosd/cmd/migrate_db.go (1)
241-250: Extend “next steps” guidance to mentionapp.tomlas wellThe “IMPORTANT NEXT STEPS” log only tells users to “Update your config.toml to use the new backend type”. For application DB migrations, operators also need to update
app.toml(app-db-backend) to match the new backend, as documented elsewhere (e.g., README and swap script).Consider expanding this section to log both:
app.toml: app-db-backend = "<backend>"config.toml: db_backend = "<backend>"(when CometBFT DBs were migrated)to keep CLI guidance consistent with the rest of the tooling.
cmd/cronosd/dbmigrate/height_filter.go (1)
429-471: Protobuf parsing heuristic is reasonable but could be fragile.The
extractBlockHashFromMetadatafunction uses a heuristic to parse protobuf wire format directly, looking for specific tag patterns (0x0a for BlockID, nested 0x0a for Hash). This works for the current CometBFT protobuf schema but could break if the schema changes.Consider documenting this assumption or adding a comment referencing the CometBFT BlockMeta protobuf definition for future maintainers.
Apply this diff to add a reference comment:
// extractBlockHashFromMetadata attempts to extract the block hash from H: (block metadata) key value // The block hash is typically stored in the BlockMeta protobuf structure +// Reference: github.com/cometbft/cometbft/types/block.proto (BlockMeta message) // Returns the hash bytes and true if successful, nil and false otherwise
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
Makefile(1 hunks)cmd/cronosd/cmd/database.go(1 hunks)cmd/cronosd/cmd/migrate_db.go(1 hunks)cmd/cronosd/cmd/patch_db.go(1 hunks)cmd/cronosd/dbmigrate/README.md(1 hunks)cmd/cronosd/dbmigrate/height_filter.go(1 hunks)cmd/cronosd/dbmigrate/patch.go(1 hunks)cmd/cronosd/dbmigrate/swap-migrated-db.sh(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (4)
cmd/cronosd/cmd/patch_db.go (6)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/dbmigrate/height_filter.go (2)
ParseHeightFlag(124-151)HeightRange(20-24)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/patch.go (3)
PatchOptions(45-58)ConflictAsk(35-35)PatchDatabase(122-200)cmd/cronosd/dbmigrate/migrate.go (1)
DefaultBatchSize(18-18)
cmd/cronosd/cmd/database.go (2)
cmd/cronosd/cmd/migrate_db.go (1)
MigrateCmd(268-273)cmd/cronosd/cmd/patch_db.go (1)
PatchCmd(343-348)
cmd/cronosd/cmd/migrate_db.go (4)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate.go (4)
MigrationStats(47-53)MigrateOptions(24-44)Migrate(73-204)DefaultBatchSize(18-18)
cmd/cronosd/dbmigrate/patch.go (2)
cmd/cronosd/dbmigrate/height_filter.go (3)
HeightRange(20-24)DBNameBlockstore(14-14)DBNameTxIndex(15-15)cmd/cronosd/dbmigrate/migrate.go (1)
MigrationStats(47-53)
🪛 LanguageTool
cmd/cronosd/dbmigrate/README.md
[style] ~399-~399: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...xample 5: Large Database Migration For very large databases, disable verification for fas...
(EN_WEAK_ADJECTIVE)
[grammar] ~976-~976: Ensure spelling is correct
Context: ... Safety and Best Practices #### Always Backup First ```bash # Timestamp your backups...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[uncategorized] ~1101-~1101: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ase content and paths. 5. "Failed to open source database" ``` Error: failed to open ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.18.1)
cmd/cronosd/dbmigrate/README.md
207-207: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
489-489: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
574-574: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
588-588: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
615-615: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
626-626: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
642-642: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
650-650: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
657-657: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
672-672: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
678-678: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
682-682: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
694-694: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
862-862: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
868-868: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1021-1021: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1047-1047: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1049-1049: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1063-1063: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1065-1065: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1075-1075: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1077-1077: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1088-1088: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1090-1090: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1101-1101: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1103-1103: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1139-1139: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1156-1156: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1179-1179: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1200-1200: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🪛 Shellcheck (0.11.0)
cmd/cronosd/dbmigrate/swap-migrated-db.sh
[warning] 348-348: Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.
(SC2010)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
- GitHub Check: Run golangci-lint
- GitHub Check: build (macos-14)
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (unmarked)
- GitHub Check: build (ubuntu-latest)
- GitHub Check: integration_tests (gas)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (slow)
- GitHub Check: Analyze (go)
- GitHub Check: integration_tests (ibc)
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (ibc_rly_gas)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: build (macos-latest)
- GitHub Check: unittest
- GitHub Check: semgrep-cloud-platform/scan
- GitHub Check: semgrep-cloud-platform/scan
🔇 Additional comments (7)
cmd/cronosd/cmd/database.go (1)
8-26: Database command wiring looks good
DatabaseCmdcleanly groups themigrateandpatchsubcommands underdatabase/db, with clear help text. No issues from a CLI structure standpoint.cmd/cronosd/cmd/patch_db.go (1)
129-340: LGTM! Well-structured CLI implementation.The patch command implementation is comprehensive and handles all edge cases properly:
- Required flag validation prevents accidental misuse
- Nil-safe stats logging addresses the previous concern
- Target path validation ensures only .db directories are accepted
- Per-database patching with proper error handling and statistics aggregation
- Clear distinction between dry-run and actual execution modes
cmd/cronosd/dbmigrate/height_filter.go (1)
383-427: Height filtering is correctly applied at the Go level.The iterator strategy uses prefix-only bounds (e.g.,
"H:"to"I:"), which means keys are not constrained by numeric height ranges due to lexical ordering. However, the code properly applies height filtering at the Go level:
- In
countKeysForPatch(patch.go:232-237)- In
patchWithIteratorviashouldProcessKey(patch.go:1114)- In
patchTxHeightKeys(patch.go:475-484)This matches the documented "Strategy B: prefix-only iterators with Go-level numeric filtering" approach. While not as efficient as numeric-bounded iterators, it's a pragmatic solution given the string-encoded height keys.
cmd/cronosd/dbmigrate/patch.go (4)
774-907: Map iteration non-determinism is acceptable for this CLI tool.The security scanner flagged the map iteration at line 774 as a potential source of non-determinism. While true, this is acceptable because:
- This is a CLI migration tool, not consensus-critical blockchain code
- The patching operation itself is deterministic (writes the same key-value pairs regardless of iteration order)
- The map iteration only affects the order of progress logging, not the final database state
Based on learnings: Non-determinism warnings from static analysis are context-dependent and acceptable for CLI tooling.
1005-1064: BH: (block header by hash) keys are automatically patched.For blockstore H: (height) keys, the code correctly patches the corresponding BH: (block header by hash) keys:
- In dry-run mode (lines 1016-1033): Logs what would be patched
- In actual mode (lines 1042-1064): Fetches and writes BH: keys from source
This addresses the concern raised in past review comments about potentially losing BH: key data during patching.
122-200: Well-structured patching orchestration with proper resource management.The
PatchDatabasefunction properly:
- Validates options before proceeding
- Opens databases with deferred cleanup
- Counts keys before patching to set realistic expectations
- Handles zero-key edge case gracefully
- Flushes RocksDB when appropriate
- Tracks timing for statistics
The time.Now() calls (lines 130, 198) are appropriate for migration statistics and don't pose a non-determinism risk in this CLI context.
226-227: No issues found.The project targets Go 1.23.12, which fully supports the built-in
minfunction introduced in Go 1.21. The code at lines 226-227 is correct as-is.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
cmd/cronosd/dbmigrate/README.md (1)
399-409: Minor style improvement: Consider removing intensifier "very".Line 407 uses "very large databases." The intensifier "very" can be removed for more concise, professional documentation:
-For very large databases, disable verification for faster migration: +For large databases, disable verification for faster migration:This is optional and improves readability.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
cmd/cronosd/dbmigrate/README.md(1 hunks)
🧰 Additional context used
🪛 Gitleaks (8.29.0)
cmd/cronosd/dbmigrate/README.md
[high] 873-873: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🪛 LanguageTool
cmd/cronosd/dbmigrate/README.md
[style] ~399-~399: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...xample 5: Large Database Migration For very large databases, disable verification for fas...
(EN_WEAK_ADJECTIVE)
[grammar] ~975-~975: Ensure spelling is correct
Context: ... Safety and Best Practices #### Always Backup First ```bash # Timestamp your backups...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[uncategorized] ~1100-~1100: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ase content and paths. 5. "Failed to open source database" ``` Error: failed to open ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.18.1)
cmd/cronosd/dbmigrate/README.md
207-207: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
489-489: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
573-573: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
587-587: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
614-614: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
625-625: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
641-641: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
649-649: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
656-656: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
671-671: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
677-677: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
681-681: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
693-693: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
861-861: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
867-867: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1020-1020: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1046-1046: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1048-1048: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1062-1062: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1064-1064: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1074-1074: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1076-1076: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1087-1087: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1089-1089: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1100-1100: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1102-1102: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1138-1138: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1155-1155: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
1178-1178: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
1199-1199: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
- GitHub Check: Run golangci-lint
- GitHub Check: build (macos-latest)
- GitHub Check: build (macos-14)
- GitHub Check: build (ubuntu-latest)
- GitHub Check: unittest
- GitHub Check: integration_tests (slow)
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (gas)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (ibc_rly_gas)
- GitHub Check: integration_tests (unmarked)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: integration_tests (ibc)
- GitHub Check: Analyze (go)
- GitHub Check: semgrep-cloud-platform/scan
- GitHub Check: semgrep-cloud-platform/scan
🔇 Additional comments (3)
cmd/cronosd/dbmigrate/README.md (3)
861-878: Confirm Gitleaks finding at line 873 is a false positive.Gitleaks flagged line 873 as detecting a "Generic API Key" with high severity. The flagged content is
0x0a8f01...in an example debug output showing a value preview (protobuf-encoded binary data). This is clearly example documentation, not a real credential.This is a false positive but warrants explicit suppression to prevent future scanning confusion.
Add a Gitleaks suppression comment if the hex example should remain:
For blockstore keys (text):+# gitleaks:allow
DBG Patched key to target database key=C:5000000 key_size=9 value_preview=0x0a8f01... value_size=143 batch_count=1Alternatively, replace with a more generic placeholder if actual hex examples are not necessary.
205-252: No issues found; path patterns are consistent across implementation and documentation.The verification confirms that all temporary database paths follow the unified pattern
<name>.migrate-temp.db:
- migrate.go: RocksDB directly uses
.migrate-temp.db; LevelDB passes.migrate-temptodbm.NewDB, which appends.db- swap-migrated-db.sh: Consistently references
${db_name}.migrate-temp.db- README.md: All 9 documented examples use
.migrate-temp.dbThe implementation and documentation are properly aligned.
519-531: MigrateOptions documentation is accurate and correctly reflects the implementation.Verification confirms the README documentation (lines 519-531) accurately represents the actual
MigrateOptionsstruct incmd/cronosd/dbmigrate/migrate.go:24-44. Both document the same 9 fields withoutHeightRange, which is correct—height-based filtering applies only topatchoperations, notmigrate. The documentation is consistent with the implementation.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
cmd/cronosd/dbmigrate/README.md (4)
104-110: Add language identifiers to fenced code blocks for markdown consistency.Multiple code blocks lack language specifications, flagged by markdownlint MD040. While the README is comprehensive, the formatting inconsistencies undermine documentation quality. Examples:
- Bash examples (lines 104-110, 134-141, etc.) should use
```bash- Go code snippets should use
```go- Plain text/output should use
```textApply language tags to all fenced code blocks. Example fix:
-``` +```bash cronosd database migrate \ --source-backend goleveldb \This improves syntax highlighting and IDE support for users copying examples.
Also applies to: 134-141, 148-153, 160-165, 225-252, 271-287, 298-316, 335-358, 365-372, 380-395, 402-409, 437-440, 447-451, 475-483
399-399: Consider replacing intensifier "very" for better clarity.Line 399 reads "For very large databases, disable verification for faster migration." The term "very large" is vague; consider more specific phrasing such as "For large databases (>100GB)" or "For database sizes exceeding available memory."
-For very large databases, disable verification for faster migration: +For large databases exceeding available memory, disable verification for faster migration:
671-681: Replace emphasis with proper heading syntax for error sections.Several error/section headers use bold emphasis (e.g.,
**1. "target database does not exist"**) instead of semantic markdown headings. This violates markdownlint MD036 and reduces document structure clarity. Lines 671, 677, 681, 1046, 1062, 1074, 1087, 1100 are affected.Replace emphasis-based headers with proper heading levels:
-**1. "target database does not exist"** +#### "target database does not exist" -``` +```text Error: target database does not exist: /path/to/blockstore.db -``` +```Consistent heading hierarchy improves readability and enables TOC generation.
Also applies to: 1046-1102
989-1001: Minor grammar check: "Timestamp your backups" capitalization.Line 989 uses "Timestamp" as a command imperative. LanguageTool flagged this; the intent is clear, but for consistency with other imperative headers, consider rephrasing to "Add a timestamp to backups" or restructuring as "Timestamped Backups" heading.
-# Timestamp your backups +# Add a timestamp to backups TIMESTAMP=$(date +%Y%m%d-%H%M%S)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
cmd/cronosd/dbmigrate/README.md(1 hunks)
🧰 Additional context used
🪛 Gitleaks (8.29.0)
cmd/cronosd/dbmigrate/README.md
[high] 887-887: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🪛 LanguageTool
cmd/cronosd/dbmigrate/README.md
[style] ~399-~399: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...xample 5: Large Database Migration For very large databases, disable verification for fas...
(EN_WEAK_ADJECTIVE)
[grammar] ~989-~989: Ensure spelling is correct
Context: ... Safety and Best Practices #### Always Backup First ```bash # Timestamp your backups...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[uncategorized] ~1114-~1114: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ... content and paths. #### 5. "Failed to open source database" ```text Error: failed to ope...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
- GitHub Check: Run golangci-lint
- GitHub Check: integration_tests (gas)
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: integration_tests (slow)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (ibc)
- GitHub Check: integration_tests (ibc_rly_gas)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (unmarked)
- GitHub Check: unittest
- GitHub Check: build (macos-14)
- GitHub Check: build (ubuntu-latest)
- GitHub Check: build (macos-latest)
- GitHub Check: Analyze (go)
- GitHub Check: semgrep-cloud-platform/scan
- GitHub Check: semgrep-cloud-platform/scan
🔇 Additional comments (6)
cmd/cronosd/dbmigrate/README.md (6)
207-216: Temp database path pattern is correct; past issue appears addressed.The documentation consistently shows the correct pattern
<name>.migrate-temp.db(e.g.,application.migrate-temp.db,blockstore.migrate-temp.db) across all examples. This aligns with the swap script and code implementation. The past review comment indicating a path mismatch appears to have been resolved in the referenced commits.Also applies to: 243-252, 388-393
887-887: Gitleaks alert on line 887 is a false positive.The static analysis tool flagged line 887 as containing a "generic API key," but this is an example of a hex-encoded transaction hash in debug output documentation (
0x1a2b3c4d5e6f7890abcdef1234567890abcdef1234567890abcdef1234567890). This is sample data for documentation purposes, not a real credential. No action needed, but consider adding a comment if similar patterns appear elsewhere.
1114-1114: Compound adjective formatting: "source database" should use hyphen when preceding noun.LanguageTool flagged line 1114: "Error: failed to open source database". When "source database" is used attributively (modifying "error"), it should be "source-database error." However, in this context, "source database" is a noun phrase (the direct object of "open"), not a compound adjective, so the current usage is correct. The linter message is a false positive.
545-700: Height filtering technical documentation is clear and well-detailed.The sections covering CometBFT key formats (lines 611-676), bounded iterator optimization (lines 568-609), and the three-pass tx_index patching approach (lines 678-716) are comprehensive and technically accurate. The documentation effectively explains performance considerations and implementation details for developers integrating this package.
818-857: Patch command flags and dry-run documentation is well-structured.The command-line flag tables (lines 818-857), dry-run mode description (lines 839-843), and debug logging examples (lines 845-892) provide clear guidance for operators. The debug output examples show practical formatting (text keys vs. hex binary values) that users will encounter.
519-531: No issues found—MigrateOptions snippet is accurate.The README documentation at lines 519-531 correctly reflects the actual
MigrateOptionsstruct definition incmd/cronosd/dbmigrate/migrate.go. All nine fields (SourceHome,TargetHome,SourceBackend,TargetBackend,BatchSize,Logger,RocksDBOptions,Verify, andDBName) match exactly in type and order. The comments provided in the README are accurate summaries of each field's purpose.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
cmd/cronosd/cmd/migrate_db_test.go (1)
43-52: Consider verifying the actual parsed value.The test currently only checks that the result is not empty for valid inputs, but doesn't verify that
parseBackendTypereturns the expected backend type. Consider asserting the actual returned value to ensure the parser correctly transforms or validates the input.Example enhancement:
for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { result, err := parseBackendType(tt.input) if tt.expectError { require.Error(t, err) } else { require.NoError(t, err) require.NotEmpty(t, result) + require.Equal(t, tt.input, result) // or the expected normalized form } }) }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
cmd/cronosd/cmd/migrate_db.go(1 hunks)cmd/cronosd/cmd/migrate_db_test.go(1 hunks)cmd/cronosd/cmd/patch_db.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- cmd/cronosd/cmd/migrate_db.go
🧰 Additional context used
🧬 Code graph analysis (2)
cmd/cronosd/cmd/migrate_db_test.go (1)
cmd/cronosd/cmd/migrate_db.go (8)
BackendType(34-34)GoLevelDB(38-38)LevelDB(39-39)RocksDB(40-40)DbType(25-25)App(29-29)CometBFT(30-30)All(31-31)
cmd/cronosd/cmd/patch_db.go (7)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/cmd/migrate_db.go (3)
BackendType(34-34)GoLevelDB(38-38)RocksDB(40-40)cmd/cronosd/dbmigrate/height_filter.go (2)
ParseHeightFlag(124-151)HeightRange(20-24)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/patch.go (3)
PatchOptions(45-58)ConflictAsk(35-35)PatchDatabase(122-200)cmd/cronosd/dbmigrate/migrate.go (1)
DefaultBatchSize(18-18)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
- GitHub Check: Analyze (go)
- GitHub Check: Run golangci-lint
- GitHub Check: build (macos-latest)
- GitHub Check: build (macos-14)
- GitHub Check: unittest
- GitHub Check: build (ubuntu-latest)
- GitHub Check: integration_tests (gas)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (slow)
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (ibc_rly_gas)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (ibc)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: integration_tests (unmarked)
- GitHub Check: semgrep-cloud-platform/scan
- GitHub Check: semgrep-cloud-platform/scan
🔇 Additional comments (6)
cmd/cronosd/cmd/migrate_db_test.go (6)
56-87: LGTM!Comprehensive coverage of valid and invalid database names with clear test cases for edge cases like empty strings and common variations.
89-175: LGTM!Excellent test coverage with clear distinction between error cases. The handling of empty strings after trimming (lines 141-145) and the different error messages for empty input vs. delimiter-only input (lines 147-157) demonstrate thoughtful design.
177-182: LGTM!Straightforward validation of backend type constants.
184-189: LGTM!Straightforward validation of database type constants.
191-241: LGTM!Comprehensive coverage of the database type mapping with all valid types (App, CometBFT, All) and proper error case handling. The mappings align with the PR's stated purpose of distinguishing between application and CometBFT databases.
243-293: LGTM!Clear test of flag precedence logic. The implementation correctly demonstrates that
--databasestakes precedence over--db-type, which is sensible UX design—the more specific flag overrides the broader categorical flag.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
cmd/cronosd/cmd/migrate_db_test.go (1)
10-54: Backend-type parsing tests are solid; consider asserting exact dbm.BackendType valuesThe table tests exercise valid/invalid inputs well. If you want stronger regression protection, you could also assert the concrete
dbm.BackendTypereturned for each case (e.g.,dbm.GoLevelDBBackend,dbm.RocksDBBackend) instead of justNotEmpty, but the current checks are functionally fine.cmd/cronosd/cmd/patch_db.go (1)
221-243: Target-path resolution logic for single vs multi-DB runs is clear but tightly coupled to.dbsuffixesThe logic now enforces:
- Single DB:
--target-pathmust be a*.dbpath.- Multi-DB:
--target-pathmust be a data directory (not ending in.db), and per-DB targets are derived as<dir>/<db>.db.This addresses the earlier risks of accidentally patching
data.dbor nested*.dbdirectories. Just be aware that “directory” in the error message for the single-DB case really means “directory path ending in.db”, so if you ever want to accept directory-style inputs like.../blockstoreyou’ll need to relax thefilepath.Ext(...) == ".db"check.cmd/cronosd/cmd/patch_db_test.go (1)
10-81: Test only partially mirrors actual target-path validation; single-DB case is now out of syncThe table defines
"single DB without .db extension - allowed (will be validated later)", and the simulated logic explicitly skips any validation whendbCount == 1. Inpatch_db.go, though, the.dbextension check runs for both single- and multi-DB cases, so a single-DB path like/path/to/blockstorewill now be rejected.That means:
- The “allowed” comment for the second test case no longer matches real CLI behavior.
- The simulation is intentionally only exercising the multi-DB
*.dbrejection, not the full target-path logic.It would be clearer (and less brittle) to either:
- Update the test to reflect the actual single-DB requirement (and enforce the
.dbsuffix there too), or- Narrow the test to only multi-DB cases and adjust the comments to say it is specifically validating the multi-DB directory vs
*.dbrules.Optionally, you could factor the target-path validation into a small helper in
patch_db.goand call that directly from the test to avoid duplicating logic.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
cmd/cronosd/cmd/migrate_db.go(1 hunks)cmd/cronosd/cmd/migrate_db_test.go(1 hunks)cmd/cronosd/cmd/patch_db.go(1 hunks)cmd/cronosd/cmd/patch_db_test.go(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
cmd/cronosd/cmd/migrate_db_test.go (1)
cmd/cronosd/cmd/migrate_db.go (14)
BackendType(34-34)GoLevelDB(38-38)LevelDB(39-39)RocksDB(40-40)DatabaseName(43-43)Application(47-47)Blockstore(48-48)State(49-49)TxIndex(50-50)Evidence(51-51)DbType(25-25)App(29-29)CometBFT(30-30)All(31-31)
cmd/cronosd/cmd/patch_db.go (7)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/cmd/migrate_db.go (3)
BackendType(34-34)GoLevelDB(38-38)RocksDB(40-40)cmd/cronosd/dbmigrate/height_filter.go (2)
ParseHeightFlag(124-151)HeightRange(20-24)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/patch.go (2)
PatchOptions(45-58)PatchDatabase(122-200)cmd/cronosd/dbmigrate/migrate.go (1)
DefaultBatchSize(18-18)
cmd/cronosd/cmd/migrate_db.go (3)
cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate.go (4)
MigrationStats(47-53)MigrateOptions(24-44)Migrate(73-204)DefaultBatchSize(18-18)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (17)
- GitHub Check: Analyze (go)
- GitHub Check: build (macos-14)
- GitHub Check: build (ubuntu-latest)
- GitHub Check: build (macos-latest)
- GitHub Check: unittest
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (gas)
- GitHub Check: Run golangci-lint
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: integration_tests (slow)
- GitHub Check: integration_tests (unmarked)
- GitHub Check: integration_tests (ibc)
- GitHub Check: integration_tests (ibc_rly_gas)
🔇 Additional comments (19)
cmd/cronosd/cmd/migrate_db_test.go (5)
56-87: Database name validity coverage looks good
TestValidDatabaseNamesand the invalid-name cases line up with thevalidDatabaseNamesmap and the exportedDatabaseNameconstants. This should catch accidental additions/removals or typos in supported DB names.
89-175: parseDatabaseNames is very well covered for edge casesThe tests here exercise single/multiple names, whitespace, invalid entries, empty segments, and fully empty input, including checking error substrings. This gives good confidence that
parseDatabaseNamesbehaves correctly under messy CLI input.
177-198: Constant value tests guard against accidental string driftVerifying
BackendType,DatabaseName, andDbTypeconstant string values is useful here since the CLI flags are stringly-typed; these tests will catch accidental renames that could silently break flag handling.
200-250: DB-type-to-database mapping and error-path are well specified
TestDBTypeMappingthoroughly validates the mapping forapp,cometbft, andall, plus the invalid type case with an error substring check. This keepsgetDBNamesFromTypehonest as you evolve the supported sets.
252-305: Flag-precedence test correctly captures--databasesvs--db-typebehavior
TestDatabasesFlagPrecedencemirrors the RunE logic and confirms that--databasesoverrides--db-typewhen both are supplied. The expectations for all three scenarios look correct.cmd/cronosd/cmd/patch_db.go (5)
142-173: Required flag checks and DB-name parsing are consistent with documented scopeThe explicit checks for
--source-home,--database,--height, and--target-pathplus the in-function parsing restricting databases toblockstoreandtx_indexmatch the command’s documented capabilities and fail fast on bad input. This looks correct for a patch tool that intentionally only supports those two stores.
175-199: Height parsing and validation protect against “patch everything” mistakesUsing
dbmigrate.ParseHeightFlag, thenValidate()andIsEmpty()with a dedicated error when the range is empty ensures you never accidentally run a full-history patch by omitting or mis-specifying--height. This is a good safety guard for an invasive operation.
210-214: RocksDB options are prepared only when requiredConditionally calling
dbmigrate.PrepareRocksDBOptions()only when the target backend is RocksDB keeps the non-RocksDB path simple and aligns with the build-tagged implementations inmigrate_rocksdb.go/migrate_no_rocksdb.go.
249-295: Per-database logging and aggregate stats look correctPer-DB logs include total/processed keys, error counts, and duration. Aggregates (
totalKeysPatched,totalErrors,totalDuration) are summed across DBs and summarized with mode-specific keys (keys_foundin dry-run,keys_patchedotherwise). This should give operators enough information to reason about patch runs.
265-281: Nil-safe error logging around PatchDatabaseThe nil check around
statsbefore logging error details avoids the panic that could occur ifdbmigrate.PatchDatabasereturns(nil, err). This resolves the earlier concern about crashing during error reporting and is the right defensive pattern here.cmd/cronosd/cmd/patch_db_test.go (1)
85-92: Custom error type keeps the test decoupled from exact error formattingUsing
targetPathErrorwith a stable message and only asserting the"must be a data directory"substring makes the test reasonably resilient while still checking the right semantics. This is fine as long as you keep the high-level phrasing aligned withpatch_db.go.cmd/cronosd/cmd/migrate_db.go (8)
25-61: Enum-like types and validDatabaseNames map are a good guardrail for CLI inputsDefining
DbType,BackendType, andDatabaseNameas distinct string types with constants, and tying them intovalidDatabaseNames, gives you compile-time checks and a single source of truth for valid values. This is a nice improvement over raw strings and helps keep the CLI and tests aligned.
63-121: Migrate command description and flag set match the underlying behaviorThe long help text clearly explains db-type vs specific-database modes, the
.migrate-temp.dbsuffix, verification, and operational cautions. Flag definitions (including defaults for backends, batch size, and verify) match those semantics and provide sane defaults.
122-193: RunE: backend parsing and target-home defaulting are handled cleanlyParsing
sourceBackend/targetBackendviaparseBackendTypeand rejecting identical backends avoids no-op or misconfigured runs. Falling back to--homewhen--target-homeis unset and logging that decision is helpful for operators reading logs.
155-177: Database selection logic correctly prioritizes--databasesover--db-typeUsing
parseDatabaseNameswhen--databasesis set, and only falling back togetDBNamesFromType(DbType(dbType))otherwise, matches the documented precedence. Converting to[]DatabaseNamekeeps things type-safe throughout the rest of the function.
195-201: RocksDB options reuse the shared preparerConditionally invoking
dbmigrate.PrepareRocksDBOptions()here keeps RocksDB tuning centralized and respects the build-tagged alternatives. That makes it easy to evolve RocksDB configuration without touching the CLI.
202-251: Per-database migration loop and nil-safe error logging are robustFor each selected DB:
- Options passed to
dbmigrate.Migratecorrectly mirror the CLI flags (home dirs, backends, batch size, verify, Rocks options, DBName).- The error branch guards against
stats == nilbefore logging counters, avoiding panics and returning a wrapped error with DB context.- Aggregate stats use the atomic fields on
MigrationStatsto sum counts across DBs.This is a solid pattern for orchestrating multiple migrations.
253-283: Summary and “next steps” logs provide actionable operator guidanceThe summary logs differentiate between explicit
--databasesand--db-typeusage, and the “IMPORTANT NEXT STEPS” section plus per-DB temp-path log (%s/data/%s.migrate-temp.db) correctly reflect the temp directory naming indbmigrate.Migrate. This should make post-migration manual steps much clearer for users.
307-355: Helper functions encapsulate backend/db-name parsing cleanly
parseBackendTypecentralizes mapping from CLI strings todbm.BackendTypeand supports theleveldbalias.parseDatabaseNameshandles trimming, validation againstvalidDatabaseNames, and informative error messages.getDBNamesFromTypecodifies the mapping forapp,cometbft, andall, with a clear error on invalid types.These helpers are small, focused, and well-covered by the tests in
migrate_db_test.go.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (5)
cmd/cronosd/dbmigrate/migrate.go (2)
19-20: Unused constantDefaultWorkers.The
DefaultWorkersconstant is defined but never used in the migration implementation. If concurrent workers are planned for a future enhancement, consider adding a TODO comment; otherwise, remove it to keep the code clean.
221-230: Unused parametersourceDBinmigrateWithIterator.The
migrateDatafunction passessourceDBtomigrateWithIterator(line 229), but the parameter is never used insidemigrateWithIterator(line 233). Consider removing this parameter from both the function signature and the call site to improve clarity.Apply this diff:
func migrateData(sourceDB, targetDB dbm.DB, opts MigrateOptions, stats *MigrationStats) error { itr, err := sourceDB.Iterator(nil, nil) if err != nil { return err } defer itr.Close() - return migrateWithIterator(itr, sourceDB, targetDB, opts, stats) + return migrateWithIterator(itr, targetDB, opts, stats) } // migrateWithIterator migrates data from a single iterator -func migrateWithIterator(itr dbm.Iterator, sourceDB, targetDB dbm.DB, opts MigrateOptions, stats *MigrationStats) error { +func migrateWithIterator(itr dbm.Iterator, targetDB dbm.DB, opts MigrateOptions, stats *MigrationStats) error {cmd/cronosd/cmd/patch_db.go (3)
156-173: Deduplicate--databaseentries to avoid patching the same store multiple times.If a user passes duplicates like
--database blockstore,blockstore,validDBNameswill contain repeated entries and the same DB will be patched multiple times, which is rarely useful and may complicate conflict handling.You can cheaply dedup while preserving order:
- dbNames := strings.Split(databases, ",") - var validDBNames []string - for _, db := range dbNames { + dbNames := strings.Split(databases, ",") + validDBSet := make(map[string]struct{}) + var validDBNames []string + for _, db := range dbNames { db = strings.TrimSpace(db) if db == "" { continue } // Validate database if db != "blockstore" && db != "tx_index" { return fmt.Errorf("invalid database: %s (must be: blockstore or tx_index)", db) } - validDBNames = append(validDBNames, db) + if _, exists := validDBSet[db]; exists { + continue + } + validDBSet[db] = struct{}{} + validDBNames = append(validDBNames, db) }This prevents accidental double work while keeping the UX of the flag unchanged.
291-322: Clarify exit semantics when per-key errors occur (totalErrors > 0).The command always returns
nilonce it reaches the summary (Lines 291–322), even iftotalErrorsis non-zero. That means a run with key-level failures still exits with status 0, which may surprise operators or scripts that interpret “no error” as “clean patch.”If your intent is:
- Best-effort patch: current behavior is fine, but consider making it explicit in the log line, e.g. “Patch completed with errors; see logs for details”, or only print “COMPLETED SUCCESSFULLY” when
totalErrors == 0.- Strict patch: consider returning an error when
totalErrors > 0, for example:if totalErrors > 0 { return fmt.Errorf("patch completed with %d key-level errors", totalErrors) } return nilWorth aligning this with how
dbmigrate.PatchDatabasetreats soft vs hard errors so callers and automation know what a zero exit code guarantees.
347-352: SimplifyPatchCmdwrapper (no behavior difference).
PatchCmdcurrently just wrapsPatchDBCmdand reassignsUseandDeprecated, butPatchDBCmdalready setsUse: "patch"and the deprecation string is empty, so this adds no behavior.You can simplify:
func PatchCmd() *cobra.Command { - cmd := PatchDBCmd() - cmd.Use = "patch" - cmd.Deprecated = "" - return cmd + return PatchDBCmd() }If you anticipate diverging top-level vs subcommand behavior in future, keeping the wrapper is fine; otherwise this reduces one layer of indirection.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
cmd/cronosd/cmd/patch_db.go(1 hunks)cmd/cronosd/cmd/patch_db_test.go(1 hunks)cmd/cronosd/dbmigrate/migrate.go(1 hunks)cmd/cronosd/dbmigrate/patch.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- cmd/cronosd/cmd/patch_db_test.go
- cmd/cronosd/dbmigrate/patch.go
🧰 Additional context used
🧬 Code graph analysis (2)
cmd/cronosd/cmd/patch_db.go (7)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/cmd/migrate_db.go (3)
BackendType(34-34)GoLevelDB(38-38)RocksDB(40-40)cmd/cronosd/dbmigrate/height_filter.go (2)
ParseHeightFlag(124-151)HeightRange(20-24)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/patch.go (4)
DbExtension(21-21)PatchOptions(45-58)ConflictAsk(35-35)PatchDatabase(122-200)cmd/cronosd/dbmigrate/migrate.go (1)
DefaultBatchSize(18-18)
cmd/cronosd/dbmigrate/migrate.go (1)
cmd/cronosd/dbmigrate/patch.go (1)
DbExtension(21-21)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
- GitHub Check: integration_tests (slow)
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (gas)
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: integration_tests (ibc_rly_gas)
- GitHub Check: integration_tests (unmarked)
- GitHub Check: integration_tests (ibc)
- GitHub Check: Run golangci-lint
- GitHub Check: Analyze (go)
- GitHub Check: build (macos-14)
- GitHub Check: build (ubuntu-latest)
- GitHub Check: build (macos-latest)
- GitHub Check: unittest
- GitHub Check: semgrep-cloud-platform/scan
- GitHub Check: semgrep-cloud-platform/scan
🔇 Additional comments (7)
cmd/cronosd/dbmigrate/migrate.go (7)
46-62: Atomic operations and progress calculation are correct.The use of atomic types for thread-safe statistics tracking is appropriate. The floating-point division for progress calculation (line 61) is flagged by the security scanner as potentially non-deterministic, but this is acceptable for a migration tool where progress reporting doesn't need deterministic reproducibility.
73-112: Migration initialization and source DB handling are correct.The default option handling, logging, and source database opening with proper cleanup are well-implemented. The security scanner flags
time.Now()at line 85 as potentially non-deterministic, but this is acceptable for tracking migration duration in an operational tool.
114-142: Target DB creation with proper nil-safe cleanup.The temp directory naming is unified across backends, and the nil check in the deferred cleanup (line 139) correctly prevents panics if target DB creation fails. The differentiation between RocksDB (exact path) and LevelDB (name-based with .db appended by dbm.NewDB) is handled correctly.
159-186: RocksDB flush and timing are appropriate.The conditional RocksDB memtable flush ensures data persistence, and recording
EndTimeat line 180 viatime.Now()is flagged by the security scanner but acceptable for migration statistics. The structured logging provides useful operational visibility.
232-289: Batch migration logic is well-implemented.The key/value copying (lines 245-248) correctly guards against iterator slice reuse, batch writes are properly sized and committed, and progress reporting provides good operational visibility. The time-based progress logging (lines 270-278) uses
time.Now(), which is flagged by the security scanner but appropriate for this use case.
291-344: Retry logic with exponential backoff is solid.Both
openDBWithRetryandopenRocksDBWithRetryimplement proper exponential backoff to handle OS-level file lock delays. The multipletime.Now()andtime.Sleep()calls are flagged by the security scanner as non-deterministic, but this is expected and acceptable for retry timing in an operational tool.
346-500: Two-phase verification ensures data integrity.The verification correctly checks both directions: source-to-target (lines 382-425) for missing/mismatched keys, and target-to-source (lines 432-473) for extra keys. This catches all categories of data inconsistency. The multiple
time.Now()calls for progress reporting (lines 379, 423, 440, 471) are flagged by the security scanner but appropriate for tracking long-running verification operations.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
cmd/cronosd/cmd/patch_db.go (2)
157-170: Optional: dedupe database list to avoid patching the same store multiple timesRight now, passing duplicate names (e.g.
--database blockstore,blockstore) will cause the same DB to be patched twice. It’s not incorrect, but it’s surprising and wastes time.You can cheaply dedupe at parse time:
- dbNames := strings.Split(databases, ",") - var validDBNames []string - for _, db := range dbNames { + dbNames := strings.Split(databases, ",") + var ( + validDBNames []string + seen = make(map[string]struct{}) + ) + for _, db := range dbNames { db = strings.TrimSpace(db) if db == "" { continue } // Validate database if db != "blockstore" && db != "tx_index" { return fmt.Errorf("invalid database: %s (must be: blockstore or tx_index)", db) } + if _, exists := seen[db]; exists { + continue + } + seen[db] = struct{}{} validDBNames = append(validDBNames, db) }
222-251: Clarify multi-DB--target-pathvalidation by checking the parent directoryThe
.dbextension and per-DB existence checks are good, but when patching multiple databases a wrong--target-path(e.g. pointing at a regular file or a non-existent directory) only surfaces later as “target database does not exist: <…>/blockstore.db”, which can be confusing.You can fail earlier with a clearer message by asserting that
targetPathis an existing directory in the multi-DB branch:- } else { - // For multiple databases, validate that targetPath is a directory, not a *.db file - cleanedTargetPath := filepath.Clean(targetPath) - if filepath.Ext(cleanedTargetPath) == dbmigrate.DbExtension { - return fmt.Errorf("when patching multiple databases, --target-path must be a data directory (e.g., ~/.cronos/data), not a *.db file path (got %q); remove the .db suffix", targetPath) - } - // Treat targetPath as data directory - dbTargetPath = filepath.Join(targetPath, dbName+dbmigrate.DbExtension) - } + } else { + // For multiple databases, validate that targetPath is a directory, not a *.db file + cleanedTargetPath := filepath.Clean(targetPath) + if filepath.Ext(cleanedTargetPath) == dbmigrate.DbExtension { + return fmt.Errorf("when patching multiple databases, --target-path must be a data directory (e.g., ~/.cronos/data), not a *.db file path (got %q); remove the .db suffix", targetPath) + } + if info, err := os.Stat(cleanedTargetPath); err != nil { + if os.IsNotExist(err) { + return fmt.Errorf("target data directory does not exist: %s", cleanedTargetPath) + } + return fmt.Errorf("failed to access target data directory %q: %w", cleanedTargetPath, err) + } else if !info.IsDir() { + return fmt.Errorf("when patching multiple databases, --target-path must be a directory (got %q)", cleanedTargetPath) + } + // Treat targetPath as data directory + dbTargetPath = filepath.Join(cleanedTargetPath, dbName+dbmigrate.DbExtension) + }This keeps the existing per-DB checks but gives operators faster, clearer feedback when the base path is wrong.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
cmd/cronosd/cmd/patch_db.go(1 hunks)cmd/cronosd/cmd/patch_db_test.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- cmd/cronosd/cmd/patch_db_test.go
🧰 Additional context used
🧬 Code graph analysis (1)
cmd/cronosd/cmd/patch_db.go (7)
memiavl/types.go (1)
Logger(6-10)cmd/cronosd/cmd/migrate_db.go (3)
BackendType(34-34)GoLevelDB(38-38)RocksDB(40-40)cmd/cronosd/dbmigrate/height_filter.go (2)
ParseHeightFlag(124-151)HeightRange(20-24)cmd/cronosd/dbmigrate/migrate_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/migrate_no_rocksdb.go (1)
PrepareRocksDBOptions(13-15)cmd/cronosd/dbmigrate/patch.go (4)
DbExtension(21-21)PatchOptions(45-58)ConflictAsk(35-35)PatchDatabase(122-200)cmd/cronosd/dbmigrate/migrate.go (1)
DefaultBatchSize(18-18)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
- GitHub Check: Run golangci-lint
- GitHub Check: build (macos-14)
- GitHub Check: build (macos-latest)
- GitHub Check: unittest
- GitHub Check: build (ubuntu-latest)
- GitHub Check: integration_tests (gas)
- GitHub Check: integration_tests (ica)
- GitHub Check: integration_tests (slow)
- GitHub Check: integration_tests (upgrade)
- GitHub Check: integration_tests (ibc_update_client)
- GitHub Check: integration_tests (ibc_rly_gas)
- GitHub Check: integration_tests (gov)
- GitHub Check: integration_tests (ibc_timeout)
- GitHub Check: integration_tests (ibc_rly_evm)
- GitHub Check: integration_tests (ibc)
- GitHub Check: integration_tests (unmarked)
- GitHub Check: Analyze (go)
- GitHub Check: semgrep-cloud-platform/scan
🔇 Additional comments (1)
cmd/cronosd/cmd/patch_db.go (1)
274-331: Stats handling, error logging, and dry-run summary look solidThe guarded
statslogging on failures, per-DB completion logs, and the final aggregated summary (including different keys for dry-run vs real execution) are all well-structured and operator-friendly. The handling ofMigrationStatsin both success and error cases matches the contract ofPatchDatabaseand avoids panics from nil dereferences.
3 participants
Implementation new CLI to support db migration or patch between goleveldb and rocksdb backend.
For details, can read the README in
cmd/cronosd/dbmigrateSummary by CodeRabbit
New Features
Documentation
Tests
Chores