ledger-tool: unify and colorize help

[brooksprumo]

Problem

The help for ledger-tool could be more user-friendly. One, we could add color. Two, we could unify the flags and the options, like most man pages/help.

Summary of Changes

Colorize and unify ledger-tool's help.

comparison - unify

ledger-tool --help

old:

agave-ledger-tool 2.0.4 (src:4234b1ba; feat:1070130477, client:Agave)
Blockchain, Rebuilt for Scale

USAGE:
    agave-ledger-tool [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
        --force-update-to-open          Allow commands that would otherwise not alter the blockstore to make necessary
                                        updates in order to open it
    -h, --help                          Prints help information
        --ignore-ulimit-nofile-error    Allow opening the blockstore to succeed even if the desired open file descriptor
                                        limit cannot be configured. Use with caution as some commands may run fine with
                                        a reduced file descriptor limit while others will not
    -V, --version                       Prints version information
    -v, --verbose                       Show additional information where supported

OPTIONS:
        --block-verification-method <METHOD>
            Switch transaction scheduling method for verifying ledger entries [default: blockstore-processor] [possible
            values: blockstore-processor, unified-scheduler]
    -l, --ledger <DIR>                                 Use DIR as ledger location [default: ledger]
        --output <FORMAT>
            Return information in specified output format, currently only available for bigtable and program subcommands
            [possible values: json, json-compact]
        --unified-scheduler-handler-threads <COUNT>
            Change the number of the unified scheduler's transaction execution threads dedicated to each block,
            otherwise calculated as cpu_cores/4 [default: 4]
        --wal-recovery-mode <MODE>
            Mode to recovery the ledger db write ahead log [possible values: tolerate_corrupted_tail_records,
            absolute_consistency, point_in_time, skip_any_corrupted_record]

SUBCOMMANDS:
    accounts                   Print account stats and contents after processing the ledger
    bank-hash                  Prints the hash of the working bank after reading the ledger
    bigtable                   Ledger data on a BigTable instance
    blockstore                 Commands to interact with a local Blockstore
    capitalization             Print capitalization (aka, total supply) while checksumming it
    compute-slot-cost          runs cost_model over the block at the given slots, computes how expensive a block was
                               based on cost_model
    create-snapshot            Create a new ledger snapshot
    genesis                    Prints the ledger's genesis config
    genesis-hash               Prints the ledger's genesis hash
    graph                      Create a Graphviz rendering of the ledger
    help                       Prints this message or the help of the given subcommand(s)
    latest-optimistic-slots    Output up to the most recent <num-slots> optimistic slots with their hashes and
                               timestamps.
    modify-genesis             Modifies genesis parameters
    program                    Run to test, debug, and analyze on-chain programs.
    shred-version              Prints the ledger's shred hash
    verify                     Verify the ledger

new:

agave-ledger-tool 2.1.0 (src:00000000; feat:1420694968, client:Agave)
Blockchain, Rebuilt for Scale

USAGE:
    agave-ledger-tool [OPTIONS] <SUBCOMMAND>

OPTIONS:
        --block-verification-method <METHOD>
            Switch transaction scheduling method for verifying ledger entries [default: blockstore-processor] [possible
            values: blockstore-processor, unified-scheduler]
        --force-update-to-open
            Allow commands that would otherwise not alter the blockstore to make necessary updates in order to open it

    -h, --help                                         Prints help information
        --ignore-ulimit-nofile-error
            Allow opening the blockstore to succeed even if the desired open file descriptor limit cannot be configured.
            Use with caution as some commands may run fine with a reduced file descriptor limit while others will not
    -l, --ledger <DIR>                                 Use DIR as ledger location [default: ledger]
        --output <FORMAT>
            Return information in specified output format, currently only available for bigtable and program subcommands
            [possible values: json, json-compact]
        --unified-scheduler-handler-threads <COUNT>
            Change the number of the unified scheduler's transaction execution threads dedicated to each block,
            otherwise calculated as cpu_cores/4 [default: 4]
    -V, --version                                      Prints version information
    -v, --verbose                                      Show additional information where supported
        --wal-recovery-mode <MODE>
            Mode to recovery the ledger db write ahead log [possible values: tolerate_corrupted_tail_records,
            absolute_consistency, point_in_time, skip_any_corrupted_record]

SUBCOMMANDS:
    accounts                   Print account stats and contents after processing the ledger
    bank-hash                  Prints the hash of the working bank after reading the ledger
    bigtable                   Ledger data on a BigTable instance
    blockstore                 Commands to interact with a local Blockstore
    capitalization             Print capitalization (aka, total supply) while checksumming it
    compute-slot-cost          runs cost_model over the block at the given slots, computes how expensive a block was
                               based on cost_model
    create-snapshot            Create a new ledger snapshot
    genesis                    Prints the ledger's genesis config
    genesis-hash               Prints the ledger's genesis hash
    graph                      Create a Graphviz rendering of the ledger
    help                       Prints this message or the help of the given subcommand(s)
    latest-optimistic-slots    Output up to the most recent <num-slots> optimistic slots with their hashes and
                               timestamps.
    modify-genesis             Modifies genesis parameters
    program                    Run to test, debug, and analyze on-chain programs.
    shred-version              Prints the ledger's shred hash
    verify                     Verify the ledger

ledger-tool verify --help

old:

agave-ledger-tool-verify 
Verify the ledger

USAGE:
    agave-ledger-tool verify [FLAGS] [OPTIONS]

FLAGS:
        --accounts-db-skip-shrink
            Enables faster starting of ledger-tool by skipping shrink. This option is for use during testing.

        --accounts-db-test-hash-calculation      
            Enable hash calculation test

        --allow-dead-slots                       
            Output dead slots as well

        --disable-accounts-disk-index
            Disable the disk-based accounts index. It is enabled by default. The entire accounts index will be kept in
            memory.
        --enable-extended-tx-metadata-storage
            Include CPI inner instructions, logs, and return data in the historical transaction info stored

        --enable-rpc-transaction-history         
            Store transaction info for processed slots into local ledger

        --force-update-to-open
            Allow commands that would otherwise not alter the blockstore to make necessary updates in order to open it

    -h, --help                                   
            Prints help information

        --ignore-ulimit-nofile-error
            Allow opening the blockstore to succeed even if the desired open file descriptor limit cannot be configured.
            Use with caution as some commands may run fine with a reduced file descriptor limit while others will not
        --no-snapshot                            
            Do not start from a local snapshot if present

        --os-memory-stats-reporting              
            Enable reporting of OS memory statistics.

        --print-accounts-stats
            After verifying the ledger, print some information about the account stores

        --print-bank-hash                        
            After verifying the ledger, print the working bank's hash

        --run-final-accounts-hash-calculation
            After 'verify' completes, run a final accounts hash calculation. Final hash calculation could race with
            accounts background service tasks and assert.
        --skip-poh-verify
            Deprecated, please use --skip-verification. Skip ledger PoH and transaction verification.

        --skip-verification                      
            Skip ledger PoH and transaction verification.

    -v, --verbose                                
            Show additional information where supported

        --verify-accounts-index                  
            For debugging and tests on accounts index.

        --write-bank-file
            After verifying the ledger, write a file that contains the information that went into computing the
            completed bank's bank hash. The file will be written within <LEDGER_DIR>/bank_hash_details/

OPTIONS:
        --accounts <PATHS>
            Persistent accounts location. May be specified multiple times. [default: <LEDGER>/accounts]

        --accounts-hash-cache-path <PATH>
            Use PATH as accounts hash cache location [default: <LEDGER>/accounts_hash_cache]

        --accounts-index-bins <BINS>
            Number of bins to divide the accounts index into

        --accounts-index-memory-limit-mb <MEGABYTES>
            How much memory the accounts index can consume. If this is exceeded, some account index entries will be
            stored on disk.
        --accounts-index-path <PATH>...
            Persistent accounts-index location. May be specified multiple times. [default: <LEDGER>/accounts_index]

        --block-verification-method <METHOD>
            Switch transaction scheduling method for verifying ledger entries [default: blockstore-processor] [possible
            values: blockstore-processor, unified-scheduler]
        --debug-key <ADDRESS>...
            Log when transactions are processed that reference the given key(s).

        --geyser-plugin-config <FILE>...
            Specify the configuration file for the Geyser plugin.

        --halt-at-slot <SLOT>                                                    
            Halt processing at the given slot

        --hard-fork <SLOT>...                                                    
            Add a hard fork at this slot

        --incremental-snapshot-archive-path <DIR>
            Use DIR for separate incremental snapshot location

    -l, --ledger <DIR>
            Use DIR as ledger location [default: ledger]

        --limit-load-slot-count-from-snapshot <SLOT>
            For debugging and profiling with large snapshots, artificially limit how many slots are loaded from a
            snapshot.
        --log-messages-bytes-limit <BYTES>
            Maximum number of bytes written to the program log before truncation

        --max-genesis-archive-unpacked-size <NUMBER>
            maximum total uncompressed size of unpacked genesis archive [default: 10485760]

        --output <FORMAT>
            Return information in specified output format, currently only available for bigtable and program subcommands
            [possible values: json, json-compact]
        --record-slots <FILENAME>
            Record slots to a file [default: slots.json]

        --record-slots-config <record_slots_config>...
            In addition to the bank hash, optionally include accounts and/or transactions details for the slot [possible
            values: accounts, tx]
        --snapshots <DIR>
            Use DIR for snapshot location [default: --ledger value]

        --unified-scheduler-handler-threads <COUNT>
            Change the number of the unified scheduler's transaction execution threads dedicated to each block,
            otherwise calculated as cpu_cores/4 [default: 4]
        --use-snapshot-archives-at-startup <use_snapshot_archives_at_startup>
            At startup, when should snapshot archives be extracted versus using what is already on disk? 
            Specifying "always" will always startup by extracting snapshot archives and disregard any snapshot-related
            state already on disk. Note that starting up from snapshot archives will incur the runtime costs associated
            with extracting the archives and rebuilding the local state. 
            Specifying "never" will never startup from snapshot archives and will only use snapshot-related state
            already on disk. If there is no state already on disk, startup will fail. Note, this will use the latest
            state available, which may be newer than the latest snapshot archive. 
            Specifying "when-newest" will use snapshot-related state already on disk unless there are snapshot archives
            newer than it. This can happen if a new snapshot archive is downloaded while the node is stopped. [default:
            always]  [possible values: always, never, when-newest]
        --verify-slots <FILENAME>
            Verify slots match contents of file [default: slots.json]

        --wal-recovery-mode <MODE>
            Mode to recovery the ledger db write ahead log [possible values: tolerate_corrupted_tail_records,
            absolute_consistency, point_in_time, skip_any_corrupted_record]

new:

agave-ledger-tool-verify 
Verify the ledger

USAGE:
    agave-ledger-tool verify [OPTIONS]

OPTIONS:
        --accounts <PATHS>
            Persistent accounts location. May be specified multiple times. [default: <LEDGER>/accounts]

        --accounts-db-skip-shrink
            Enables faster starting of ledger-tool by skipping shrink. This option is for use during testing.

        --accounts-db-test-hash-calculation                                      
            Enable hash calculation test

        --accounts-hash-cache-path <PATH>
            Use PATH as accounts hash cache location [default: <LEDGER>/accounts_hash_cache]

        --accounts-index-bins <BINS>
            Number of bins to divide the accounts index into

        --accounts-index-memory-limit-mb <MEGABYTES>
            How much memory the accounts index can consume. If this is exceeded, some account index entries will be
            stored on disk.
        --accounts-index-path <PATH>...
            Persistent accounts-index location. May be specified multiple times. [default: <LEDGER>/accounts_index]

        --allow-dead-slots                                                       
            Output dead slots as well

        --block-verification-method <METHOD>
            Switch transaction scheduling method for verifying ledger entries [default: blockstore-processor] [possible
            values: blockstore-processor, unified-scheduler]
        --debug-key <ADDRESS>...
            Log when transactions are processed that reference the given key(s).

        --disable-accounts-disk-index
            Disable the disk-based accounts index. It is enabled by default. The entire accounts index will be kept in
            memory.
        --enable-extended-tx-metadata-storage
            Include CPI inner instructions, logs, and return data in the historical transaction info stored

        --enable-rpc-transaction-history
            Store transaction info for processed slots into local ledger

        --force-update-to-open
            Allow commands that would otherwise not alter the blockstore to make necessary updates in order to open it

        --geyser-plugin-config <FILE>...
            Specify the configuration file for the Geyser plugin.

        --halt-at-slot <SLOT>                                                    
            Halt processing at the given slot

        --hard-fork <SLOT>...                                                    
            Add a hard fork at this slot

    -h, --help                                                                   
            Prints help information

        --ignore-ulimit-nofile-error
            Allow opening the blockstore to succeed even if the desired open file descriptor limit cannot be configured.
            Use with caution as some commands may run fine with a reduced file descriptor limit while others will not
        --incremental-snapshot-archive-path <DIR>
            Use DIR for separate incremental snapshot location

    -l, --ledger <DIR>
            Use DIR as ledger location [default: ledger]

        --limit-load-slot-count-from-snapshot <SLOT>
            For debugging and profiling with large snapshots, artificially limit how many slots are loaded from a
            snapshot.
        --log-messages-bytes-limit <BYTES>
            Maximum number of bytes written to the program log before truncation

        --max-genesis-archive-unpacked-size <NUMBER>
            maximum total uncompressed size of unpacked genesis archive [default: 10485760]

        --no-snapshot
            Do not start from a local snapshot if present

        --os-memory-stats-reporting
            Enable reporting of OS memory statistics.

        --output <FORMAT>
            Return information in specified output format, currently only available for bigtable and program subcommands
            [possible values: json, json-compact]
        --print-accounts-stats
            After verifying the ledger, print some information about the account stores

        --print-bank-hash
            After verifying the ledger, print the working bank's hash

        --record-slots <FILENAME>
            Record slots to a file [default: slots.json]

        --record-slots-config <record_slots_config>...
            In addition to the bank hash, optionally include accounts and/or transactions details for the slot [possible
            values: accounts, tx]
        --run-final-accounts-hash-calculation
            After 'verify' completes, run a final accounts hash calculation. Final hash calculation could race with
            accounts background service tasks and assert.
        --skip-poh-verify
            Deprecated, please use --skip-verification. Skip ledger PoH and transaction verification.

        --skip-verification
            Skip ledger PoH and transaction verification.

        --snapshots <DIR>
            Use DIR for snapshot location [default: --ledger value]

        --unified-scheduler-handler-threads <COUNT>
            Change the number of the unified scheduler's transaction execution threads dedicated to each block,
            otherwise calculated as cpu_cores/4 [default: 4]
        --use-snapshot-archives-at-startup <use_snapshot_archives_at_startup>
            At startup, when should snapshot archives be extracted versus using what is already on disk? 
            Specifying "always" will always startup by extracting snapshot archives and disregard any snapshot-related
            state already on disk. Note that starting up from snapshot archives will incur the runtime costs associated
            with extracting the archives and rebuilding the local state. 
            Specifying "never" will never startup from snapshot archives and will only use snapshot-related state
            already on disk. If there is no state already on disk, startup will fail. Note, this will use the latest
            state available, which may be newer than the latest snapshot archive. 
            Specifying "when-newest" will use snapshot-related state already on disk unless there are snapshot archives
            newer than it. This can happen if a new snapshot archive is downloaded while the node is stopped. [default:
            always]  [possible values: always, never, when-newest]
    -v, --verbose
            Show additional information where supported

        --verify-accounts-index
            For debugging and tests on accounts index.

        --verify-slots <FILENAME>
            Verify slots match contents of file [default: slots.json]

        --wal-recovery-mode <MODE>
            Mode to recovery the ledger db write ahead log [possible values: tolerate_corrupted_tail_records,
            absolute_consistency, point_in_time, skip_any_corrupted_record]
        --write-bank-file
            After verifying the ledger, write a file that contains the information that went into computing the
            completed bank's bank hash. The file will be written within <LEDGER_DIR>/bank_hash_details/

comparison - color

old:

new:

[mergify]

Backports to the beta branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule. Exceptions include CI/metrics changes, CLI improvements and documentation updates on a case by case basis.

[CriesofCarrots]

wfm

[steviez]

LGTM, I think this is the default in newer version of CLAP so if we ever get there, we'd have this happen for free.

I imagine that we should keep this PR and #2323 in sync such that they we have same colorization on same versions