Merge pull request #25 from informalsystems/luca_joss/update-yuji-1.7…

….1-namada

Update yuji 1.7.1 namada

.changelog/unreleased/bug-fixes/ibc-relayer/3770-non-utf8-packet-data.md

@@ -0,0 +1,6 @@
+- Allow relaying ICS-04 packets with non-UTF-8 payloads ([\#3770](https://github.com/informalsystems/hermes/issues/3770))
+ Hermes does not assume anymore that an ICS-04 packet data is valid UTF-8,
+ by using the `packet_data_hex` attribute when assembling a packet from events, instead of the deprecated `packet_data` attribute.
+ Relying on the `packet_data` attribute enforces a UTF-8 encoded payload (eg. JSON), disallowing eg. Protobuf-encoded payloads.
+ The `packet_data` atttribute [has been deprecated][0] in favor of `packet_data_hex` since IBC-Go v1.0.0.
+ [0]: https://github.com/cosmos/ibc-go/blob/fadf8f2b0ab184798d021d220d877e00c7634e26/CHANGELOG.md?plain=1#L1417

.changelog/unreleased/improvements/3814-grpc-syncing.md

@@ -0,0 +1,3 @@
+- Add syncing check for gRPC node ([#3814])
+
+[#3814]: https://github.com/informalsystems/ibc-rs/issues/3814

.changelog/unreleased/improvements/3814-status-use-client-latest-height.md

@@ -0,0 +1,3 @@
+- Use the consensus state at client latest height in status CLI ([#3814])
+
+[#3814]: https://github.com/informalsystems/ibc-rs/issues/3814

.changelog/v1.7.2/features/ibc-telemetry/3707-skipped-client-update-metric.md

@@ -0,0 +1,3 @@
+- Added metric `client_updates_skipped` to track the number of client
+ update messages skipped due to the conscensus state existing already.
+ ([\#3707](https://github.com/informalsystems/hermes/issues/3707))

.changelog/v1.7.2/features/ibc-telemetry/3708-add-broadcast-errors.md

@@ -0,0 +1,3 @@
+- Add a new metric `broadcast_errors` which
+ records the number of times a specific error is observed by Hermes when broadcasting transactions
+ ([\#3708](https://github.com/informalsystems/hermes/issues/3708))

.changelog/v1.7.2/summary.md

@@ -0,0 +1,7 @@
+*November 28th, 2023*
+
+This patch release of Hermes adds a metric to improve monitoring errors and one
+to measure the efficiency of the client update skip feature released in patch v1.7.1.
+
+* `broadcast_errors` records the number of times a specific error is observed by Hermes when broadcasting transactions.
+* `client_updates_skipped` records the number of client updates skipped due to the consensus states already existing.

.changelog/v1.7.3/bug-fixes/ibc-relayer-cli/3702-evidence-patch.md

@@ -0,0 +1,3 @@
+- Improve reliability of `evidence` command and fix a bug that was
+ preventing evidence to be reported, as seen on the Gaia RS testnet
+ ([\#3702](https://github.com/informalsystems/hermes/pull/3702))

.changelog/v1.7.3/summary.md

@@ -0,0 +1,5 @@
+*November 29th, 2023*
+
+This release improves the reliability of the `evidence` command and 
+fixes a bug that was preventing evidence to be reported,
+as seen on the Gaia RS testnet.

.changelog/v1.7.4/bug-fixes/ibc-relayer-cli/3697-fix-evidence-report.md

@@ -0,0 +1,4 @@
+- Fix a bug in the `evidence` command which would sometimes
+ prevent the detected misbehaviour evidence from being submitted,
+ instead erroring out with a validator set hash mismatch.
+ ([\#3697](https://github.com/informalsystems/hermes/pull/3697))

.changelog/v1.7.4/bug-fixes/ibc-relayer/3703-avoid-returning-stopped-worker.md

@@ -0,0 +1,3 @@
+- Avoid retrieving a worker which is being removed by the idle worker clean-up
+ process.
+ process ([\#3703](https://github.com/informalsystems/hermes/issues/3703))

.changelog/v1.7.4/bug-fixes/ibc-telemetry/3720-fix-broadcasting-errors-metric.md

@@ -0,0 +1,3 @@
+- Fix the issue where `broadcast_errors` metric would not correctly batch
+ the same errors together.
+ together ([\#3720](https://github.com/informalsystems/hermes/issues/3720))

.changelog/v1.7.4/bug-fixes/ibc-telemetry/3723-fix-backlog-metrics.md

@@ -0,0 +1,4 @@
+- Update the values of `backlog` metrics when clearing packets.
+ Change the `backlog_oldest_timestamp` to `backlog_latest_update_timestamp`
+ which shows the last time the `backlog` metrics have been updated.
+ ([\#3723](https://github.com/informalsystems/hermes/issues/3723))

.changelog/v1.7.4/summary.md

@@ -0,0 +1,9 @@
+*December 15th, 2023*
+
+This release improves the monitoring of Hermes instances by fixing the `broadcast_errors` metric so
+that it correctly batches the same errors together. It also improves the metrics `backlog_*` by
+updating them whenever Hermes queries pending packets.
+
+This release also improves the reliability of the idle worker clean-up and 
+fixes a bug within the `evidence` command which would sometimes prevent
+the misbehaviour evidence from being reported.

.changelog/v1.8.0/features/ibc-integration-test/3455-async-icq-test.md

@@ -0,0 +1,2 @@
+- Add a test for asynchronous Interchain Query relaying
+ ([\#3455](https://github.com/informalsystems/hermes/issues/3455))

.changelog/v1.8.0/features/ibc-integration-test/3778-ordered-channel-timeout.md

@@ -0,0 +1,2 @@
+- Add an ICA test to assert a channel correctly closes after a packet time-outs
+ ([\#3778](https://github.com/informalsystems/hermes/issues/3778))

.changelog/v1.8.0/features/ibc-relayer-cli/3402-lc-refresh.md

@@ -0,0 +1,3 @@
+- Add a `client_refresh_rate` setting to specify the rate at which to
+ refresh clients referencing this chain, relative to its trusting period.
+ ([\#3402](https://github.com/informalsystems/hermes/issues/3402))

.changelog/v1.8.0/features/ibc-relayer-cli/3672-clears-packet-seq.md

@@ -0,0 +1,18 @@
+- Add a `--packet-sequences` flag to the `clear packets`, `tx packet-recv`, and `tx packet-ack` commands.
+ When this flag is specified, these commands will only clear the packets with the specified sequence numbers
+ on the given chain. If not provided, all pending packets will be cleared on both chains, as before.
+
+ This flag takes either a single sequence number or a range of sequences numbers.
+ Each element of the comma-separated list must be either a single sequence number or 
+ a range of sequence numbers.
+
+ Examples:
+ - `10` will clear a single packet with sequence nymber `10`
+ - `1,2,3` will clear packets with sequence numbers `1, 2, 3`
+ - `1..5` will clear packets with sequence numbers `1, 2, 3, 4, 5`
+ - `..5` will clear packets with sequence numbers `1, 2, 3, 4, 5`
+ - `5..` will clear packets with sequence numbers greater than or equal to `5`
+ - `..5,10..20,25,30..` will clear packets with sequence numbers `1, 2, 3, 4, 5, 10, 11, ..., 20, 25, 30, 31, ...`
+ - `..5,10..20,25,30..` will clear packets with sequence numbers `1, 2, 3, 4, 5, 10, 11, ..., 20, 25, 30, 31, ...`
+
+ ([\#3672](https://github.com/informalsystems/hermes/issues/3672))

.changelog/v1.8.0/features/ibc-relayer-cli/3743-query_packets_chunk_size.md

@@ -0,0 +1,6 @@
+- Add a `query_packets_chunk_size` config option and a `--query-
+ packets-chunk-size flag to the `clear packets` CLI to configure how
+ many packets to query at once from the chain when clearing pending
+ packets. Lower this setting if one or more of packets you are
+ trying to clear are huge and make the packet query time out or fail.
+ ([\#3743](https://github.com/informalsystems/hermes/issues/3743))

.changelog/v1.8.0/features/ibc-relayer/3738-dynamic-gas-fees.md

@@ -0,0 +1,12 @@
+- Add a new per-chain configuration table `dynamic_gas_price` which enables
+ querying the current gas price from the chain instead of the static `gas_price`, 
+ when the chain has [EIP-1559][eip]-like dynamic gas price. 
+ The new configuration setting can be configured per-chain as follows:
+ ```toml
+ dynamic_gas_price = { enabled = true, multiplier = 1.1, max = 0.6 }
+ ```
+ At the moment, only chains which support the `osmosis.txfees.v1beta1.Query/GetEipBaseFee`
+ query can be used with dynamic gas price enabled.
+ ([\#3738](https://github.com/informalsystems/hermes/issues/3738))
+
+[eip]: https://metamask.io/1559/

.changelog/v1.8.0/features/ibc-relayer/3766-max-memo-receiver-config.md

@@ -0,0 +1,6 @@
+- Add two new packet configurations:
+ * `ics20_max_memo_size` which filters ICS20 packets with memo
+ field bigger than the configured value
+ * `ics20_max_receiver_size` which filters ICS20 packets with receiver
+ field bigger than the configured value
+ ([\#3766](https://github.com/informalsystems/hermes/issues/3766))

.changelog/v1.8.0/features/ibc-telemetry/3738-dynamic-gas-fees-metrics.md

@@ -0,0 +1,8 @@
+- Add three metrics related to EIP gas price:
+ - `dynamic_gas_queried_fees` contains data on the queried values
+ before applying any filter
+ - `dynamic_gas_queried_success_fees` contains data on the queried
+ values if the query was successful and before applying any filter
+ - `dynamic_gas_paid_fees` contains data on the queried values after
+ applying the `max` filter
+ ([\#3738](https://github.com/informalsystems/hermes/issues/3738))

.changelog/v1.8.0/features/ibc-telemetry/3794-memo-receiver-filter-metric.md

@@ -0,0 +1,3 @@
+- Add a new metric `filtered_packets` which counts the number of
+ packets filtered due to having a memo or receiver field too big
+ ([\#3794](https://github.com/informalsystems/hermes/issues/3794))

.changelog/v1.8.0/improvements/ibc-relayer-cli/3745-compat-0.50.md

@@ -0,0 +1,2 @@
+- Update compatibility check to allow IBC-Go 4.1.1 to 8.x and SDK 0.45.x to 0.50.x.
+ ([\#3745](https://github.com/informalsystems/hermes/issues/3745))

.changelog/v1.8.0/summary.md

@@ -0,0 +1,19 @@
+*January 23rd, 2024*
+
+This v1.8.0 release introduces new features and improvements to Hermes.
+
+One key feature is that Hermes is now compatible with both the legacy `UpgradeProposal` and the newer `MsgIbcSoftwareUpgrade` message when upgrading a chain.
+This allows Hermes to be compatible with ibc-go v8.0.0. The compatibility check that Hermes performs on startup has been updated to reflect this.
+
+Additional configuration settings have been added:
+
+- The new global settings `ics20_max_memo_size` and `ics20_max_receiver_size` allow users to specify a limit for the size of the memo and receiver fields for ICS20 packets. Any packet with either field having a size exceeding the configured values will not be relayed.
+- The new per-chain setting `query_packets_chunk_size` allows users to specify how many packets are queried at once from the chain when clearing pending packets. This is useful to tweak when there are many large pending packets and the RPC endpoints times out or refuses to answer the pending packets query.
+- The new per-chain setting `client_refresh_rate` can be use to specify how often the clients referencing this chain should be refreshed. The rate is expressed as a fraction of the trusting period.
+- The new per-chain setting `dynamic_gas_price` can be enabled to have the relayer query for and use a dynamic gas price instead of using the static `gas_price` specified in the config. This should only be used for chains which have a [EIP-1559][eip-1559]-like fee market enabled and support the `osmosis.txfees.v1beta1.Query/GetEipBaseFee` gRPC query.
+
+Telemetry now features new metrics:
+- Monitoring the ICS20 packets filtered due to the memo and/or receiver field size exceeding the configured limits.
+- Monitoring the distribution of dynamic gas fees queried from the chain, if enabled.
+
+[eip-1559]: https://metamask.io/1559/

.github/ISSUE_TEMPLATE/release-template.md

@@ -20,3 +20,4 @@ v without deliberation
 - [ ] Create a new release in the changelog, using [`unclog`](https://github.com/informalsystems/unclog)
  - If doing a release candidate (`rc`) version, then skip the `unclog release` step
 - [ ] Reassign unfinished issues of previous milestone to the next milestone
+- [ ] Notify the comms team about the pending new release and [prepare a message](https://www.notion.so/informalsystems/Communications-pipeline-b8c0eeb71dc24203a048fa6ccf189e1a?pvs=4) to announce it

.github/codespell/codespell.ini

@@ -0,0 +1,3 @@
+[codespell]
+skip = *.js,*.ts,*.css,*.svg,*.html,*.json,./target,./tools/integration-test/data,./tools/check-guide/target,./ci/misbehaviour/data
+ignore-words = .github/codespell/words.txt

.github/codespell/words.txt

@@ -0,0 +1,7 @@
+crate
+shs
+ser
+numer
+nam
+inout
+wast

.github/workflows/audit.yaml

@@ -10,7 +10,7 @@ jobs:
  steps:
  - uses: actions/checkout@v4
  - name: Cache cargo bin
- uses: actions/cache@v3
+ uses: actions/cache@v4
  with:
  path: ~/.cargo/bin
  key: ${{ runner.os }}-cargo-audit-v0.11.2

.github/workflows/cargo-doc.yaml

@@ -4,7 +4,17 @@ on:
  push:
  branches:
  - master
- pull_request: {}
+ paths:
+ - .github/workflows/cargo-doc.yml
+ - Cargo.toml
+ - Cargo.lock
+ - crates/**
+ pull_request:
+ paths:
+ - .github/workflows/cargo-doc.yml
+ - Cargo.toml
+ - Cargo.lock
+ - crates/**
 
 # Cancel previous runs of this workflow when a new commit is added to the PR, branch or tag
 concurrency:
@@ -18,7 +28,7 @@ jobs:
  - uses: actions/checkout@v4
  - uses: actions-rs/toolchain@v1
  with:
- toolchain: nightly-2023-07-13
+ toolchain: nightly
  override: true
  - name: Install Protoc
  uses: heliaxdev/setup-protoc@v2

.github/workflows/codespell.yml

@@ -0,0 +1,22 @@
+name: Codespell
+on:
+ pull_request:
+ push:
+ branches: master
+
+# Cancel previous runs of this workflow when a new commit is added to the PR, branch or tag
+concurrency:
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ codespell:
+ name: Check spelling
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: codespell-project/actions-codespell@v2
+ with:
+ skip: '*.js,*.ts,*.css,*.svg,*.html,*.json,./target,./tools/integration-test/data,./tools/check-guide/target,./ci/misbehaviour/data'
+ ignore_words_file: .github/codespell/words.txt
+

.github/workflows/docker.yml

@@ -13,13 +13,9 @@ env:
 
 jobs:
  docker-build:
- runs-on: ubuntu-latest
+ runs-on: macos-14
  strategy:
  fail-fast: false
- matrix:
- platform:
- - linux/amd64
- - linux/arm64
  steps:
  - name: Checkout
  uses: actions/checkout@v4
@@ -29,91 +25,43 @@ jobs:
  uses: docker/metadata-action@v5
  with:
  images: ${{ env.REGISTRY_IMAGE }}
-
- - name: Set up QEMU
- uses: docker/setup-qemu-action@v3
+ tags: |
+  type=ref,event=tag
+  type=ref,event=workflow_dispatch
 
  - name: Set up Docker Buildx
  id: buildx
  uses: docker/setup-buildx-action@v3
+ with:
+ platforms: linux/amd64,linux/arm64
 
  - name: Login to Docker Hub
  uses: docker/login-action@v3
  with:
  username: ${{ secrets.DOCKER_HUB_USERNAME }}
  password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
 
- - name: Build and push by digest
+ - name: Login to GitHub Container Registry
+ uses: docker/login-action@v3
+ with:
+ registry: ghcr.io
+ username: ${{ github.repository_owner }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Build and push
  id: build
  uses: docker/build-push-action@v5
  with:
  context: .
  file: ./ci/release/hermes.Dockerfile
- platforms: ${{ matrix.platform }}
- labels: ${{ steps.meta.outputs.labels }}
- outputs: type=image,name=${{ env.REGISTRY_IMAGE }},push-by-digest=true,name-canonical=true,push=true
+ platforms: linux/amd64,linux/arm64
+ push: true
  cache-from: type=gha
  cache-to: type=gha,mode=max
-
- - name: Export digest
- run: |
- mkdir -p /tmp/digests
- digest="${{ steps.build.outputs.digest }}"
- touch "/tmp/digests/${digest#sha256:}"
-
- - name: Upload digest
- uses: actions/upload-artifact@v3
- with:
- name: digests
- path: /tmp/digests/*
- if-no-files-found: error
- retention-days: 1
-
- docker-merge:
- runs-on: ubuntu-latest
- needs:
- - docker-build
- steps:
- - name: Download digests
- uses: actions/download-artifact@v3
- with:
- name: digests
- path: /tmp/digests
-
- - name: Set up Docker Buildx
- uses: docker/setup-buildx-action@v3
-
- - name: Docker meta
- id: meta
- uses: docker/metadata-action@v5
- with:
- images: ${{ env.REGISTRY_IMAGE }}
-
- - name: Login to Docker Hub
- uses: docker/login-action@v3
- with:
- username: ${{ secrets.DOCKER_HUB_USERNAME }}
- password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
-
- - name: Create manifest list and push
- working-directory: /tmp/digests
- run: |
- docker buildx imagetools create --tag ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }} \
- $(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *)
+ tags: |
+ ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
+ ghcr.io/${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
 
  - name: Inspect image
  run: |
  docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
-
- - name: Login to GitHub Container Registry
- uses: docker/login-action@v3
- with:
- registry: ghcr.io
- username: ${{ github.repository_owner }}
- password: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Push image to GHCR
- run: |
- docker buildx imagetools create \
- --tag ghcr.io/${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }} \
- ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}