Merge branch 'master' into master

.changelog/unreleased/bug-fixes/ibc-relayer/3770-non-utf8-packet-data.md

@@ -0,0 +1,6 @@
+- Allow relaying ICS-04 packets with non-UTF-8 payloads ([\#3770](https://github.com/informalsystems/hermes/issues/3770))
+  Hermes does not assume anymore that an ICS-04 packet data is valid UTF-8,
+  by using the `packet_data_hex` attribute when assembling a packet from events, instead of the deprecated `packet_data` attribute.
+  Relying on the `packet_data` attribute enforces a UTF-8 encoded payload (eg. JSON), disallowing eg. Protobuf-encoded payloads.
+  The `packet_data` atttribute [has been deprecated][0] in favor of `packet_data_hex` since IBC-Go v1.0.0.
+  [0]: https://github.com/cosmos/ibc-go/blob/fadf8f2b0ab184798d021d220d877e00c7634e26/CHANGELOG.md?plain=1#L1417

.changelog/unreleased/bug-fixes/ibc-relayer/3831-better-compat-check.md

@@ -0,0 +1,2 @@
+- Improve reliability of compatibility check and fix parsing of expected modules
+  versions ([\#3831](https://github.com/informalsystems/hermes/issues/3831))

.changelog/unreleased/features/ibc-relayer/3811-relayer-memo-overwrite.md

@@ -0,0 +1,3 @@
+- Add a per-chain configuration `memo_overwrite` allowing users
+  to overwite the relayer memo used for each transaction
+  ([\#3811](https://github.com/informalsystems/hermes/issues/3811))

.changelog/unreleased/features/ibc-telemetry/3845-add-simulate-errors-metric.md

@@ -0,0 +1,11 @@
+- Added a new Prometheus metric `simulate_errors` for tracking when a transaction simulation fails, with the following labels: 
+  * `recoverable` (can the execution continue if this happened?) 
+  * `account` (account from which the tx was sent) 
+  * `error_description` (description of the error) 
+  ([\#3845](https://github.com/informalsystems/hermes/issues/3845))
+
+  ```
+  # HELP simulate_errors_total Number of errors observed by Hermes when simulating a Tx
+  # TYPE simulate_errors_total counter
+  simulate_errors_total{account="osmo17ndx5qfku28ymxgmq6zq4a6d02dvpfjjul0hyh",error_description="Unknown error",recoverable="false",service_name="unknown_service",otel_scope_name="hermes",otel_scope_version=""} 4
+  ```

.changelog/unreleased/improvements/3814-grpc-syncing.md

@@ -0,0 +1,3 @@
+- Add syncing check for gRPC node ([#3814])
+
+[#3814]: https://github.com/informalsystems/ibc-rs/issues/3814

.changelog/unreleased/improvements/3814-status-use-client-latest-height.md

@@ -0,0 +1,3 @@
+- Use the consensus state at client latest height in status CLI ([#3814])
+
+[#3814]: https://github.com/informalsystems/ibc-rs/issues/3814

.changelog/unreleased/improvements/ibc-relayer/3540-ordered-channels-resilience.md

@@ -0,0 +1,6 @@
+- Improve resilience when relaying on ordered channels. 
+  When relaying packets on an ordered channel, Hermes will now attempt
+  to detect whether the next message to send has the sequence number
+  expected on that channel. If there is a mismatch, then Hermes will trigger a packet
+  clear on the channel to unblock it before resuming operations on that channel.
+  ([\#3540](https://github.com/informalsystems/hermes/issues/3540))

.changelog/v1.8.0/features/ibc-relayer/3738-dynamic-gas-fees.md

@@ -0,0 +1,12 @@
+- Add a new per-chain configuration table `dynamic_gas_price` which enables
+  querying the current gas price from the chain instead of the static `gas_price`, 
+  when the chain has [EIP-1559][eip]-like dynamic gas price. 
+  The new configuration setting can be configured per-chain as follows:
+  ```toml
+  dynamic_gas_price = { enabled = true, multiplier = 1.1, max = 0.6 }
+  ```
+  At the moment, only chains which support the `osmosis.txfees.v1beta1.Query/GetEipBaseFee`
+  query can be used with dynamic gas price enabled.
+  ([\#3738](https://github.com/informalsystems/hermes/issues/3738))
+
+[eip]: https://metamask.io/1559/

.changelog/v1.8.0/features/ibc-telemetry/3738-dynamic-gas-fees-metrics.md

@@ -0,0 +1,8 @@
+- Add three metrics related to EIP gas price:
+    - `dynamic_gas_queried_fees` contains data on the queried values
+      before applying any filter
+    - `dynamic_gas_queried_success_fees` contains data on the queried
+      values if the query was successful and before applying any filter
+    - `dynamic_gas_paid_fees` contains data on the queried values after
+      applying the `max` filter
+  ([\#3738](https://github.com/informalsystems/hermes/issues/3738))

.changelog/v1.8.0/features/ibc-telemetry/3794-memo-receiver-filter-metric.md

@@ -0,0 +1,3 @@
+- Add a new metric `filtered_packets` which counts the number of
+  packets filtered due to having a memo or receiver field too big
+  ([\#3794](https://github.com/informalsystems/hermes/issues/3794))

.changelog/v1.8.0/summary.md

@@ -0,0 +1,19 @@
+*January 23rd, 2024*
+
+This v1.8.0 release introduces new features and improvements to Hermes.
+
+One key feature is that Hermes is now compatible with both the legacy `UpgradeProposal` and the newer `MsgIbcSoftwareUpgrade` message when upgrading a chain.
+This allows Hermes to be compatible with ibc-go v8.0.0. The compatibility check that Hermes performs on startup has been updated to reflect this.
+
+Additional configuration settings have been added:
+
+- The new global settings `ics20_max_memo_size` and `ics20_max_receiver_size` allow users to specify a limit for the size of the memo and receiver fields for ICS20 packets. Any packet with either field having a size exceeding the configured values will not be relayed.
+- The new per-chain setting `query_packets_chunk_size` allows users to specify how many packets are queried at once from the chain when clearing pending packets. This is useful to tweak when there are many large pending packets and the RPC endpoints times out or refuses to answer the pending packets query.
+- The new per-chain setting `client_refresh_rate` can be use to specify how often the clients referencing this chain should be refreshed. The rate is expressed as a fraction of the trusting period.
+- The new per-chain setting `dynamic_gas_price` can be enabled to have the relayer query for and use a dynamic gas price instead of using the static `gas_price` specified in the config. This should only be used for chains which have a [EIP-1559][eip-1559]-like fee market enabled and support the `osmosis.txfees.v1beta1.Query/GetEipBaseFee` gRPC query.
+
+Telemetry now features new metrics:
+- Monitoring the ICS20 packets filtered due to the memo and/or receiver field size exceeding the configured limits.
+- Monitoring the distribution of dynamic gas fees queried from the chain, if enabled.
+
+[eip-1559]: https://metamask.io/1559/

.github/ISSUE_TEMPLATE/release-template.md

@@ -20,3 +20,4 @@ v                     without deliberation
 - [ ] Create a new release in the changelog, using [`unclog`](https://github.com/informalsystems/unclog)
   - If doing a release candidate (`rc`) version, then skip the `unclog release` step
 - [ ] Reassign unfinished issues of previous milestone to the next milestone
+- [ ] Notify the comms team about the pending new release and [prepare a message](https://www.notion.so/informalsystems/Communications-pipeline-b8c0eeb71dc24203a048fa6ccf189e1a?pvs=4) to announce it

.github/workflows/audit.yaml

@@ -10,7 +10,7 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - name: Cache cargo bin
-        uses: actions/cache@v3
+        uses: actions/cache@v4
         with:
           path: ~/.cargo/bin
           key: ${{ runner.os }}-cargo-audit-v0.11.2

.github/workflows/cargo-doc.yaml

@@ -28,7 +28,7 @@ jobs:
       - uses: actions/checkout@v4
       - uses: actions-rs/toolchain@v1
         with:
-          toolchain: nightly-2023-07-13
+          toolchain: nightly
           override: true
 
       - name: Build API documentation

.github/workflows/docker.yml

@@ -13,15 +13,9 @@ env:
 
 jobs:
   docker-build:
-    runs-on: ubuntu-latest
+    runs-on: macos-14
     strategy:
       fail-fast: false
-      matrix:
-        platform:
-          - id: linux/amd64
-            name: amd64
-          - id: linux/arm64
-            name: arm64
     steps:
       - name: Checkout
         uses: actions/checkout@v4
@@ -31,92 +25,43 @@ jobs:
         uses: docker/metadata-action@v5
         with:
           images: ${{ env.REGISTRY_IMAGE }}
-
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v3
+          tags: |
+            type=ref,event=tag
+            type=ref,event=workflow_dispatch
 
       - name: Set up Docker Buildx
         id: buildx
         uses: docker/setup-buildx-action@v3
+        with:
+          platforms: linux/amd64,linux/arm64
 
       - name: Login to Docker Hub
         uses: docker/login-action@v3
         with:
           username: ${{ secrets.DOCKER_HUB_USERNAME }}
           password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
 
-      - name: Build and push by digest
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Build and push
         id: build
         uses: docker/build-push-action@v5
         with:
           context: .
           file: ./ci/release/hermes.Dockerfile
-          platforms: ${{ matrix.platform.id }}
-          labels: ${{ steps.meta.outputs.labels }}
-          outputs: type=image,name=${{ env.REGISTRY_IMAGE }},push-by-digest=true,name-canonical=true,push=true
+          platforms: linux/amd64,linux/arm64
+          push: true
           cache-from: type=gha
           cache-to: type=gha,mode=max
-
-      - name: Export digest
-        run: |
-          mkdir -p /tmp/digests
-          digest="${{ steps.build.outputs.digest }}"
-          touch "/tmp/digests/${digest#sha256:}"
-
-      - name: Upload digest
-        uses: actions/upload-artifact@v4
-        with:
-          name: digests-${{ matrix.platform.name }}
-          path: /tmp/digests/*
-          if-no-files-found: error
-          retention-days: 1
-
-  docker-merge:
-    runs-on: ubuntu-latest
-    needs:
-      - docker-build
-    steps:
-      - name: Download digests
-        uses: actions/download-artifact@v4
-        with:
-          pattern: digests-*
-          merge-multiple: true
-          path: /tmp/digests
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v5
-        with:
-          images: ${{ env.REGISTRY_IMAGE }}
-
-      - name: Login to Docker Hub
-        uses: docker/login-action@v3
-        with:
-          username: ${{ secrets.DOCKER_HUB_USERNAME }}
-          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
-
-      - name: Create manifest list and push
-        working-directory: /tmp/digests
-        run: |
-          docker buildx imagetools create --tag ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }} \
-            $(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *)
+          tags: |
+            ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
+            ghcr.io/${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
 
       - name: Inspect image
         run: |
           docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}
-
-      - name: Login to GitHub Container Registry
-        uses: docker/login-action@v3
-        with:
-          registry: ghcr.io
-          username: ${{ github.repository_owner }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Push image to GHCR
-        run: |
-          docker buildx imagetools create \
-            --tag ghcr.io/${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }} \
-            ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}