Queries on new modules not working after on-chain upgrade

[RiccardoM]

Summary of Bug

After an on-chain upgrade, the queries on newly added modules are not working.

Version

v0.40.0-rc6

Steps to Reproduce

While trying to debug #8265, I stumbled on a bug that makes it impossible to run CLI & REST queries on a module that was added during an on-chain upgrade.

Here are the steps I've done to reproduce this bug:

1. Run a chain without a specific module (reports).
2. Create an on-chain upgrade to add such module.
3. Wait until the upgrade height comes, and the upgrade is performed.
4. Try performing a transaction using the newly added module.
5. Try queries the newly added module.

Everything works properly until point 5. When querying the data the result is always empty (like nothing exists), even if the transaction performed at step 4 is successful.

Specific example

The test I've made is based on the following branches of our project:

• riccardo/no-module, which does not have one particular module (reports) installed
• riccardo/with-module which does have such module installed

The precise steps I've taken are:

1. Start a chain based on no-module and create a post using the following transaction:

   Post creation transaction

     $ cosmovisor tx posts create 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08 Test --from jack --chain-id=testchain --yes --broadcast-mode=block

2. Query the posts to make sure that the post was created properly, and to get its id (which is generated while inserting the post):

   Query posts

   $ cosmovisor q posts posts
   pagination:
     next_key: null
     total: "0"
   posts:
   - children: []
     poll_answers: []
     post:
       allows_comments: true
       attachments: []
       created: "2021-01-07T07:42:42.793985778Z"
       creator: desmos13yp2fq3tslq6mmtq4628q38xzj75ethzela9uu
       last_edited: "0001-01-01T00:00:00Z"
       message: Test
       optional_data: []
       parent_id: ""
       poll_data: null
       post_id: 158bc7b14484163dc146a7ff6efdf97d68f04a5dc1552350008f64e1bcc7237d
       subspace: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
     reactions: []

3. Submit an upgrade proposal for height 50 so that we can switch the daemon version to be based on the with-module branch.

4. Wait until the upgrade has been completed successfully.

5. Query the posts to make sure that everything is still there:

   Query posts

   $ cosmovisor q posts posts
   pagination:
     next_key: null
     total: "0"
   posts:
   - children: []
     poll_answers: []
     post:
       allows_comments: true
       attachments: []
       created: "2021-01-07T07:42:42.793985778Z"
       creator: desmos13yp2fq3tslq6mmtq4628q38xzj75ethzela9uu
       last_edited: "0001-01-01T00:00:00Z"
       message: Test
       optional_data: []
       parent_id: ""
       poll_data: null
       post_id: 158bc7b14484163dc146a7ff6efdf97d68f04a5dc1552350008f64e1bcc7237d
       subspace: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
     reactions: []

6. Try creating a post report:

   Create post report

   cosmovisor tx reports create 158bc7b14484163dc146a7ff6efdf97d68f04a5dc1552350008f64e1bcc7237d spam spam --from jack --yes --chain-id testchain

7. Try querying the newly inserted report:

   Query the report for the post

   $ cosmovisor q reports post 158bc7b14484163dc146a7ff6efdf97d68f04a5dc1552350008f64e1bcc7237d
   reports: []

As you can see, the returned reports are empty.

Interesting fact

While trying to sorting this out, I have also discovered that the created report is indeed stored properly, and it looks like there's something wrong with the gRPC queries somehow.

We can see that the report is stored properly because when a report is created, this is the code that is called:

// SaveReport allows to save the given report inside the current context.
// It assumes that the given report has already been validated.
// If the same report has already been inserted, nothing will be changed.
func (k Keeper) SaveReport(ctx sdk.Context, report types.Report) error {
  store := ctx.KVStore(k.storeKey)
  key := types.ReportStoreKey(report.PostId)

  // Get the list of reports related to the given postID
  reports := types.MustUnmarshalReports(store.Get(key), k.cdc)

  // Append the given report
  newSlice, appended := types.AppendIfMissing(reports, report)
  if !appended {
    return sdkerrors.Wrap(sdkerrors.ErrInvalidRequest, "report already exists")
  }

  store.Set(key, types.MustMarshalReports(newSlice, k.cdc))
  return nil
}

As you can see, if the same exact report already exists inside the store, an error is returned. So, what I did is I tried creating an exact copy of the first report I created after the upgrade by running the same exact command:

Create post report

cosmovisor tx reports create 158bc7b14484163dc146a7ff6efdf97d68f04a5dc1552350008f64e1bcc7237d spam spam --from jack --yes --chain-id testchain

This, as expected, this returned an error:

{"height":"68","txhash":"63A2F960F3A39C6D179917B51BEEF3DE73FFE7A51451F33B709345D8D1D0FF7F","codespace":"sdk","code":18,"data":"","raw_log":"failed to execute message; message index: 0: report already exists: invalid request","logs":[],"info":"","gas_wanted":"200000","gas_used":"41002","tx":null,"timestamp":""}

At the same time, this is the method that is called when I try to query the reports associated with a post:

// GetPostReports returns the list of reports associated with the given postID.
// If no report is associated with the given postID the function will returns an empty list.
func (k Keeper) GetPostReports(ctx sdk.Context, postID string) []types.Report {
  store := ctx.KVStore(k.storeKey)
  return types.MustUnmarshalReports(store.Get(types.ReportStoreKey(postID)), k.cdc)
}

As you can see, this code is also used inside SaveReport, and it returns a non-empty slice. At the same time, when querying the post reports, the returned slice is empty. This is particularly strange.

Note. Please note that I have tested the creation of a report and the query of all the reports associated with a post on a new chain that has such module installed from the beginning. In that case, all the reports were returned properly.

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[RiccardoM]

Looks like this is closed thanks to the solution provided by @aaronc inside #8265 with his comment