Endorser support for updating DID endpoints on ledger

[TheTechmage]

As stated in #1447, the changes by Harsh fix an issue where ACA-Py could not publish ATTRIB records because they were not signed by the endorser. We need those changes in for 0.7.4 if possible, but PR #1616 has not seen any updates and has merge conflicts with the latest changes in main. I have attempted to fix those merge conflicts and make the requested changes from @ianco in that PR, as well as fix all the unit test failures.

This closes #1616. I am not sure if more work is needed for #1447.

[codecov-commenter]

Codecov Report

Merging #1696 (459b72f) into main (6b1066a) will decrease coverage by 0.20%.
The diff coverage is 42.06%.

@@            Coverage Diff             @@
##             main    #1696      +/-   ##
==========================================
- Coverage   95.50%   95.29%   -0.21%     
==========================================
  Files         528      528              
  Lines       32766    32878     +112     
==========================================
+ Hits        31293    31331      +38     
- Misses       1473     1547      +74     

[ianco]

Some black formatting issues in aries_cloudagent/wallet/routes.py

[dbluhm]

Evidently new versions of black behave slightly differently. I've updated the pre-commit hook to point to the most recent version that is now being used by the format checker.

[TheTechmage]

I took a peek at that integration test failure. Looks like the error is from this code which is part of these test cases. I'm not quite sure how it's supposed to work, but I'll do my best to figure it out and provide a fix.

[ianco]

I think the error is in the order of the operations in the test:

      Given we have "2" agents
         | name  | role     | capabilities        |
         | Acme  | endorser | <Acme_capabilities> |
         | Bob   | author   | <Bob_capabilities>  |
      And "Acme" and "Bob" have an existing connection
      When "Acme" has a DID with role "ENDORSER"
      And "Bob" has a DID with role "AUTHOR"
      And "Acme" connection has job role "TRANSACTION_ENDORSER"
      And "Bob" connection has job role "TRANSACTION_AUTHOR"
      And "Bob" connection sets endorser info
      And "Bob" authors a schema transaction with <Schema_name>

The "Bob has a DID ..." line should happen after the connection endorser settings.

[ianco]

Like this:

      Given we have "2" agents
         | name  | role     | capabilities        |
         | Acme  | endorser | <Acme_capabilities> |
         | Bob   | author   | <Bob_capabilities>  |
      And "Acme" and "Bob" have an existing connection
      When "Acme" has a DID with role "ENDORSER"
      And "Acme" connection has job role "TRANSACTION_ENDORSER"
      And "Bob" connection has job role "TRANSACTION_AUTHOR"
      And "Bob" connection sets endorser info
      And "Bob" has a DID with role "AUTHOR"
      And "Bob" authors a schema transaction with <Schema_name>
...

... now that the DID creation requires an endorser.

[TheTechmage]

That's what I'm thinking as well. Running the tests locally after making the changes A) to see if I can run the integration tests and B) to see if it fixes the problem. I really like how these tests are structured in that they're more "human readable" than actual code.

[TheTechmage]

I'm seeing tests that don't exist in the failed CI, so I'm assuming that the answer is yes to both of those! Pushing the changes now. Thank you @ianco for the help. :)

[ianco]

./run_bdd -t @T003.1-RFC0586 works

However BDD_EXTRA_AGENT_ARGS="{\"wallet-type\":\"askar\"}" ./run_bdd -t @T003.1-RFC0586 fils with an exception:

Shutting down agent ...
Aries      |   File "/home/indy/aries_cloudagent/wallet/routes.py", line 559, in promote_wallet_public_did
Aries      |     endorser_did=endorser_did,
Aries      |   File "/home/indy/aries_cloudagent/wallet/askar.py", line 488, in set_did_endpoint
Aries      |     endorser_did=endorser_did,
Aries      |   File "/home/indy/aries_cloudagent/ledger/indy_vdr.py", line 946, in update_endpoint_for_did
Aries      |     attrib_req = await ledger.append_request_endorser(
Aries      | AttributeError: module 'indy_vdr.ledger' has no attribute 'append_request_endorser'
Aries      | 2022-03-29 15:27:37,388 aiohttp.server ERROR Error handling request
Aries      | Traceback (most recent call last):
Aries      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
Aries      |     resp = await request_handler(request)
Aries      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aiohttp/web_app.py", line 504, in _handle
Aries      |     resp = await handler(request)
Aries      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 117, in impl
Aries      |     return await handler(request)
Aries      |   File "/home/indy/aries_cloudagent/admin/server.py", line 159, in ready_middleware
Aries      |     return await handler(request)
Aries      |   File "/home/indy/aries_cloudagent/admin/server.py", line 196, in debug_middleware
Aries      |     return await handler(request)
Aries      |   File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aiohttp_apispec/middlewares.py", line 45, in validation_middleware
Aries      |     return await handler(request)
Aries      |   File "/home/indy/aries_cloudagent/admin/server.py", line 383, in setup_context
Aries      |     return await task
Aries      |   File "/home/indy/aries_cloudagent/wallet/routes.py", line 437, in wallet_set_public_did
Aries      |     context.profile, context, context.session, did, write_ledger=write_ledger
Aries      |   File "/home/indy/aries_cloudagent/wallet/routes.py", line 559, in promote_wallet_public_did
Aries      |     endorser_did=endorser_did,
Aries      |   File "/home/indy/aries_cloudagent/wallet/askar.py", line 488, in set_did_endpoint
Aries      |     endorser_did=endorser_did,
Aries      |   File "/home/indy/aries_cloudagent/ledger/indy_vdr.py", line 946, in update_endpoint_for_did
Aries      |     attrib_req = await ledger.append_request_endorser(
Aries      | AttributeError: module 'indy_vdr.ledger' has no attribute 'append_request_endorser'

[TheTechmage]

Currently running into an issue where a couple of tests in @T003-RFC0586 are complaining about not having an endorser. Specifically the two multi-tenant ones (last two examples). Currently trying to figure out why those would error out.

[TheTechmage]

Looking at that test, it appears to be skipped during GitHub actions anyways. I'm not sure how critical it is to try and fix it at this time.

[ianco]

Looking at that test, it appears to be skipped during GitHub actions anyways. I'm not sure how critical it is to try and fix it at this time.

Github actions runs a subset of tests because if you try to run too many (i.e. all of them) it seems to blow up.
I'm just taking a look at this failing test now ...

[ianco]

Weird that something is failing with multi-tenancy, I suggest to log a separate issue. We can merge this PR and then deal with the multi-tenant issue separately.

[dbluhm]

FWIW 🙂