registration.py create_cli_registration failed with error "InvalidAppId"

[gdhuper]

Information

• Onefuzz version: 2.31.0
• OS: Windows 10

Provide detailed reproduction steps (if any)

1. az login
2. Create new cli registration with:
   python registration.py create_cli_registration <fuzzing_instance> <subscription_id> --registration_name <sp_name>

Expected result

Successful new cli registration.

INFO:client_id: <CLIENT_ID
INFO:client_secret: <SECRET>
{
"@odata.context": "https://graph.microsoft.com/v1.0/$metadata#servicePrincipals('ea84fc27-f6f2-4520-8aca-ef37dbec4a45')/appRoleAssignedTo/$entity",
"@odata.id": "https://graph.microsoft.com/v2/directoryObjects/$/Microsoft.DirectoryServices.ServicePrincipal('')/appRoleAssignedTo/",
"appRoleId": "",
"createdDateTime": "2021-06-15T16:36:39.2994863Z",
"deletedDateTime": null,
"id": "KVHhsJkbf6i90E",
"principalDisplayName": "msazure_rescue_dns",
"principalId": "ea84dsf-dfdf-fsfsfa-ef37dbec4a45",
"principalType": "ServicePrincipal",
"resourceDisplayName": "fuzzingprod01",
"resourceId": "36fsdd-16sd-4ba-b209-e67c266671"
} 

Actual result

What is the actual result of the above steps?

INFO:Updating application registration
INFO:retrieving the application registration vs_ide_cpp
INFO:No existing registration found. creating a new one
INFO:creating service principal
WARNING:failed 'authorize application': request did not succeed: HTTP 400 - {"error":{"code":"InvalidAppId","message":"Property PreAuthorizedApplication references applications cffc04f9-86e4-45b2-8906-93e7bf86e3b8 that cannot be found.","innerError":{"date":"2021-08-23T18:38:24","request-id":"479aec37-b74e-4864-9dd8-40bb29be2098","client-request-id":"479aec37-b74e-4864-9dd8-40bb29be2098"}}}
Traceback (most recent call last):
  File "C:\Users\Desktop\2.31.0\registration.py", line 667, in <module>
    main()
  File "C:\Users\Desktop\2.31.0\registration.py", line 649, in main
    create_and_display_registration(
  File "C:\Users\Desktop\2.31.0\registration.py", line 407, in create_and_display_registration
    application_info = register_application(
  File "C:\Users\Desktop\2.31.0\registration.py", line 151, in register_application
    app = create_application_registration(
  File "C:\Users\Desktop\2.31.0\registration.py", line 268, in create_application_registration
    authorize_application(UUID(registered_app.app_id), UUID(app.app_id))
  File "C:\Users\Desktop\2.31.0\registration.py", line 394, in authorize_application
    retry(add_preauthorized_app, "authorize application")
  File "C:\Users\Desktop\2.31.0\registration.py", line 111, in retry
    time.sleep(wait_duration)

[bmc-msft]

Given the traceback ends at time.sleep, did you stop the deployment after seeing the warning, or did python generate an exception there?

[gdhuper]

I stopped the deployment.

[bmc-msft]

While the logging could be more clear, the warning is informing the user of a known issue that AAD object creation takes a while to propagate.

As such, we automatically retry the commands with a delay between each iteration. You can see this in the retry command in the callback.

[chkeita]

Did you manually delete an application registration before executing this command?
This looks like a corrupted configuration.
Can you check the application manifest of your deployment registration in the section preAuthorizedApplications?
if there is an entry with "appId": "cffc04f9-86e4-45b2-8906-93e7bf86e3b8" remove it and try again.

[gdhuper]

Yes, there is an entry with "appId": "cffc04f9-86e4-45b2-8906-93e7bf86e3b8". Let me remove it and try again.

[gdhuper]

Retried after removing the entry and I got a new error:

python registration.py create_cli_registration fuzzingprod01 <subscription_id> --registration_name vside_cpp
INFO:Updating application registration
INFO:retrieving the application registration vside_cpp
INFO:No existing registration found. creating a new one
INFO:creating service principal
Traceback (most recent call last):
  File "C:\Users\gursing\Desktop\2.13.0\registration.py", line 667, in <module>
    main()
  File "C:\Users\gursing\Desktop\2.13.0\registration.py", line 649, in main
    create_and_display_registration(
  File "C:\Users\gursing\Desktop\2.13.0\registration.py", line 407, in create_and_display_registration
    application_info = register_application(
  File "C:\Users\gursing\Desktop\2.13.0\registration.py", line 151, in register_application
    app = create_application_registration(
  File "C:\Users\gursing\Desktop\2.13.0\registration.py", line 269, in create_application_registration
    assign_app_role(
  File "C:\Users\gursing\Desktop\2.13.0\registration.py", line 528, in assign_app_role
    raise Exception("scaleset service principal not found")
Exception: scaleset service principal not found

[chkeita]

This issue was caused by multiple corrupted entries in the manifest. This is most likely because an application registration was deleted.
as a mitigation i added in #1175 a logic to remove the invalid entry when creating the registration