Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

got HTTP 400 error when calling WebSiteManagementClient.web_apps.install_site_extension() #2828

Closed
APAC-DevOps opened this issue Jun 28, 2018 · 13 comments
Labels
App Services customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-author-feedback Workflow: More information is needed from author to address the issue. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team no-recent-activity There has been no recent activity on this issue. RestAPI Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@APAC-DevOps
Copy link

I have a python function which does Azure Web App service deployment along with the extension AspNetCore. If I commented out the lines for installing site extension, my script ran happily; however, with the lines for AspNet Core extension installation, my script can not run successfully.

If I do the manual installation via Azure Portal, then, I can complete the extension installation successfully. I wrote ARM templates with site and extension provision, again, my code runs happily. But calling WebSiteManagementClient.web_apps.install_site_extension() like below doesn't work, and I got
msrestazure.azure_exceptions.CloudError: 400 Client Error: Bad Request for url: https://management.azure.com/subscriptions/XXXXX(sensitive info)/siteextensions/AspNetCoreRuntime?api-version=2016-08-01

can you take a look and see if I missed anything here?

ARM Template
{
"apiVersion": "2015-08-01",
"name": "[parameters('siteName')]",
"type": "Microsoft.Web/sites",
"location": "[parameters('location')]",
"properties": {
"name": "[parameters('siteName')]",
"serverFarmId": "[parameters('appServicePlanId')]"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "web",
"type": "config",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites', parameters('siteName'))]"
],
"properties": {
"phpVersion": "off",
"netFrameworkVersion": "v4.0",
"webSocketsEnabled": true,
"requestTracingEnabled": true,
"httpLoggingEnabled": true,
"logsDirectorySizeLimit": 40,
"detailedErrorLoggingEnabled": true,
"scmType": "LocalGit"
}
},

      {
        "apiVersion": "2016-08-01",
        "name": "AspNetCoreRuntime",
        "type": "siteextensions",
        "dependsOn": [
          "[resourceId('Microsoft.Web/Sites', parameters('siteName'))]"
        ],
        "properties": {}
      }
    ]
  }

Python Code
def deploy_app_service():
print("\nDeploying App Service To Resource Group\n")
AzureServicePrincipalcredentials = ServicePrincipalCredentials(
client_id = my_azure_client_id,
secret = my_azure_secret_id,
tenant = my_azure_tenant_id
)
web_client = WebSiteManagementClient(AzureServicePrincipalcredentials, my_subscription_id)

site_async_operation = web_client.web_apps.create_or_update(
    my_rg_name,
    "jianhuafirstWebApp",
    {
        'location': my_location,
        'server_farm_id': my_service_plan_id,
        'reserved': False,
        'enabled': True,
        'kind': 'app',
        'site_config': {
            "kind": "web",
            "phpVersion": "off",
            "netFrameworkVersion": "v4.0",
            "webSocketsEnabled": True,
            "logsDirectorySizeLimit": 40,
            "requestTracingEnabled": True,
            "httpLoggingEnabled": True,
            "detailedErrorLoggingEnabled": True,
            "scmType": "LocalGit"
        }
    }
)

install site extension

my_site_extension = web_client.web_apps.install_site_extension(
    my_rg_name,
    "jianhuafirstWebApp",
    "AspNetCoreRuntime"
)
@lmazuel
Copy link
Member

lmazuel commented Jun 28, 2018

@yugangw-msft ?

@yugangw-msft
Copy link

got msrestazure.azure_exceptions.CloudError: 400 Client Error: Bad Request for url: https://management.azure.com/subscriptions/XXXXX(sensitive info)/siteextensions/AspNetCoreRuntime?api-version=2016-08-01

Can you capture the error response on the wire and see whether there is any error detail inside?

@APAC-DevOps
Copy link
Author

APAC-DevOps commented Jul 11, 2018

msrestazure.azure_exceptions.CloudError: 400 Client Error: Bad Request for url: https://management.azure.com/subscriptions/_mysubsciptionId_/resourceGroups/agri-api-dev-appservice/providers/Microsoft.Web/sites/agriOUSDevAUE2/siteextensions/Microsoft.ApplicationInsights.AzureWebSites?api-version=2016-08-01

Below is a copy of my code. I can confirm that the Azure WebApp does get provisioned successfully. It is the provision of extension, which failed. I tried to capture error via AzureException, however, it doesn't seem logging anything which is useful for debugging.

import os.path
import json
import sys
from azure.mgmt.web import WebSiteManagementClient
from azure.common.credentials import ServicePrincipalCredentials
from azure.common import AzureException


parameters_common = "./azure/config/parameters-common.json"
with open(parameters_common, 'r') as parameters_common_fd:
    arm_parameters_common = json.load(parameters_common_fd)

parameters_appService_plan = "./azure/config/output-agri-api-dev-appservice.json"
with open(parameters_appService_plan, 'r') as parameters_appService_plan_fd:
    arm_parameters_appService_plan = json.load(parameters_appService_plan_fd)

my_subscription_id = arm_parameters_common['SubscriptionId']
my_azure_client_id = arm_parameters_common['ClientId']
my_azure_secret_id = arm_parameters_common['SecretId']
my_azure_tenant_id = arm_parameters_common['TenantId']

my_location = arm_parameters_appService_plan['location']
my_rg_id = arm_parameters_appService_plan['resource_group_id']
my_rg_name = my_rg_id.split('/')[-1]
my_service_plan_id = arm_parameters_appService_plan['appServicePlanName']

json_output = {} #Initialize a dictory container for JSON data processing

def deploy_app_service():
    print("\nDeploying App Service To Resource Group\n")
    AzureServicePrincipalcredentials = ServicePrincipalCredentials(
        client_id = my_azure_client_id,
        secret = my_azure_secret_id,
        tenant = my_azure_tenant_id
    )
    web_client = WebSiteManagementClient(AzureServicePrincipalcredentials, my_subscription_id)

    site_async_operation = web_client.web_apps.create_or_update(
        my_rg_name,
        "agriOUSDevAUE2",
        {
            'location': my_location,
            'server_farm_id': my_service_plan_id,
            'reserved': False,
            'enabled': True,
            'kind': 'app',
            'site_config': {
                "scmType": "LocalGit"
            }
        }
    )

    # my_site_extension = web_client.web_apps.list_site_extensions(
    #     my_rg_name,
    #     'agriOUSDevAUE'
    # )

    # for each_site in my_site_extension:
    #     print(each_site)
    try:
        my_site_extension = web_client.web_apps.install_site_extension(
            my_rg_name,
            "agriOUSDevAUE2",
            "Microsoft.ApplicationInsights.AzureWebSites"
        )
    except AzureException as e:
        print(e.response)

if __name__ == "__main__":
    deploy_app_service()

@lmazuel
Copy link
Member

lmazuel commented Jul 11, 2018

Hi @APAC-DevOps
To enable wiring log, you need to do this:

web_client.config.enable_http_logger=True

and then use "logging" to show DEBUG logs. You will the HTTP answer with all details.
Thanks,

@AaronTimms
Copy link

I'd try running this without "phpVersion": "off"
I've seen this setting triggering 400 since the latest php version update for Azure

@APAC-DevOps
Copy link
Author

APAC-DevOps commented Jul 17, 2018

I added the line "web_client.config.enable_http_logger=True" to my code, and set my python code's logging level to "debug", below is what I got

(I commented out subscription ID from the output, but apart from that, all of the output from my Mac computer is displayed below)

WARNING:msrest.service_client:Accept header absent and forced to application/json
DEBUG:msrest.pipeline:Configuring request: timeout=100, verify=True, cert=None
DEBUG:msrest.pipeline:Configuring proxies: ''
DEBUG:msrest.pipeline:Evaluate proxies against ENV settings: True
DEBUG:msrest.pipeline:Configuring redirects: allow=True, max=30
DEBUG:msrest.pipeline:Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): management.azure.com:443
DEBUG:urllib3.connectionpool:https://management.azure.com:443 "PUT /subscriptions/subscriptionID/resourceGroups/agri-api-dev-appservice/providers/Microsoft.Web/sites/agriOUSDevAUE2/siteextensions/Microsoft.AspNetCore.AzureAppServices.SiteExtension?api-version=2016-08-01 HTTP/1.1" 400 0
DEBUG:msrest.http_logger:Request URL: 'https://management.azure.com/subscriptions/subscriptionID/resourceGroups/agri-api-dev-appservice/providers/Microsoft.Web/sites/agriOUSDevAUE2/siteextensions/Microsoft.AspNetCore.AzureAppServices.SiteExtension?api-version=2016-08-01'
DEBUG:msrest.http_logger:Request method: 'PUT'
DEBUG:msrest.http_logger:Request headers:
DEBUG:msrest.http_logger: 'User-Agent': 'python/3.6.5 (Darwin-17.6.0-x86_64-i386-64bit) requests/2.19.1 msrest/0.5.0 msrest_azure/0.4.32 websitemanagementclient/0.34.1 Azure-SDK-For-Python'
DEBUG:msrest.http_logger: 'Accept-Encoding': 'gzip, deflate'
DEBUG:msrest.http_logger: 'Accept': 'application/json'
DEBUG:msrest.http_logger: 'Connection': 'keep-alive'
DEBUG:msrest.http_logger: 'Content-Type': 'application/json; charset=utf-8'
DEBUG:msrest.http_logger: 'x-ms-client-request-id': 'ac7a4ff4-898f-11e8-8894-9801a7968dff'
DEBUG:msrest.http_logger: 'accept-language': 'en-US'
DEBUG:msrest.http_logger: 'Content-Length': '0'
DEBUG:msrest.http_logger: 'Authorization': '*****'
DEBUG:msrest.http_logger:Request body:
DEBUG:msrest.http_logger:None
DEBUG:msrest.http_logger:Response status: 400
DEBUG:msrest.http_logger:Response headers:
DEBUG:msrest.http_logger: 'Cache-Control': 'no-cache'
DEBUG:msrest.http_logger: 'Pragma': 'no-cache'
DEBUG:msrest.http_logger: 'Expires': '-1'
DEBUG:msrest.http_logger: 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
DEBUG:msrest.http_logger: 'x-ms-request-id': '74be45e5-50ff-484b-b537-0960813123ee'
DEBUG:msrest.http_logger: 'Set-Cookie': 'ARRAffinity=60ecc41db6b19c150858a0271e02abb08bc4c432f0eeabbaa9ae45d1e3895350;Path=/;HttpOnly;Domain=agriousdevaue2.scm.azurewebsites.net'
DEBUG:msrest.http_logger: 'Server': 'Microsoft-IIS/10.0, Microsoft-IIS/10.0'
DEBUG:msrest.http_logger: 'X-AspNet-Version': '4.0.30319'
DEBUG:msrest.http_logger: 'X-Powered-By': 'ASP.NET, ASP.NET'
DEBUG:msrest.http_logger: 'x-ms-ratelimit-remaining-subscription-writes': '1199'
DEBUG:msrest.http_logger: 'x-ms-correlation-request-id': '99291ad1-ddd2-484f-b7fc-019edab5029f'
DEBUG:msrest.http_logger: 'x-ms-routing-request-id': 'AUSTRALIAEAST:20180717T070442Z:99291ad1-ddd2-484f-b7fc-019edab5029f'
DEBUG:msrest.http_logger: 'X-Content-Type-Options': 'nosniff'
DEBUG:msrest.http_logger: 'Date': 'Tue, 17 Jul 2018 07:04:41 GMT'
DEBUG:msrest.http_logger: 'Content-Length': '0'
DEBUG:msrest.http_logger:Response content:
DEBUG:msrest.http_logger:b''
DEBUG:msrest.exceptions:Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://management.azure.com/subscriptions/14fcb15d-3af7-4499-8677-ae4e3497a99e/resourceGroups/agri-api-dev-appservice/providers/Microsoft.Web/sites/agriOUSDevAUE2/siteextensions/Microsoft.AspNetCore.AzureAppServices.SiteExtension?api-version=2016-08-01
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/msrestazure/azure_operation.py", line 357, in init
self._operation.set_initial_status(self._response)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/msrestazure/azure_operation.py", line 230, in set_initial_status
self._raise_if_bad_http_status_and_method(response)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/msrestazure/azure_operation.py", line 148, in _raise_if_bad_http_status_and_method
"Invalid return status for {!r} operation".format(self.method))
msrestazure.azure_operation.BadStatus: Invalid return status for 'PUT' operation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "azure/cicd/devApi/appService/provision-web-app-service.py", line 78, in
deploy_app_service()
File "azure/cicd/devApi/appService/provision-web-app-service.py", line 69, in deploy_app_service
"Microsoft.AspNetCore.AzureAppServices.SiteExtension"
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/azure/mgmt/web/operations/web_apps_operations.py", line 9276, in install_site_extension
get_long_running_status, long_running_operation_timeout)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/msrestazure/azure_operation.py", line 360, in init
raise CloudError(self._response)
msrestazure.azure_exceptions.CloudError: 400 Client Error: Bad Request for url: https://management.azure.com/subscriptions/subscriptionID/resourceGroups/agri-api-dev-appservice/providers/Microsoft.Web/sites/agriOUSDevAUE2/siteextensions/Microsoft.AspNetCore.AzureAppServices.SiteExtension?api-version=2016-08-01

@lmazuel
Copy link
Member

lmazuel commented Jul 18, 2018

Indeed, I don't see any troubling thing here :(. And the error from the server is 400 and has no body to explain the trouble :(

400 usually means the resource does not exist. @yugangw-msft I see that web_apps.create_or_update is not defined as LRO, is this possible that the resource is not yet fully provisioned and we should do a custom LRO?

@APAC-DevOps could you add after your web_apps.create_or_update this:

while True:
    try:
        web_client.web_apps.get(
            my_rg_name,
            "agriOUSDevAUE2"
        )
        break
    except Exception as err:
       print("Assuming WebApp is not ready")
       time.sleep(10)

And see:

  • If this fails for a few amount of time (you see the message a few times), and then this passes, and then your site extension works
  • If it never fails (you never see the message)

Thanks,

@yugangw-msft
Copy link

Maybe @panchagnula can help

@panchagnula
Copy link

Did not read entire thread completely but i see from the ARM template you have "phpVersion": "off",
we had a regression on our API end a while back that has since been fixed where off as a value for PhpVersion was being rejected & was throwing an error. "phpVersion": "", would work. Are you still seeing this issue will changing this line in the ARM template help, if not?

@APAC-DevOps
Copy link
Author

@panchagnula
My ARM templates work fine (and I have set up my CI/CD pipeline with this set of code already, which works fine). It is my python version of Azure WebApp provision, which gets this issue.

With WebApp siteextensions, if I write define my ARM resource as embedded resource as below, the provision of the extension "Microsoft.ApplicationInsights.AzureWebSites" does fail occasionally.

"resources": [
{
"comment": "embedded resource definition"
"apiVersion": "2016-08-01",
"name": "[parameters('siteName')]",
"kind": "app",
"type": "Microsoft.Web/sites",
"location": "[parameters('location')]",
"properties": {
"reserved": false,
"serverFarmId": "[parameters('appServicePlanId')]",
"siteConfig": {
"appSettings": [
{
"name": "ASPNETCORE_ENVIRONMENT",
"value": "[parameters('coreEnvironment')]"
}
]
}
},
"resources": [
{
"apiVersion": "2016-08-01",
"name": "web",
"type": "config",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites', parameters('siteName'))]"
],
"properties": {
"phpVersion": "off",
"netFrameworkVersion": "v4.0",
"webSocketsEnabled": false,
"always_on": true
}
},

      {
        "apiVersion": "2016-08-01",
        "name": "Microsoft.AspNetCore.AzureAppServices.SiteExtension",
        "type": "siteextensions",
        "dependsOn": [
          "[resourceId('Microsoft.Web/Sites', parameters('siteName'))]"
        ],
        "properties": {}
      },

      {
        "apiVersion": "2016-08-01",
        "name": "Microsoft.ApplicationInsights.AzureWebSites",
        "type": "siteextensions",
        "dependsOn": [
          "[resourceId('Microsoft.Web/Sites', parameters('siteName'))]"
        ],
        "properties": {}
      }
    ]
  }
]

however, since I define resources by using "dependson" as below, my WebApp provision works fine. To me, it is something bizarre, and should not have happened, but it did.
{
"apiVersion": "2016-08-01",
"name": "[parameters('siteName')]",
"kind": "app",
"type": "Microsoft.Web/sites",
"location": "[parameters('location')]",
"properties": {
"reserved": false,
"serverFarmId": "[parameters('appServicePlanId')]",
"siteConfig": {
"appSettings": [
{
"name": "ASPNETCORE_ENVIRONMENT",
"value": "[parameters('coreEnvironment')]"
}
]
}
}

  },

  {
    "comments": "no comments",
    "type": "Microsoft.Web/sites/hostNameBindings",
    "name": "[concat(parameters('siteName'), '/', parameters('customDomain'))]",
    "apiVersion": "2016-08-01",
    "location": "[parameters('location')]",
    "properties": {
        "siteName": "[parameters('siteName')]",
        "hostNameType": "Verified",
        "sslState": "SniEnabled",
        "thumbprint": "[parameters('thumbPrint')]"
    },
    "dependsOn": [
        "[resourceId('Microsoft.Web/sites', parameters('siteName'))]"
    ]
    },

  {
        "name": "[concat(parameters('siteName'), '/', 'Microsoft.AspNetCore.AzureAppServices.SiteExtension')]",
        "type": "Microsoft.Web/sites/siteextensions",
        "apiVersion": "2016-08-01",
        "dependsOn": [
            "[resourceId('Microsoft.Web/sites', parameters('siteName'))]"
        ]
    },

    {
        "name": "[concat(parameters('siteName'), '/', 'Microsoft.ApplicationInsights.AzureWebSites')]",
        "type": "Microsoft.Web/sites/siteextensions",
        "apiVersion": "2016-08-01",
        "dependsOn": [
            "[resourceId('Microsoft.Web/sites', parameters('siteName'))]"
        ]
    }
]

@lmazuel
I thought of that when the WebApp extension resource provision executes, the "WebApp" resource might not be ready, so I tried to re-run my Python code a few minutes after the WebApp has provisioned successfully, that failed. I manually added WebApp, and use python code to provision extension only, that failed as well. Lastly, I copied the HTTP URL and pasted it into my browser, this threw out authentication error as well, on the occasion that I had logged into my Azure with an account with Owner permission.

As I have got my pipeline working fine with ARM templates, and the issue with python code seems too difficult to be debugged, so I am OK to let it pass for the time being.

@bsiegel bsiegel added the Service Attention Workflow: This issue is responsible by Azure service team. label Sep 26, 2018
@APAC-DevOps
Copy link
Author

Up to this point, I would say this seems likely to be a bug in Azure Rest Api.

Apart from Python SDK, I also tried the rest api call at: https://docs.microsoft.com/en-us/rest/api/appservice/webapps/webapps_installsiteextension
neither appears to be working.

Eventually, I found a sample on Internet which used kudu api in powershell code excerpt for enabling extension on Azure WebApp. I did reading, and probing, and here comes my python code for enabling extensions of Azure WebApp.

add asp net core extension to OUS WebApp

web_app_profile = web_client.web_apps.list_publishing_profile_xml_with_secrets(
    resource_group,
    resource_group + '-ous',
    format = 'WebDeploy'
)

for web_app_profile_element in web_app_profile:
    my_web_app_profile = web_app_profile_element

web_app_profile_xml = ET.fromstring(my_web_app_profile)
web_app_site_name = web_app_profile_xml[0].attrib['msdeploySite']
web_app_url = web_app_profile_xml[0].attrib['publishUrl']
web_app_user_name = web_app_profile_xml[0].attrib['userName']
web_app_user_pwd = web_app_profile_xml[0].attrib['userPWD']

asp_net_core_id = "https://" + resource_group + '-ous' + ".scm.azurewebsites.net/api/siteextensions/" + "Microsoft.AspNetCore.AzureAppServices.SiteExtension"
requests.put(url = asp_net_core_id, auth = (web_app_user_name, web_app_user_pwd))

I found it is very difficult to find python code sample for AWS SDK on Internet. Can you tell me if there is any good website which can give me some guide when I went into deadend (apart from github).

@azure-sdk azure-sdk added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Sep 24, 2020
@lmazuel lmazuel added the Mgmt This issue is related to a management-plane library. label Dec 16, 2021
@ghost ghost added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Dec 16, 2021
@xiangyan99
Copy link
Member

Is this issue still valid in azure-mgmt-web 6.1.0?

@xiangyan99 xiangyan99 added the needs-author-feedback Workflow: More information is needed from author to address the issue. label May 24, 2022
@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label May 31, 2022
@ghost
Copy link

ghost commented May 31, 2022

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
App Services customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-author-feedback Workflow: More information is needed from author to address the issue. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team no-recent-activity There has been no recent activity on this issue. RestAPI Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

8 participants