[Storage] Support Failover #4107

blueww · 2018-10-09T07:57:14Z

This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.

The feature target to release on 10/15. Since the time limitation, send PR directly to public repo. Please help to review it ASAP.

Rest Spec:
Failover: https://microsoft.sharepoint.com/:w:/r/teams/AzureStorage/_layouts/15/Doc.aspx?sourcedoc=%7B77CFF0F4-AFAD-4AC1-B51C-1F17C92E06AF%7D&file=SRP%20Customer-controlled%20failover%20API.docx&action=default&mobileredirect=true

Get Last Sync Time：https://microsoft.sharepoint.com/:w:/r/teams/AzureStorage/_layouts/15/doc2.aspx?sourcedoc=%7bad30891d-7c70-4d4f-9b95-046d40fa900d%7d&action=edit&wdPid=660b6155

PR information

The title of the PR is clear and informative.
There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
If applicable, the PR references the bug/issue that it fixes.
Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

I have read the contribution guidelines.
My spec meets the review criteria:
- The spec conforms to the Swagger 2.0 specification.
- The spec follows the guidelines described in the Swagger checklist document.
- Validation tools were run on swagger spec(s) and have all been fixed in this PR.

AutorestCI · 2018-10-09T07:57:51Z

Automation for azure-sdk-for-js

Encountered an unknown error: (azure-sdk-for-js)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 4a52e83a78130ed79b957d734999b005534cd6e5
  stderr: 'fatal: reference is not a tree: 4a52e83a78130ed79b957d734999b005534cd6e5'

AutorestCI · 2018-10-09T07:58:35Z

Automation for azure-sdk-for-python

Encountered an unknown error: (azure-sdk-for-python)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 4a52e83a78130ed79b957d734999b005534cd6e5
  stderr: 'fatal: reference is not a tree: 4a52e83a78130ed79b957d734999b005534cd6e5'

AutorestCI · 2018-10-09T07:59:28Z

Automation for azure-sdk-for-node

Encountered an unknown error: (azure-sdk-for-node)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 4a52e83a78130ed79b957d734999b005534cd6e5
  stderr: 'fatal: reference is not a tree: 4a52e83a78130ed79b957d734999b005534cd6e5'

AutorestCI · 2018-10-09T07:59:34Z

Automation for azure-sdk-for-ruby

Encountered an unknown error: (azure-sdk-for-ruby)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 4a52e83a78130ed79b957d734999b005534cd6e5
  stderr: 'fatal: reference is not a tree: 4a52e83a78130ed79b957d734999b005534cd6e5'

azuresdkci · 2018-10-09T07:59:58Z

Can one of the admins verify this patch?

AutorestCI · 2018-10-09T08:00:59Z

Automation for azure-sdk-for-java

A PR has been created for you based on this PR content.

Once this PR will be merged, content will be added to your service PR:
Azure/azure-sdk-for-java#2479

annatisch · 2018-10-09T14:34:47Z

Adding @ravbhatnagar for ARM feedback

ravbhatnagar · 2018-10-10T21:53:47Z

specification/storage/resource-manager/Microsoft.Storage/stable/2018-07-01/storage.json

+        "x-ms-long-running-operation": true
+      }
+    },
+    "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{accountName}/lastSyncTime": {      


Why isnt last sync time exposes as a property on the storage account itself?

From the feature team:

The design of returning storage account’s properties is synchronized and RSRP returns local cache only and it does not refresh from the stamp, given the fact the all the existing properties are somewhat still and it only gets updated when the user performs some action. LST on the other hand gets computed every 2 minutes and the account row on backend get updated accordingly. It does not make sense for us to sync between RSRP and stamp every two minutes for all geo-replicated accounts, which is a very expensive operation. We rather make it on demand.

In that case, it could be returned using a $expand query parameter. Only when the user explicitly specifies this, then you return this as a property on the storage account. By default, just doing get does not return it.
It really seems like a property of the account and in the current way of modeling it makes it looks like a nested resource under storageAccount which it is not.

ravbhatnagar · 2018-10-10T21:55:38Z

specification/storage/resource-manager/Microsoft.Storage/stable/2018-07-01/storage.json

+          "x-ms-client-name": "IsHnsEnabled",
+          "description": "Account HierarchicalNamespace enabled if sets to true."
+        },
+        "failoverInProgress": {


can you make this a generic property like "status" with states like failing over etc. It will allow you reuse the same property for other status related information in future.

This status is a very special status just for Get LST from secondary location status. I don't see any possibility that it can be reused by any other APIs in the recent future. So I would prefer to keep it as what it is now.

Ok lets atleast make this a string enum. bools are less descriptive and dont allow for future expansion of states.

@ravbhatnagar
Do you mean make the "failoverInProgress" a enum. Actually for this property's meaning, it almost won't have future expansion (won't add more value). It means if the failover is in progress. The value can only be "True" (means failover in progress) or null (means failover not in progress). I think a bool? can make user more easier to us the value in SDK, they only need to check the value is True, and don't need to compare the string is "True".

Ok, thats fine. i just wanted to note that we are strongly recommending teams against using bools. Even if its just two states, string enums work much better. Its fine if you dont want to change it.

Thanks！ I will take care of every bool value usage in swagger.

blueww · 2018-10-11T02:13:49Z

@ravbhatnagar
I have replied all your comments. Please help to review.
Since this feature target to release on 10/15, please help to review it ASAP.

annatisch

Hi @blueww - just one change from my side.
Just waiting on ARM sign off.

annatisch · 2018-10-15T15:56:29Z

specification/storage/resource-manager/Microsoft.Storage/stable/2018-07-01/storage.json

+            "description": "Accepted -- Failover request accepted; operation will complete asynchronously."
+          }
+        },
+        "x-ms-long-running-operation": true


Please add the "x-ms-long-running-operation-options" section as documented here:
https://github.com/Azure/autorest/tree/master/docs/extensions#x-ms-long-running-operation-options
https://github.com/Azure/azure-rest-api-specs/blob/master/documentation/openapi-authoring-automated-guidelines.md#R2010

This is a new requirement for long running POST operations.

@annatisch
I have added following to the 2 long running api.

"x-ms-long-running-operation-options": { "final-state-via": "location" }

Since following in responds of the request, I think we should use location here. Let me know if you see any issue.

"Location": [
"https://management.azure.com/subscriptions/45b60d85-fd72-427a-a708-f994d26e593e/providers/Microsoft.Storage/locations/eastus2euap/asyncoperations/7c7b797d-7e03-4afb-a4bf-c98ca54d1c32?monitor=true&api-version=2018-07-01"
],

Thanks @blueww - so long as the responses do indeed follow the "location" header for both PUT and POST then this should be okay.

ravbhatnagar · 2018-10-15T18:17:01Z

@blueww - I still havent heard back from Daniel on the other open feedback item.

AutorestCI · 2018-10-17T02:31:37Z

Automation for azure-sdk-for-go

Encountered an unknown error: (azure-sdk-for-go)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 4a52e83a78130ed79b957d734999b005534cd6e5
  stderr: 'fatal: reference is not a tree: 4a52e83a78130ed79b957d734999b005534cd6e5'

blueww · 2018-10-17T02:42:36Z

@ravbhatnagar
Daniel will be OOF till 10/29. I have ping Chris for this.
But it seems the release on 10/15 is postponded. So we should not be on hurry for this.

annatisch · 2018-10-22T14:59:00Z

Hi @blueww & @ravbhatnagar - what is the status here?

blueww · 2018-10-24T03:52:38Z

@annakhach ， @ravbhatnagar
It seems the feature won't release in the recent future.
We might can close this, and resend when the new feature release data is decided.

[Storage] Support Failover

4bbbc1b

AutorestCI mentioned this pull request Oct 9, 2018

[AutoPR storage/resource-manager] [Storage] Support Failover Azure/azure-sdk-for-python#3510

Closed

AutorestCI mentioned this pull request Oct 9, 2018

[AutoPR storage/resource-manager] [Storage] Support Failover Azure/azure-sdk-for-node#3784

Closed

AutorestCI mentioned this pull request Oct 9, 2018

[AutoPR storage/resource-manager] [Storage] Support Failover Azure/azure-sdk-for-java#2478

Closed

blueww mentioned this pull request Oct 9, 2018

[Storage] Support Storage Account failover Azure/azure-sdk-for-net#4850

Closed

9 tasks

azuresdkci assigned annatisch Oct 9, 2018

azuresdkci requested a review from annatisch October 9, 2018 08:09

annatisch added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Oct 9, 2018

ravbhatnagar reviewed Oct 10, 2018

View reviewed changes

ravbhatnagar added the ARMReviewInProgress label Oct 10, 2018

annatisch requested changes Oct 15, 2018

View reviewed changes

This was referenced Oct 17, 2018

[AutoPR storage/resource-manager] [Storage] Support Failover Azure/azure-sdk-for-js#210

Closed

[AutoPR storage/resource-manager] [Storage] Support Failover Azure/azure-sdk-for-go#3047

Closed

annatisch approved these changes Oct 17, 2018

View reviewed changes

[Storage] update Long Running operation

8ef8f60

blueww force-pushed the failover2 branch from 88b84e1 to 8ef8f60 Compare October 17, 2018 03:15

KrisBash added the ARMChangesRequested <valid label in PR review process>add this label when require changes after ARM review label Oct 22, 2018

blueww closed this Oct 24, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Storage] Support Failover #4107

[Storage] Support Failover #4107

blueww commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

azuresdkci commented Oct 9, 2018

AutorestCI commented Oct 9, 2018 •

edited

Loading

annatisch commented Oct 9, 2018

ravbhatnagar Oct 10, 2018

blueww Oct 11, 2018 •

edited

Loading

ravbhatnagar Oct 12, 2018

ravbhatnagar Oct 10, 2018

blueww Oct 11, 2018 •

edited

Loading

ravbhatnagar Oct 12, 2018

blueww Oct 12, 2018

ravbhatnagar Oct 15, 2018

blueww Oct 16, 2018

blueww commented Oct 11, 2018

annatisch left a comment

annatisch Oct 15, 2018

blueww Oct 17, 2018

annatisch Oct 17, 2018

ravbhatnagar commented Oct 15, 2018

AutorestCI commented Oct 17, 2018 •

edited

Loading

blueww commented Oct 17, 2018

annatisch commented Oct 22, 2018

blueww commented Oct 24, 2018

[Storage] Support Failover #4107

[Storage] Support Failover #4107

Conversation

blueww commented Oct 9, 2018 • edited Loading

PR information

Quality of Swagger

AutorestCI commented Oct 9, 2018 • edited Loading

Automation for azure-sdk-for-js

AutorestCI commented Oct 9, 2018 • edited Loading

Automation for azure-sdk-for-python

AutorestCI commented Oct 9, 2018 • edited Loading

Automation for azure-sdk-for-node

AutorestCI commented Oct 9, 2018 • edited Loading

Automation for azure-sdk-for-ruby

azuresdkci commented Oct 9, 2018

AutorestCI commented Oct 9, 2018 • edited Loading

Automation for azure-sdk-for-java

annatisch commented Oct 9, 2018

Choose a reason for hiding this comment

blueww Oct 11, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blueww Oct 11, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blueww commented Oct 11, 2018

annatisch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravbhatnagar commented Oct 15, 2018

AutorestCI commented Oct 17, 2018 • edited Loading

Automation for azure-sdk-for-go

blueww commented Oct 17, 2018

annatisch commented Oct 22, 2018

blueww commented Oct 24, 2018

blueww commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

AutorestCI commented Oct 9, 2018 •

edited

Loading

blueww Oct 11, 2018 •

edited

Loading

blueww Oct 11, 2018 •

edited

Loading

AutorestCI commented Oct 17, 2018 •

edited

Loading