[DataFactory] Enable Avro Dataset in Public Swagger #6560

yuzhangyi · 2019-07-05T11:24:26Z

Latest improvements:

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Contribution checklist:

I have reviewed the documentation for the workflow.
Validation tools were run on swagger spec(s) and have all been fixed in this PR.
The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

openapi-sdkautomation · 2019-07-05T11:24:28Z

SDK Automation [Logs] (Generated from 045dd60)

Python: Azure/azure-sdk-for-python [Logs]

No packages generated.

Java: Azure/azure-sdk-for-java [Logs]

Package generation in progress.

Go: Azure/azure-sdk-for-go [Logs]

Package generation in progress.

JavaScript: Azure/azure-sdk-for-js [Logs]

Package generation in progress.

AutorestCI · 2019-07-05T11:26:41Z

Automation for azure-sdk-for-python

Encountered an unknown error: (azure-sdk-for-python)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 4e45c64e279b2cd86a988c459c0296c2c5e0b020
  stderr: 'fatal: reference is not a tree: 4e45c64e279b2cd86a988c459c0296c2c5e0b020'

azuresdkci · 2019-07-05T11:30:25Z

Can one of the admins verify this patch?

AutorestCI · 2019-07-05T11:30:59Z

Automation for azure-sdk-for-go

Encountered an unknown error: (azure-sdk-for-go)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 4e45c64e279b2cd86a988c459c0296c2c5e0b020
  stderr: 'fatal: reference is not a tree: 4e45c64e279b2cd86a988c459c0296c2c5e0b020'

AutorestCI · 2019-07-05T11:31:23Z

Automation for azure-sdk-for-java

A PR has been created for you based on this PR content.

Once this PR will be merged, content will be added to your service PR:
Azure/azure-sdk-for-java#3084

michaeljqzq · 2019-07-08T06:03:49Z

Please provide example including the newly added property

michaeljqzq

Please add example

yuzhangyi · 2019-07-19T02:02:41Z

Please add example
Hi, after having sync with our engineer team, we don't need external example to validate it, because we already has example at "datafactory\resource-manager\Microsoft.DataFactory\stable\2018-06-01\examples"

yuzhangyi · 2019-07-19T02:03:25Z

Please add example

Hi, after having sync with our engineer team, we don't need external example to validate it, because we already has example at "datafactory\resource-manager\Microsoft.DataFactory\stable\2018-06-01\examples"

openapi-sdkautomation · 2019-07-19T02:03:28Z

SDK Automation [Logs] (Generated from 045dd60, Iteration 3)

Python: Azure/azure-sdk-for-python [Logs] [Diff]

sdk/datafactory/azure-mgmt-datafactory [Logs]

Java: Azure/azure-sdk-for-java [Logs]

Package generation in progress.

Go: Azure/azure-sdk-for-go [Logs]

Package generation in progress.

JavaScript: Azure/azure-sdk-for-js [Logs]

Package generation in progress.

yuzhangyi · 2019-07-19T02:03:45Z

Please provide example including the newly added property

Hi, after having sync with our engineer team, we don't need external example to validate it, because we already has example at "datafactory\resource-manager\Microsoft.DataFactory\stable\2018-06-01\examples"

hvermis · 2019-07-19T21:03:42Z

...tafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Pipeline.json

@@ -657,6 +657,40 @@
        "fileExtension"
      ]
    },
+    "AvroWriteSettings": {
+      "description": "Delimited text write settings.",


Fix the description, should say Avro

Thanks, will fix it later.

hvermis

Make sure to fix the description

hvermis · 2019-07-19T21:05:10Z

@KrisBash Could you review this one too? This is another dataset type with settings, again similar to what you already reviewed.

yuzhangyi · 2019-07-22T11:44:56Z

@KrisBash Could you review this one too? This is another dataset type with settings, again similar to what you already reviewed.

Hi, @KrisBash, can you help finish reviewing this change and merge it into master? thanks!

michaeljqzq · 2019-07-30T05:26:54Z

@yuzhangyi , is this PR still valid? If yes, please fix the comments. Otherwise I will close it

yuzhangyi · 2019-07-31T01:10:38Z

@yuzhangyi , is this PR still valid? If yes, please fix the comments. Otherwise I will close it

Hi, @michaeljqzq, this public swagger change is valid, can you help signoff and merge it into master? For the comment raised by hvermis, I will fix it in next change.

michaeljqzq · 2019-07-31T04:42:35Z

@yuzhangyi , we can not merge a PR in a mid-state. Please either update this one or create another PR

yuzhangyi · 2019-07-31T05:53:14Z

Close it and use new one.

zhanyu2024 added 2 commits July 5, 2019 15:55

Enable avro in public swagger

d45a842

update

045dd60

azuresdkci assigned michaeljqzq Jul 5, 2019

azuresdkci requested a review from michaeljqzq July 5, 2019 11:25

AutorestCI mentioned this pull request Jul 5, 2019

[AutoPR datafactory/resource-manager] [DataFactory] Enable Avro Dataset in Public Swagger Azure/azure-sdk-for-python#6242

Closed

AutorestCI mentioned this pull request Jul 5, 2019

[AutoPR datafactory/resource-manager] [DataFactory] Enable Avro Dataset in Public Swagger Azure/azure-sdk-for-go#5203

Closed

AutorestCI mentioned this pull request Jul 5, 2019

[AutoPR datafactory/resource-manager] [DataFactory] Enable Avro Dataset in Public Swagger Azure/azure-sdk-for-java#4290

Closed

michaeljqzq suggested changes Jul 8, 2019

View reviewed changes

yuzhangyi mentioned this pull request Jul 19, 2019

[DataFactory] Enable Avro Dataset in SDK Azure/azure-sdk-for-net#6828

Closed

yuzhangyi closed this Jul 19, 2019

yuzhangyi reopened this Jul 19, 2019

AutorestCI mentioned this pull request Jul 19, 2019

[AutoPR datafactory/resource-manager] [DataFactory] Enable Avro Dataset in Public Swagger Azure/azure-sdk-for-go#5303

Closed

michaeljqzq added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Jul 19, 2019

hvermis reviewed Jul 19, 2019

View reviewed changes

hvermis approved these changes Jul 19, 2019

View reviewed changes

yuzhangyi closed this Jul 31, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DataFactory] Enable Avro Dataset in Public Swagger #6560

[DataFactory] Enable Avro Dataset in Public Swagger #6560

yuzhangyi commented Jul 5, 2019

openapi-sdkautomation bot commented Jul 5, 2019 •

edited

Loading

AutorestCI commented Jul 5, 2019 •

edited

Loading

azuresdkci commented Jul 5, 2019

AutorestCI commented Jul 5, 2019 •

edited

Loading

AutorestCI commented Jul 5, 2019 •

edited

Loading

michaeljqzq commented Jul 8, 2019

michaeljqzq left a comment

yuzhangyi commented Jul 19, 2019

yuzhangyi commented Jul 19, 2019

openapi-sdkautomation bot commented Jul 19, 2019 •

edited

Loading

yuzhangyi commented Jul 19, 2019

hvermis Jul 19, 2019 •

edited

Loading

yuzhangyi Jul 22, 2019

hvermis left a comment

hvermis commented Jul 19, 2019

yuzhangyi commented Jul 22, 2019

michaeljqzq commented Jul 30, 2019

yuzhangyi commented Jul 31, 2019

michaeljqzq commented Jul 31, 2019

yuzhangyi commented Jul 31, 2019

[DataFactory] Enable Avro Dataset in Public Swagger #6560

[DataFactory] Enable Avro Dataset in Public Swagger #6560

Conversation

yuzhangyi commented Jul 5, 2019

Latest improvements:

Contribution checklist:

ARM API Review Checklist

openapi-sdkautomation bot commented Jul 5, 2019 • edited Loading

SDK Automation [Logs] (Generated from 045dd60)

AutorestCI commented Jul 5, 2019 • edited Loading

Automation for azure-sdk-for-python

azuresdkci commented Jul 5, 2019

AutorestCI commented Jul 5, 2019 • edited Loading

Automation for azure-sdk-for-go

AutorestCI commented Jul 5, 2019 • edited Loading

Automation for azure-sdk-for-java

michaeljqzq commented Jul 8, 2019

michaeljqzq left a comment

Choose a reason for hiding this comment

yuzhangyi commented Jul 19, 2019

yuzhangyi commented Jul 19, 2019

openapi-sdkautomation bot commented Jul 19, 2019 • edited Loading

SDK Automation [Logs] (Generated from 045dd60, Iteration 3)

yuzhangyi commented Jul 19, 2019

hvermis Jul 19, 2019 • edited Loading

Choose a reason for hiding this comment

yuzhangyi Jul 22, 2019

Choose a reason for hiding this comment

hvermis left a comment

Choose a reason for hiding this comment

hvermis commented Jul 19, 2019

yuzhangyi commented Jul 22, 2019

michaeljqzq commented Jul 30, 2019

yuzhangyi commented Jul 31, 2019

michaeljqzq commented Jul 31, 2019

yuzhangyi commented Jul 31, 2019

openapi-sdkautomation bot commented Jul 5, 2019 •

edited

Loading

AutorestCI commented Jul 5, 2019 •

edited

Loading

AutorestCI commented Jul 5, 2019 •

edited

Loading

AutorestCI commented Jul 5, 2019 •

edited

Loading

openapi-sdkautomation bot commented Jul 19, 2019 •

edited

Loading

hvermis Jul 19, 2019 •

edited

Loading