Example names very inconsistent

[zikalino]

Are there any guidelines for example naming?

• sometimes it's description
• sometimes long description
• sometimes just sample filename
• sometimes spaces
• sometimes underscores

Below 3 different examples:

"x-ms-examples": {
          "Delete network interface": { "$ref": "./examples/NetworkInterfaceDelete.json" }
        }

        "x-ms-examples": {
          "Updates the Data Lake Analytics account object specified by the accountName with the contents of the account object.": {
            "$ref": "./examples/Accounts_Update.json"
          }
        }

        "x-ms-examples": {
          "CanonicalListSupportPlanInfo_Post": {
            "$ref": "./examples/CanonicalListSupportPlanInfo_Post.json"
          }
        },

[jhendrixMSFT]

The best practice is to describe the scenario without being too wordy if possible. The docs for this are here and I've created a PR to suggest this.

[zikalino]

@jhendrixMSFT
Thanks for quick answer! :-)
I actually have a few additional questions.

What I am trying to do is to actually use examples to generate integration tests (and also samples) for Ansible. (they are generated automatically).

Instead of just creating my own samples, tests, I am trying to backpropagate fixes to the to the REST API specs.

I have noticed in the doc:

• It forms the basis of better documentation. The doc team will leverage this in lightening up the REST API documentation.
• It will also be used to test the API/endpoint by running a live request with the help of some custom tool.

Do you think it's right effort? I have some ideas related to that. Are you the right person to talk?

[jhendrixMSFT]

CC @sergey-shandar
Sergey and I investigated generating tests from examples in some depths last year. While it works for simple cases we found it becomes very complex for APIs that have many prerequisites (i.e. creating a VM, you first have to create a NIC which means an IP which means a VNET etc). We decided that in order to support this across all RPs it would really require a DSL to properly describe dependencies which was more cost then we were budgeted so we abandoned the project. If you have suggestions on how to solve this we'd love to hear them.

[zikalino]

@jhendrixMSFT @sergey-shandar
i am trying to figure out whether i could somehow reuse existing samples without adding anything complicated on the top.
for instance, if sample resource ids were always formatted properly, i could probably find appropriate samples to create dependencies, but i guess there may be some exceptions.

[jhendrixMSFT]

@sergey-shandar do you have the write-up we did for this? It should cover a lot of the problems we ran into.
@zikalino Properly formatting resource IDs would help but from a quick look for creating a VM and its dependencies you can see that there is no standard today and nothing is enforced in CI. See here and here for quick examples.
Next hurdle would be how do you know which API version to invoke when creating a dependency? I suppose you could assume latest, but that might not always work (e.g. sample specifies parameters specific to some API version).
And of course samples must be complete, i.e. supply all required parameters and response bodies. For things that are long-running operations we don't model the non-terminal responses.
I'm sure there's more I'm forgetting, hopefully Sergey has the doc we put together.

[zikalino]

@jhendrixMSFT
yes, I may have a bit simplified view right now. I am thinking from DevOps tools perspective, for instance Ansible right now, so:

• API version are fixed for me, as my modules are based on specific version API, so I need to handle only dependencies between modules.
• enforcing consistent resource id would much simplify this task, and could probably be sufficient to automatically generate all dependencies, I could actually do an experiment with compute and network
  So maybe it's worth looking from DevOps tools perspective, as there are several already depending on Azure REST API and testing is a major problem.

[jhendrixMSFT]

Closing as I believe there's no further action here (feel free to reach out offline if you have further questions regarding the test automation).