[BUG] Azure.ResourceManager.Network extension calls fail with System.MissingMethodException

[stishkin]

Library name and version

Azure.ResourceManager.Network 1.0.0-beta.5

Describe the bug

All of methods fail in a similar manner.

System.MissingMethodException: Method not found: 'Azure.ResourceManager.ArmClientOptions Azure.ResourceManager.Core.ArmResource.get_ClientOptions()'.
at Azure.ResourceManager.Network.VirtualNetworkCollection..ctor(ArmResource parent)
at Azure.ResourceManager.Network.ResourceGroupExtensions.GetVirtualNetworks(ResourceGroup resourceGroup)

Expected behavior

Method call to succeed

Actual behavior

System.MissingMethodException: Method not found: 'Azure.ResourceManager.ArmClientOptions Azure.ResourceManager.Core.ArmResource.get_ClientOptions()'.
at Azure.ResourceManager.Network.VirtualNetworkCollection..ctor(ArmResource parent)
at Azure.ResourceManager.Network.ResourceGroupExtensions.GetVirtualNetworks(ResourceGroup resourceGroup)

Reproduction Steps

1. Create ArmClient object: var armClient = new ArmClient()
2. Get existing resource group: var rg = armClientGetResourceGroup("ExisitngResourceGroup")
3. Call rg.GetVirtualNetworks() on resource group object

using Azure.ResourceManager;
using Azure.ResourceManager.Network;

Environment

No response

[jsquire]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[Vevish]

I have the same issue: Azure.ResourceManager.Network version 1.0.0-beta.5

        ArmClient armClient = new ArmClient(credential);

        var subs = armClient.GetSubscriptions();
   
        foreach (var s in subs)
        {
            ResourceGroupCollection rgs = s.GetResourceGroups();
            //    var rgs = v.GetResourceGroups();
            foreach (var r in rgs)
            {
             //   Console.WriteLine(r.Id);
                var localrg = armClient.GetResourceGroup(r.Id);
           
                ResourceGroup resourceGroup = armClient.GetSubscription(s.Id).GetResourceGroups().Where(e=>e.Id==r.Id).FirstOrDefault();

                Console.WriteLine(resourceGroup.Id.Name);


                
                try
                {
                  var myNets=  resourceGroup.GetVirtualNetworks(); //Error occurs here and any other network related requests
                    foreach(var mynet in myNets)
                    {
                        Console.WriteLine(myNets.Id);
                    }
                }
                catch (Exception f)
                {
                    Console.WriteLine(f);
                }

            }
        }

=======================================================
System.MissingMethodException: Method not found: 'Azure.ResourceManager.ArmClientOptions Azure.ResourceManager.Core.ArmResource.get_ClientOptions()'.
at Azure.ResourceManager.Network.VirtualNetworkCollection..ctor(ArmResource parent)
at Azure.ResourceManager.Network.ResourceGroupExtensions.GetVirtualNetworks(ResourceGroup resourceGroup)

[DizzyDeveloper]

Not the same method, but I am also encountering a missing method exception in Azure.ResourceManager.Network version 1.0.0-beta.5.

System.MissingMethodException: Method not found: '!!0 Azure.ResourceManager.Resources.Subscription.UseClientContext(System.Func`5<System.Uri,Azure.Core.TokenCredential,Azure.ResourceManager.ArmClientOptions,Azure.Core.Pipeline.HttpPipeline,!!0>)'.
   at Azure.ResourceManager.Network.SubscriptionExtensions.GetVirtualNetworksAsync(Subscription subscription, CancellationToken cancellationToken)

[archerzz]

@stishkin @Vevish @DizzyDeveloper I could not re-produce this problem. Here is my test case.

using NUnit.Framework;
using Azure.ResourceManager;
using Azure.ResourceManager.Network;
using Azure.ResourceManager.Resources;
using System;
using Azure.Identity;

namespace TestNetwork2;

public class Tests
{
    [SetUp]
    public void Setup()
    {
    }

    [Test]
    public void Test1()
    {
        ArmClient armClient = new ArmClient(new DefaultAzureCredential());

        var subs = armClient.GetSubscriptions();

        foreach (var s in subs)
        {
            Console.Error.WriteLine($"Subscription: {s.Id.Name}");
            ResourceGroupCollection rgs = s.GetResourceGroups();
            foreach (var rg in rgs)
            {
                Console.Error.WriteLine($"Resource Group: {rg.Id.Name}");

                try
                {
                    var myNets = rg.GetVirtualNetworks(); //Error occurs here and any other network related requests
                    foreach (var mynet in myNets)
                    {
                        Console.Error.WriteLine($"Virtual Network: myNets.Id");
                    }
                }
                catch (Exception f)
                {
                    Console.Error.WriteLine(f);
                }
            }
        }
    }
}

From the error stack trace, it seems that it's a compatibility problem between Azure.ResourceManager and Azure.ResourceManager.Network. Azure.ResourceManager.Network depends on Azure.ResourceManager. And we've made some breaking changes in last couple of versions, so it's quite subtle to keep their versions compatible with each other.

What are the versions of those two packages in your .csproj? I'm testing using Azure.ResourceManager version 1.0.0-beta.9 and Azure.ResourceManager.Network version 1.0.0-beta.6. If you're using Azure.ResourceManager.Network version 1.0.0-beta.5, then probably you should use Azure.ResourceManager version 1.0.0-beta.7.

I suggest to upgrade to the latest versions.

[Vevish]

I was using Visual Studio 2019, Upgraded to Visual 2022, and using the latest Azure.ResourceManager, i did try a fewer lower versions on visual studio 2019, unfortunately, i was not able to go to the latest on visual 2019. Moving to Visual 2022 with these package versions have resolved the issue, seems like a package mismatch like [archerzz] mentioned Thank you .

Exe net6.0

[DizzyDeveloper]

@archerzz My issue came down to incompatibility between versions different versions of the SDK.
Azure.ResourceManager.Network version 1.0.0-beta.5 is not compatible with Azure.Core 1.22.
In our project we had a scenario as follows (when we encounter this issue):
Azure.ResourceManager.Network 1.0.0-beta.5 which has a min Azure.Core version of '1.21andAzure.ResourceManager.Compute 1.0.0-beta.8which has a minAzure.Coreversion requirement of1.22`.

Those two packages didn't mix very well as it was using Azure.Core 1.22 which produced that exception I mentioned earlier.

Downgrading Azure.ResourceManager.Compute 1.0.0-beta.8 to Azure.ResourceManager.Compute 1.0.0-beta.7 solved it for us.

However, I have noticed you have recently release a new version for Azure.ResourceManager.Network. Havn't tried it yet, but it appears to share the same Azure.Core requirement as the latest Azure.ResourceManager.Compute so should be good.

[archerzz]

@Vevish @DizzyDeveloper I'm glad the problem has been resolved for you. We're in beta and there are breaking changes between each version, so normally you have to pick up versions of different packages which are released at the same time. The compatibility issue should happen rarely after we go for GA.

@stishkin Does it work for you?

[stishkin]

@archerzz yes, this issue is resolved after upgrade to

Azure.ResourceManager.Network version 1.0.0-beta.6

[DizzyDeveloper]

@archerzz May I ask what happened with the UseClientContext.
This was available in Azure.ResourceManager.Compute 1.0.0-beta.7 however it seems to disappeared as part of the newer versions? I was using it to execute Azure API routes that are not yet supported in the SDK. i.e. retrieving network interfaces for virtual machine scale set instances.
Is there a newer or different function I can use to roll may own functions? It is prevent me from upgrading the newer versions of the sdk.

[archerzz]

@DizzyDeveloper Do you mean ClientContext? It's removed in recent refactoring after receiving some review comments.

How do you use it? I think most information can be retrieved in ArmClient. There is a short cut API to create a cached client, see:

azure-sdk-for-net/sdk/resourcemanager/Azure.ResourceManager/src/ArmResource.cs

Line 124 in 65a47bf

public virtual T GetCachedClient<T>(Func<ArmClient, T> clientFactory)

Here is an example of how it is used: https://cs.github.com/Azure/azure-sdk-for-net/blob/d3eb6ee84e2a42e1308f55192c74dd04e57e3240/sdk/cdn/Azure.ResourceManager.Cdn/src/Generated/Extensions/ResourceGroupResourceExtensionClient.cs#L50

[DizzyDeveloper]

Heya @archerzz,
The older public class Subscription : ArmResource had a public virtual T UseClientContext<T> function that I used in a simplistic adhoc manner to my make calls to routes that the SDK doesn't support yet.

        internal static async Task<PublicIpAddressDto> GetPublicIpAddress(this Subscription subscription, ResourceIdentifier virtualMachineScaleSetInstanceId)
        {
            // GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/microsoft.Compute/virtualMachineScaleSets/{virtualMachineScaleSetName}/virtualMachines/{virtualmachineIndex}/networkInterfaces?api-version=2018-10-01
            return await subscription.UseClientContext(async (uri, credential, opt, pipe) =>
            {
                using (var message = pipe.CreateMessage())
                {
                    message.Request.Method = RequestMethod.Get;
                    var builder = new RequestUriBuilder();
                    builder.Reset(uri);
                    builder.AppendPath(virtualMachineScaleSetInstanceId.ToString());
                    builder.AppendQuery("api-version", "2018-10-01");
                    message.Request.Uri = builder;
                    message.Request.Headers.Add("Accept", "application/json");

                    await pipe.SendAsync(message, default).ConfigureAwait(false);

                    using (var reader = new StreamReader(message.Response.ContentStream!))
                    {
                        var content = await reader.ReadToEndAsync().ConfigureAwait(false);
                        var response = JsonSerializer.Deserialize<PublicIpAddressDto>(content);
                        return response!;
                    }
                }
            }).ConfigureAwait(false);
        }

Not the grandest...but got the job done.

Yea I noticed the public virtual T GetCachedClient<T> function, but when I looked at it, it appeared to have internalised aspects of itself that meant I could do some raw requests. Caveat to this statement, I haven't had a chance to look at this, again, in more detail as I had hoped. So I am unable to say for sure that I won't be able to use it has I have needed. But when I looked at it the first time, it wasn't going to be a straight port.

[m-nash]

The reason this was removed it breaks the immutability requirement of clients.

Do I understand correctly the issue here is a point in time issue that will be resolved once we have coverage on all APIs which will be in the GA version this month?

I believe if we wanted to go the route of providing raw access to all APIs we would add in low level client support which is a separate set of untyped methods but give you more low level control.

[DizzyDeveloper]

@m-nash Yip yip I did believe that most of the scenarios were I needed to use the UseClientContext were simply point in time issues. It just became an issue initial when not all the routes that I needed where supported yet, and this function was removed. If those routes for which I had to roll my own are support in the GA version then happy days. Also when I wrote that comment before I wasn't sure when these routes would become supported.
However, could I suggest still exposing a low level client? I am asking because I am assuming that the sdk supports all documented routes? Which means that there could be some un-documented routes that the SDK doesn't support. Note, when I am referring to documented routes I am simply referring to what I have found on the msdn rest api documentation. If there is a hidden more comprehensive documentation that exists then happy days, could someone point me at it.

In the undocumented route scenario that I am referring to in my case, is a modified version of the permission rest api. On the page it documents the ability to list the "callers" permissions for a specific resource. Which I suspect will be supported by the SDK, however, as I mentioned above I am using a slighty modified version of said route. Specifically, I am needing to list all the callers permissions at the subscription level not the resource/resource group level.
In othewords, I am effectively invoking this route:
GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Authorization/permissions?api-version=2015-07-01
If the SDK supports this route out of the box, then I have no qualms about adding in a low level client, expect if there are other such routes. However, if it doesn't support I would really preferring using some built in functionality, rather than having to role my own to an even greater extent.

[DizzyDeveloper]

@m-nash @archerzz
Hi Guys, I am just checking in.
Is there any word on whether a low level client or some similar is going to be included?

[archerzz]

@DizzyDeveloper Low level client (or Data plane generated client) is for date plane, not for mgmt plane. Network only has mgmt plane API, so there won't be corresponding low level client SDK for it.

[DizzyDeveloper]

@archerzz So would there be away for me to execute the route as I defined the previous comment?

GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Authorization/permissions?api-version=2015-07-01

At the moment I am invoking this manually because the SDK doesn't have a support it out of the box. Will this get added in a later release?

[archerzz]

@DizzyDeveloper Authorization is on the GA list. Currently it's under review: #29075

[madd0]

@m-nash @archerzz in our case, we were using the technique described by @DizzyDeveloper in order to poll operation state on URIs such as:

https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/locations/{location}/operations/{operationId}

We get this URL from the Azure-AsyncOperation header, place it storage and query it later when we need to know if the operation is done. Unless I'm mistaken, the SDK does not provide a dedicated mechanism to do the same thing. What would be the recommended approach for doing this with the latest versions of the SDK?

[archerzz]

@madd0 Which SDK are you using? I believe the SDK provides polling implementation. Do you want to manually poll the status of an operation?

• If it's Azure.ResourceManager.XXX (e.g. Track2), we're discussing a pattern. See details in docs(core): add an example of custom polling #29180
• If it's Microsoft.Azure.Management.XXX (e.g. Track1), there is no easy way provided by SDK.
  • Either you manually poll the status according to the LRO spec (e.g. Azure-AsyncOperation header)
  • Or if there is a GET operation which can return the status. For example, assume you update a database and you want to check the status, if the backend provides a GET method which return the databases with status properties. Then you can call the corresponding GET API in SDK to do the check. That will be easier than polling Azure-AsyncOperatoin.

[madd0]

We are using Track2, which does provide polling, but in our scenario, the asynchronous operation is performed by one process, which stores the polling URI in storage, and the polling is performed by another process, which only has the URI to perform the call.

We used to use Azure.ResourceManager.ArmClient.UseClientContext to make the call:

ValueTask SendPollingRequestTask(Uri uri, TokenCredential creds, ArmClientOptions clientOptions, HttpPipeline httpPipeline)
{
    httpMessage = httpPipeline.CreateMessage();
    messageInitializer(httpMessage);
    return httpPipeline.SendAsync(httpMessage, ct);
}

var armClient = new ArmClient(credentials, subscriptionId, options);

await armClient.UseClientContext(SendPollingRequestTask);

With messageInitializer being:

private static HttpMessage InitializeHttpMessageFromOperation(SingleOperation operation, HttpMessage httpMessage)
{
    Request request = httpMessage.Request;
    request.Method = RequestMethod.Get;
    RequestUriBuilder requestUriBuilder = new RequestUriBuilder();
    requestUriBuilder.Reset(operation.PollingUri);
    request.Uri = requestUriBuilder;
    request.Headers.Add("Accept", "application/json");
    return httpMessage;
}

And operation.PollingUri similar to https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/locations/{location}/operations/{operationId}

As an alternative, we are now using Azure.Core.Pipeline.HttpPipelineBuilder like this:

var pipeline = HttpPipelineBuilder.Build(options, new BearerTokenAuthenticationPolicy(credential, scope));

using (var httpMessage = pipeline.CreateMessage())
{
    messageInitializer(httpMessage);
    await pipeline.SendAsync(httpMessage, ct);
}

The builder takes the same TokenCredential and ArmClientOptions instances that the ArmClient takes.

(Disclaimer: the code approximates our actual code to give an idea of how we're using the SDK, but is not guaranteed to compile.)

[archerzz]

Closing since the original problem had been resolved. For the issue about polling state, let's continue the discussion in Azure/autorest.csharp#2158 Thanks.