Add config option to select a different azure cloud env in the azure-eventhub input and azure module #17659
Conversation
|
Pinging @elastic/integrations (Team:Integrations) |
|
Whats the reasoning for the prefix of |
|
I have the same question as Blake. Also do we need to add this for Metricbeat? |
| ==== `override_resource_manager_endpoint` | ||
|
|
||
| Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different azure environment. |
There was a problem hiding this comment.
Agree with the comments by Blake and Kaiyan, don't consider this an override, just a normal setting. Add also something about possible values. And possibly something about hybrid clouds support.
| ==== `override_resource_manager_endpoint` | |
| Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different azure environment. | |
| ==== `resource_manager_endpoint` | |
| Optional, resource manager endpoint to use, defaults to the public environment, possible values are... . If a URL is provided, the environment is obtained from the URL, this is useful in Hybrid Cloud deployments. |
Maybe we could add an explicit option for environment URLs, instead of accepting names and urls in the same option. And maybe use something like environment instead of resource_manager_endpoint, in case we use the same settings for features that need access to other endpoints.
| ==== `override_resource_manager_endpoint` | |
| Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different azure environment. | |
| ==== `environment.name` | |
| Optional, environment to use, defaults to the public environment, possible values are... . | |
| ==== `environment.url` | |
| Optional, loads the environment from a URL, this is useful in the Hybrid Cloud model. If this setting is used, `environment.name` is ignored. |
There was a problem hiding this comment.
@jsoriano , the reason I went for the override- is because I do not want users to necessary specify this setting, this should be only used for other private env. To look for the azure resource manager endpoint might take some effort for the users and I want to avoid that in most of the cases where the are using the public cloud.
One workaround I would go for is to rename this option to resource_manager_endpoint but not add it in the config files so users will not be tempted to enter it if unnecessary.
Not sure what you mean by " instead of accepting names and urls in the same option" , we only accept azure resource manager endpoints, the azure-eventhub sdk can retrieve the specific environment based on that.
Also adding environment.name and environment.url might cause more confusion as users will have a hard time figuring those out, also there is no environment.url , we can talk about management_portal_URL or resource_manager_endpoint or service_endpoint but with the resource_manager_endpoint the SDK can help us identify the right env.
There was a problem hiding this comment.
@jsoriano , the reason I went for the override- is because I do not want users to necessary specify this setting, this should be only used for other private env.
Other optional settings behave the same way, and don't have an override prefix, would it be different to other optional settings?
To look for the azure resource manager endpoint might take some effort for the users and I want to avoid that in most of the cases where the are using the public cloud. One workaround I would go for is to rename this option to
resource_manager_endpointbut not add it in the config files so users will not be tempted to enter it if unnecessary.
Yes, I find perfectly fine to have this option in the reference docs, but not in the config files.
Not sure what you mean by " instead of accepting names and urls in the same option" , we only accept azure resource manager endpoints, the azure-eventhub sdk can retrieve the specific environment based on that.
Oh, ok, my mistake then, I saw that we accept URLs, and I thought we also accepted some kind of identifier there. One more reason to add some example values in the docs 🙂
I have still a question about that, the URLs accepted by EnvironmentFromURL, are of the same kind as the ones of the resource manager endpoints?
Also adding
environment.nameandenvironment.urlmight cause more confusion as users will have a hard time figuring those out, also there is noenvironment.url, we can talk aboutmanagement_portal_URLorresource_manager_endpointorservice_endpointbut with theresource_manager_endpointthe SDK can help us identify the right env.
I was suggesting that to make configuration and its documentation easier. A user wanting to use the US Gov cloud would use environment.name: usgov, or something similar, accepted values could be in the documentation, so they don't need to look for specific endpoints in other places. A user that is using a hybrid cloud would use environment.url: http://.... with an URL that can be accepted by EnvironmentFromURL.
Something like this would be specially helpful if we need to use at some point other endpoints, e.g: the Service Manager Endpoint, so we don't have to add a setting like service_manager_endpoint.
But I don't know much about these endpoints and URLs, so as you prefer 🙂
There was a problem hiding this comment.
I answer myself after reading EnvironmentFromURL source. It seems that the URL accepted by this method is actually the resource manager endpoints and from that it obtains the other endpoints, so this solves my concerns about having different kind of values for hybrid clouds.
So I think that the only pending things to consider would be:
- Rename of the setting to
resource_manager_endpoint. - Document possible values for this setting, and how it works for hybrid clouds.
|
Someone from the community has tried this change and it worked for them 🙂 https://discuss.elastic.co/t/azure-module-error/227210/13?u=jsoriano |
| } | ||
| if env, ok := environments[overrideResManager]; ok { | ||
| return env, nil | ||
| } |
There was a problem hiding this comment.
Nit. Should we try without the / path? So it works both for https://management.azure.com/ and https://management.azure.com.
jsoriano
added
needs_backport
…eventhub input and azure module (elastic#17659) * add option to select a different azure env * changelog * rename * add test * mage fmt update (cherry picked from commit 4e1d4b9)
…ure cloud env in the azure-eventhub input and azure module (#17859) * Add config option to select a different azure cloud env in the azure-eventhub input and azure module (#17659) * add option to select a different azure env * changelog * rename * add test * mage fmt update (cherry picked from commit 4e1d4b9) * update changelog
What does this PR do?
Adds
override_resource_manager_endpointconfig option to select a different azure cloud env in the azure-eventhub input and azure module.Users should enter the resource manager endpoint in this case.
Ex:
https://management.chinacloudapi.cn/: azure.ChinaCloud,https://management.microsoftazure.de/: azure.GermanCloud,https://management.azure.com/: azure.PublicCloud,https://management.usgovcloudapi.net/: azure.USGovernmentCloud,Why is it important?
To support other azure cloud env besides the public one
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Related issues