[MetricsAdvisor] Added tests.yml file for automated test runs

[kinelski]

Part of #15924.

[kinelski]

It seems we're getting PermissionDenied from the service, but tests work locally with the same settings:
https://dev.azure.com/azure-sdk/internal/_build/results?buildId=682791&view=logs&j=9e4810bd-6c8e-5aff-29e0-d5fb4fb1935f&t=aebe1dc3-650d-52a7-62d2-615892e27a94&l=14919

Is there anything wrong with the way we're declaring the env vars, danieljurek ?

azure-sdk-for-net/sdk/metricsadvisor/tests.yml

Lines 7 to 20 in afe625a

	EnvVars:
	METRICSADVISOR_ACCOUNT_NAME: js-metrics-advisor
	METRICSADVISOR_SUBSCRIPTION_KEY: $(metricsadvisor-test-subscription-key)
	METRICSADVISOR_PRIMARY_API_KEY: $(net-metricsadvisor-test-api-key)
	METRICSADVISOR_DATA_FEED_ID: 9860df01-e740-40ec-94a2-6351813552ba
	METRICSADVISOR_METRIC_ID: 27e3015f-04fd-44ba-a20b-bc529a0aebae
	METRICSADVISOR_DETECTION_CONFIGURATION_ID: fb5a6ed6-2b9e-4b72-8b0c-0046ead1c15c
	METRICSADVISOR_ALERT_CONFIGURATION_ID: 204a211a-c5f4-45f3-a30e-512fb25d1d2c
	METRICSADVISOR_HOOK_ID: e3499688-99c9-4a6c-bbcb-6866fcd593a5
	METRICSADVISOR_ALERT_ID: 17548749000
	METRICSADVISOR_INCIDENT_ID: 2c8a72ba0e8eb41643cc0e5cff368412-17548749000
	METRICSADVISOR_FEEDBACK_ID: 50a63795-d631-43ae-9a05-d744df4b986d
	METRICSADVISOR_SQL_SERVER_CONNECTION_STRING: $(metricsadvisor-test-sql-server-connection-string)
	METRICSADVISOR_SQL_SERVER_QUERY: select * from adsample2 where Timestamp = @StartTime and City = 'Mexico City' and Category = 'Shoes Handbags & Sunglasses'

Edit: problem solved. We just needed to add quotes to the string env vars.

[danieljurek]

@kinelski - I'm behind on the context here. Can we not use dynamic resources for these live tests? Also, .NET tests should not use a metrics advisor account name that starts with js. Keeping these test resources organized and consistent with the rest of our live tests will make debugging problems much easier.

For example in the future (assuming we need static resources), if someone on the engineering system or JS team sees an account called js-metrics-advisor and knows that the JS SDK has migrated away from it and deletes that account, we wouldn't expect the .NET tests to break.

cc @benbp

[kinelski]

@danieljurek We fixed the issue. We were only missing quotes in the env vars declarations (fixed here #17910).

Can we not use dynamic resources for these live tests?

Unfortunately this is not feasible in Metrics Advisor. The service takes too long to create the resource (30 minutes or so). Also, once the resource is created, it still needs an indefinite amount of time to process the data we need to request in our tests.

Also, .NET tests should not use a metrics advisor account name that starts with js.

The service limits the amount of resources we can create per subscription (only 2, if I'm not mistaken), so all languages are using the same resource for testing. It has the js prefix because it was only being used by JS originally (I can agree that this was not a wise decision). We can create and set up a new resource with a better name, but it will take some effort from other languages as well.

[danieljurek]

Thanks for that info! This all makes sense. :)

Please track the work to fix this in a bug so it doesn't get lost.

[kinelski]

Tracking it here: #17918