Assistance with environment variables for credentials

[gcormier]

Similar to the other SHIPYARD_ environment variables, it would be great if we could have the credentials as environment variables.

SHIPYARD_BATCH_ACCOUNT_KEY
SHIPYARD_BATCH_ACCOUNT_URL
SHIPYARD_BATCH_STORAGE_ACCOUNT
SHIPYARD_BATCH_STORAGE_KEY
SHIPYARD_BATCH_STORAGE_ENDPOINT

[alfpark]

Can you expand on your specific scenario on why you require these credentials?

[gcormier]

These would be as an alternative to the credentials that sit inside config.yaml.

[alfpark]

You can use the SHIPYARD_AAD_* for this purpose. Or alternatively, store your credential.yaml file (or individual fields) in keyvault: https://github.com/Azure/batch-shipyard/blob/master/docs/74-batch-shipyard-azure-keyvault.md.

[gcormier]

Wow, I totally glazed over the SHIPYARD_AAD_* !

[gcormier]

Sorry, still having problems finding which config value corresponds to which environment variable.

credentials:
batch:
account_key: ???
account_service_url: ???
storage:
mystorageaccount:
account: ???
account_key: ???
endpoint: SHIPYARD_AAD_ENDPOINT

[alfpark]

With AAD, you don't need to populate any secrets, please see: https://github.com/Azure/batch-shipyard/blob/master/docs/11-batch-shipyard-configuration-credentials.md, specifically the aad section. Please read the entire doc however, as there are certain properties that are required when using AAD (e.g., resource_group under storage and management).

Additionally, this section is helpful (even without using keyvault): https://github.com/Azure/batch-shipyard/blob/master/docs/74-batch-shipyard-azure-keyvault.md#authenticating-with-aad-and-azure-keyvault

[gcormier]

Okay, I've gone through it and done a bit of jumping around, and I'm getting an error.

Error : ValueError: You need to provide an account name and either an account_key or sas_token when creating a storage service.

Docs state:

(required unless aad or account_key_keyvault_secret_id is specified) account_key is the storage account key

My credentials.yaml:

credentials:
  batch:
    resource_group: batch-shipyard
    account_service_url: https://abcbatch.eastus.batch.azure.com
  storage:
    mystorageaccount:
      resource_group: batch-shipyard
      account: xyz

Environment (obfuscated)

export SHIPYARD_AAD_DIRECTORY_ID=123..
export SHIPYARD_AAD_AUTH_KEY=456..
export SHIPYARD_AAD_APPLICATION_ID=789..

Since I have the env defined, that should be the global AAD section, in which case I should not need to specify account_key in the yaml file.

[alfpark]

Looks like this is a case of bad errors and missing documentation, apologies.

You need to also populate:

credentials:
  # other stuff
  management:
    subscription_id: 123...

[gcormier]

That worked, I'm up and running! Thanks very much for the assistance!

I doubt a subscription ID is anything sensitive, but it probably couldn't hurt to move that to an environment variable as well?

[alfpark]

Apparently this is already supported, but I forgot to doc it. Use the environment variable SHIPYARD_SUBSCRIPTION_ID.

Edit: It's actually documented here: https://github.com/Azure/batch-shipyard/blob/master/docs/20-batch-shipyard-usage.md#shared-options