Welcome! This repository contains two projects.
For an introduction refer to How to copy data from Azure Blob Storage to Amazon S3 using code.
To deploy using AWS CloudFormation, follow instructions in the blog post.
To deploy using Terraform follow the instructions below.
-
Download / Clone repository
-
Locally, maintain the folder structure as is, and execute Terraform builds from the TF directory. Both the TF and CFN folders are necessary.
-
You can use the example in the AzureblobtoAmazonS3copy folder to configure an Azure Storage account. Just move the files to the AzureblobtoAmazonS3copy/TF directory. If you decide to use this method, please edit the specified lines in azs3copy-aws.tf as below:
## NOTE: Comment out these lines if deploying Azure Example
#resource "aws_secretsmanager_secret_version" "SecretsManagerSecret" {
# secret_id = aws_secretsmanager_secret.SecretsManagerSecret.id
# secret_string = jsonencode({
# bloburl = var.AzureBlobURL
# tenantid = var.AzureTenantID
# appid = var.AzureApplicationID
# appsecret = var.AzureSecretKey
# bucket_name = "${aws_s3_bucket.S3Bucket.bucket}"
# isactive = "True"
# begindate = var.BlobToS3SyncStartDate
# sns_arn_l1 = "${aws_sns_topic.SNSTopicL1L2.arn}"
# sns_arn_l2 = "${aws_sns_topic.SNSTopicL2L3.arn}"
# sns_arn_l3 = "${aws_sns_topic.SNSTopicLargeFileInit.arn}"
# sns_arn_l4 = "${aws_sns_topic.SNSTopicLargeFilePart.arn}"
# sns_arn_l5 = "${aws_sns_topic.SNSTopicLargeFileRecomb.arn}"
# })
#}
# NOTE: Uncomment these lines if deploying Azure Example
resource "aws_secretsmanager_secret_version" "SecretsManagerSecret" {
secret_id = aws_secretsmanager_secret.SecretsManagerSecret.id
secret_string = jsonencode({
bloburl = azurerm_storage_account.StorageAccount.primary_blob_endpoint
tenantid = data.azurerm_client_config.current.tenant_id
appid = azuread_application.AppRegistration.application_id
appsecret = azuread_application_password.AppPassword.value
bucket_name = "${aws_s3_bucket.S3Bucket.bucket}"
isactive = "True"
begindate = var.BlobToS3SyncStartDate
sns_arn_l1 = "${aws_sns_topic.SNSTopicL1L2.arn}"
sns_arn_l2 = "${aws_sns_topic.SNSTopicL2L3.arn}"
sns_arn_l3 = "${aws_sns_topic.SNSTopicLargeFileInit.arn}"
sns_arn_l4 = "${aws_sns_topic.SNSTopicLargeFilePart.arn}"
sns_arn_l5 = "${aws_sns_topic.SNSTopicLargeFileRecomb.arn}"
})
}
-
Configure
terraform.tfvars
. Refer to descriptions invariables-aws.tf
for help. -
Modify main-aws.tf to include an appropriate profile or make changes to match your preferred deployment configuration.
-
Run
terraform apply
from the AzureblobtoAmazonS3copy/TF directory.
For an introduction refer to How to view Azure costs using Amazon QuickSight
To deploy using Cloudformation or Terraform, follow instructions in the Cloud Intelligence Dashboard for Azure workshop.
To understand how the Azure Cost Usage files should be provided, take a look at the examples here.
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.