Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Malmiller/1661 data factory module mvp #4

Merged
merged 33 commits into from
Jan 14, 2021
Merged
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
bf1f5f1
intial commit of data_factory module
malcmiller Dec 10, 2020
8ef0bb4
changed folder structer and updated datasets files
malcmiller Dec 14, 2020
09f180b
Merge branch 'master' into malmiller/1661-add-data-factory-module
malcmiller Dec 16, 2020
476036d
add linked_service_azure_blob_storage
kottofy Dec 16, 2020
7f86e00
add linked_service_azure_file_storage
kottofy Dec 16, 2020
b8f9ec3
add linked_service_azure_function
kottofy Dec 16, 2020
bc2dec2
add linked_service_azure_sql_database
kottofy Dec 16, 2020
a23052b
add linked_service_cosmosdb
kottofy Dec 16, 2020
82476b1
add linked_service_data_lake_storage_gen2
kottofy Dec 16, 2020
ae1947f
add linked_service_key_vault
kottofy Dec 17, 2020
b375dfe
add linked_service_mysql
kottofy Dec 17, 2020
8f1bacc
add linked_service_postgresql
kottofy Dec 17, 2020
b1d6f25
add linked_service_sql_server
kottofy Dec 17, 2020
6458dda
add linked_service_sftp
kottofy Dec 17, 2020
f4e0afb
add linked_service_web
kottofy Dec 17, 2020
f36e8ff
updates
kottofy Dec 17, 2020
6df03dd
added data_factory, data_factory_pipeline and data_factory_trigger_sc…
malcmiller Dec 17, 2020
74ef208
Merge branch 'malmiller/1661-add-data-factory-module' of https://gith…
malcmiller Dec 17, 2020
796af1d
corrections
kottofy Dec 17, 2020
9cfb4cd
Merge branch 'malmiller/1661-add-data-factory-module' of https://gith…
kottofy Dec 17, 2020
ac5a839
updated data_factory dataset files
malcmiller Dec 17, 2020
fc0f416
Merge branch 'malmiller/1661-add-data-factory-module' of https://gith…
malcmiller Dec 17, 2020
1298e41
added data_factory_datasets module
malcmiller Dec 18, 2020
71fb5ca
ran terraform fmt
kottofy Dec 21, 2020
55d8262
fixed delimited_text variables
malcmiller Dec 21, 2020
0597237
WIP: Rename data_factory & add blob linked service
kottofy Dec 22, 2020
5d2705a
Added connection string requirement to blob linked service
malcmiller Dec 22, 2020
4a9d3ea
renamed some linked services folders and added modules
malcmiller Dec 28, 2020
e98c1c3
Updated linked services module
malcmiller Jan 4, 2021
2be2e25
Merge branch 'master' into malmiller/1661-add-data-factory-module
malcmiller Jan 5, 2021
33d85e0
Merge branch 'master' into malmiller/1661-data-factory-module-mvp
malcmiller Jan 7, 2021
dec3b76
removed untested linked services
malcmiller Jan 7, 2021
52fb172
updated dynamic block
malcmiller Jan 12, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add linked_service_cosmosdb
kottofy committed Dec 16, 2020
commit a23052b0c1123f541012b11700f5fd98a99faafc
34 changes: 12 additions & 22 deletions modules/data_factory/linked_services/cosmosdb/module.tf
Original file line number Diff line number Diff line change
@@ -1,23 +1,13 @@
resource "azurerm_data_factory_dataset_azure_blob" "dataset" {
name = var.name
resource_group_name = var.resource_group_name
data_factory_name = var.data_factory_name
linked_service_name = var.linked_service_name
folder = var.folder
description = var.description
annotations = var.annotations
parameters = var.parameters
additional_properties = var.additional_properties
path = var.path
filename = var.filename

dynamic "schema_column" {
for_each = lookup(var.settings, "schema_column", {}) == {} ? [] : [1]

content {
name = var.settings.schema_column.name
type = var.settings.schema_column.type
description = var.settings.schema_column.description
}
}
resource "azurerm_data_factory_linked_service_cosmosdb" "linked_service_cosmosdb" {
name = var.name
resource_group_name = var.resource_group_name
data_factory_name = var.data_factory_name
description = try(var.description, null)
integration_runtime_name = try(var.integration_runtime_name, null)
annotations = try(var.annotations, null)
parameters = try(var.parameters, null)
account_endpoint = try(var.account_endpoint, null)
account_key = try(var.account_key, null)
database = try(var.database, null)
connection_string = try(var.connection_string, null)
}
4 changes: 2 additions & 2 deletions modules/data_factory/linked_services/cosmosdb/output.tf
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
output id {
value = azurerm_data_factory_dataset_azure_blob.dataset.id
sensitive = true
value = azurerm_data_factory_linked_service_cosmosdb.linked_service_cosmosdb.id
sensitive = true
}
42 changes: 21 additions & 21 deletions modules/data_factory/linked_services/cosmosdb/variables.tf
Original file line number Diff line number Diff line change
@@ -1,47 +1,47 @@
variable name {
description = "(Required) Specifies the name of the Data Factory Dataset"
description = "(Required) Specifies the name of the Data Factory Linked Service. Changing this forces a new resource to be created. Must be globally unique"
}

variable resource_group_name {
description = "(Required) The name of the resource group in which to create the Data Factory Dataset"
description = "(Required) The name of the resource group in which to create the Data Factory Linked Service. Changing this forces a new resource"
}

variable data_factory_name {
description = "(Required) The Data Factory name in which to associate the Dataset with"
description = "(Required) The Data Factory name in which to associate the Linked Service with. Changing this forces a new resource"
}

variable linked_service_name {
description = "(Required) The Data Factory Linked Service name in which to associate the Dataset with"
variable description {
description = "(Optional) The description for the Data Factory Linked Service"
}

variable folder {
description = "(Optional) The folder that this Dataset is in. If not specified, the Dataset will appear at the root level"
variable integration_runtime_name {
description = "(Optional) The integration runtime reference to associate with the Data Factory Linked Service"
}

variable schema_column {
description = "(Optional) A schema_column block"
variable annotations {
description = "(Optional) List of tags that can be used for describing the Data Factory Linked Service"
}

variable description {
description = "(Optional) The description for the Data Factory Dataset"
variable parameters {
description = "(Optional) A map of parameters to associate with the Data Factory Linked Service"
}

variable annotations {
description = "(Optional) List of tags that can be used for describing the Data Factory Dataset"
variable additional_properties {
description = "(Optional) A map of additional properties to associate with the Data Factory Linked Service"
}

variable parameters {
description = "(Optional) A map of parameters to associate with the Data Factory Dataset"
variable account_endpoint {
description = "(Optional) The endpoint of the Azure CosmosDB account. Required if connection_string is unspecified"
}

variable additional_properties {
description = "(Optional) A map of additional properties to associate with the Data Factory Dataset"
variable account_key {
description = " (Optional) The account key of the Azure Cosmos DB account. Required if connection_string is unspecified"
}

variable path {
description = "(Required) The path of the Azure Blob"
variable database {
description = "(Optional) The name of the database. Required if connection_string is unspecified"
}

variable filename {
description = "(Required) The filename of the Azure Blob"
variable connection_string {
description = "(Optional) The connection string. Required if account_endpoint, account_key, and database are unspecified"
}