Skip to content

08. Metadata specific settings & options

Jump to bottom
Jörn Berkefeld edited this page Jul 8, 2024 · 15 revisions

Automation (automation)

E-Mail Notifications

Relevant code to add / update notifications in your automation's JSON:

    "notifications": [
        {
            "type": "Complete",
            "email": "automation-completed@demo.accenture.com",
            "message": "optional COMPLETION note that will be included in the email notification"
        },
        {
            "type": "Error",
            "email": ["automation-errors@demo.accenture.com","qa-team@demo.accenture.com"]
            "message": "optional ERROR note that will be included in the email notification"
        }
    ],

Only setting a notification for one type / removing notifications:

    "notifications": [
        {
            "type": "Error",
            "email": ["automation-errors@demo.accenture.com"]
            "message": "optional ERROR note that will be included in the email notification"
        }
    ],

Only setting a notification for one type / removing notifications without a custom message:

    "notifications": [
        {
            "type": "Error",
            "email": ["automation-errors@demo.accenture.com"]
            "message": ""
        }
    ],

To delete notifications, simply remove them from notifications or remove the notifications attribute all together. The above example would effectively remove existing notifications for type Complete and add/update the notification for Error.

Full example for adding notifications for Complete and Error:

{
    "name": "Automation_with_notification",
    "description": "",
    "key": "Automation_with_notification",
    "type": "scheduled",
    "status": "PausedSchedule",
    "schedule": {
        "typeId": 3,
        "startDate": "2023-05-31T00:00:00",
        "endDate": "2023-05-31T00:00:00",
        "icalRecur": "FREQ=DAILY;COUNT=1;INTERVAL=1",
        "timezoneName": "W. Europe Standard Time"
    },
    "steps": [
        {
            "name": "",
            "activities": [
                {
                    "name": "myWonderfulscript",
                    "r__type": "script"
                }
            ]
        }
    ],
    "notifications": [
        {
            "type": "Complete",
            "email": "automation-completed@demo.accenture.com",
            "message": "optional COMPLETION note that will be included in the email notification"
        },
        {
            "type": "Error",
            "email": ["automation-errors@demo.accenture.com","qa-team@demo.accenture.com"]
            "message": "optional ERROR note that will be included in the email notification"
        }
    ],
    "r__folder_Path": "my automations"
}

Adding Verification Activities

When Verifications are created, they automatically get a new ID assigned. That same field also acts as key as there is no other field. This comes with the unique challenge of not being able to use the templating system nor pre-defined keys during deployments to ensure the automation can a) work and b) be deployed. To circumvent the issue, make sure to deploy verifications and automations at the same time. That way, the key as seen in the deploy folder is auto-replaced with the newly generated one in all automations that you are trying to deploy.

How does the mapping work? The system tracks the key in the deploy folder vs the new key on a per-BU basis. That way, the mapping even works in massive multi-BU deployments.

Content Builder (asset)

How mcdev auto-appends the MID to keys

Content Builder is special among all systems in SFMC because the metadata it maintains shares the namespace for its keys throughout the entire instance. Normally, keys are only required to be unique across a single BU (except shared data extensions). That presents a unique challenge when you want to clone assets from one BU to another. Mcdev's internal logic, therefore, automatically checks the field memberId, which holds the MID of the BU on which the asset was created - practically the owning BU. If memberId is not set or otherwise not equal to the target BU, mcdev takes appropriate actions during execution of deploy.

Deploying to the owning BU

If you try to deploy to the owning BU, it will not touch the key, assuming you wish to override your build.

Deploying to other BUs with automatic key change

If you try to deploy to any other BU (including on other instances), it will append the MID to key, separated by a hyphen (e.g. "-123456"). The maximum length of asset keys is 36 characters. If your key is too long to fit the appended MID, mcdev will automatically shorten it accordingly. This CAN lead to unwanted abbreviations and hence is not ideal.

Deploying to other BUs with custom suffixes / templating

If you choose to use your own key across BUs, make sure to use buildTemplate and buildDefinition(-Bulk) to disable the automatic key change:

  1. ensure your markets auto-replace the MID in memberID
  2. append a suffix to every asset you have in your lower environments; make sure you leave enough character space within the max length of 36 chars to fit suffixes across all environments!
  3. ensure your markets auto-replace the suffix in the keys a) if you have one production environment, either keep the suffix empty there or append something like "_PROD" b) if you have multiple production environments, come up with easily identifiable suffixes for them (e.g. 2-letter-iso codes for countries, or something similar for your brands)

Example:

"markets": {
        "dev": {
            "mid": "12345",
            "suffix": "_DEV",
        },
        "qa": {
            "mid": "23456",
            "suffix": "_QAS",
        },
        "prod": {
            "mid": "34567",
            "suffix": "",
        }
 },

Note: The auto-replacement of the key could potentially lead to unwanted overrides and hence is not to be assumed a perfect solution. Meanwhile, setting up your own suffix scheme can be painful at first and you need to manually ensure that you will have enough characters left to append suffixes in all environments. Rule of thumb: if you use 2-letter iso codes for countries, then your more generic suffixes (e.g. _DEV) better have at least 3 letters to avoid clashing with country codes. QA and QAS are good examples: QA is often used for Quality Assurance but also happens to be the ISO code for Qatar.

deploy with --keySuffix and --noMidSuffix

Do check out these 2 options for deploy that offer a very pragmatic alternative to the above! More details in the documentation of deploy.

Data Extension (dataExtension)

Retention Policy fields in Data Extensions

The way the retention policy is saved is misleading, and hence we wanted to provide guidance if you ever need to do a deep dive here.

Field Description Values
c__retentionType main switch allRecords,
individialRecords,
allRecordsAndDataextension,
none
c__dataRetentionPeriodUnitOfMeasure represents drop-down for "period after" selection Years
Months
Weeks
Days
DataRetentionPeriodLength represents number field for "period after" selection min: 1
max: 999
RowBasedRetention only true if "delete individual records" is selected, otherwise false true, false
ResetRetentionPeriodOnImport true if "Reset period on import" is checked. This option is always false if "delete individual records" is selected true, false
DeleteAtEndOfRetentionPeriod true if "delete all records" is selected, otherwise omitted true, false
c__retainUntil Normally, this should only be filled if a date, rather than a period, was set.

This is empty for "delete individual records", but filled with a (calculated) date for the other 2 delete options even if "period after" was used.
Warning: trying to update a DE is denied when "period after" fields & this is provided		 not set
or date in ISO format (YYYY-MM-DD) "2021-06-12")

To disable retention completely, ensure that you remove all fields except for c__retentionType which needs to be set to "none".

Retention disabled:

{
    "c__retentionType": "none"
}

Retention allRecordsAndDataextension:

{
    "c__retentionType": "allRecordsAndDataextension",
    "DataRetentionPeriodLength": 6,
    "c__dataRetentionPeriodUnitOfMeasure": 'Days',
    "ResetRetentionPeriodOnImport": false,
    "c__retainUntil": "2024-05-09"
}

Retention individialRecords:

{
    "c__retentionType": "individialRecords",
    "DataRetentionPeriodLength": 6,
    "c__DataRetentionPeriodUnitOfMeasure": 'Days',
    "ResetRetentionPeriodOnImport": false
}

Retention allRecords:

{
    "c__retentionType": "allRecords",
    "DataRetentionPeriodLength": 6,
    "c__DataRetentionPeriodUnitOfMeasure": 'Days',
    "ResetRetentionPeriodOnImport": false,
    "c__retainUntil": "2024-05-09"
}

Retention allRecords and resetOnImport:

{
    "c__retentionType": "allRecords",
    "DataRetentionPeriodLength": 6,
    "c__DataRetentionPeriodUnitOfMeasure": 'Days',
    "ResetRetentionPeriodOnImport": true,
    "c__retainUntil": "2024-05-09"
}

Adding/Updating Fields on existing Data Extensions

There are a few rules to keep in mind when playing with Data Extensions fields:

  • The FieldType cannot be changed on existing fields; the API returns in error is the attribute is even provided unchanged during an update
  • MaxLength can be increased or kept on the same value but never decreased during an update
  • A Non-Required/Nullable field cannot be set to be required during an UPDATE
  • When new fields are added, they can be required, but then also have to have a DefaultValue set
  • The value for IsRequired should be 'true' or 'false'
  • The value for IsPrimary should be 'true' or 'false'

Renaming fields of a Data Extensions

With a small addition to the Data Extension's JSON, it is possible to rename fields via MC DevTools. Imagine the following Data Extension:

{
    "CustomerKey": "Account",
    "Name": "Account",
    "Description": "",
    "IsSendable": "false",
    "IsTestable": "false",
    "Fields": [
        {
            "Name": "BillingCity",
            "Scale": "0",
            "DefaultValue": "",
            "MaxLength": "40",
            "IsRequired": "false",
            "IsPrimaryKey": "true",
            "FieldType": "Text"
        },
        {
            "Name": "BillingCountry",
            "Scale": "0",
            "DefaultValue": "",
            "MaxLength": "80",
            "IsRequired": "false",
            "IsPrimaryKey": "false",
            "FieldType": "Text"
        }
    ],
    "r__folder_Path": "Data Extensions"
}

Imagine you wanted to rename BillingCountry to BillingZip for some reason. Previously, you could either go into the GUI or delete & recreate the field. Now, MC DevTools allows you to specify Name_new on the field and the tool will take care of the rest during deployment:

{
    "CustomerKey": "Account",
    "Name": "Account",
    "Description": "",
    "IsSendable": "false",
    "IsTestable": "false",
    "Fields": [
        {
            "Name": "BillingCity",
            "Scale": "0",
            "DefaultValue": "",
            "MaxLength": "40",
            "IsRequired": "false",
            "IsPrimaryKey": "true",
            "FieldType": "Text"
        },
        {
            "Name": "BillingCountry" /* old name, keep here for reference during the update! */,
            "Name_new": "BillingZip" /* new name */,
            "Scale": "0",
            "DefaultValue": "",
            "MaxLength": "80",
            "IsRequired": "false",
            "IsPrimaryKey": "false",
            "FieldType": "Text"
        }
    ],
    "r__folder_Path": "Data Extensions"
}

All you have to do is deploy the data extension again with Name_new specified for each field that needs to be renamed.

Automatically filtered DataExtensions

There is a long list of automatically created dataExtensions that you tend to not need for typical operations with mcdev and hence we filter them during retrieve:
_ChatMessagingSubscription CloudPages_DataExtension PI_ABANDONED_CART_ITEMS
_EnterpriseAttribute Einstein_MC_Email_Frequency_Oversaturation PI_CONTENT
_MobileAddress Einstein_MC_Email_Frequency_Undersaturation PI_CONTENTATTRIBS
_MobileAddressApplication Einstein_MC_Predictive_Scores PI_CONTENTVIEWS
_MobileLineAddress ExpressionBuilderAttributes PI_SESSION_ENDS
_MobileLineAddressContact IGO_PRODUCTATTRIBS PI_SESSIONS
_MobileLineProfile IGO_PRODUCTS PI_TRIGGEREVENT
_MobileLineProfileAttribute IGO_PROFILES PI_TRIGGEREVENTDETAIL
_MobileLineSubscription IGO_PURCHASES PREDICTIVE_SCORES
_MobileSubscription IGO_VIEWS SocialPages_DataExtension
_PushAddress MobileLineOrphanContact
_PushTag PI_ABANDONED_CART_EVENT

Event (event)

Update event definitions

Once created, you may only update the fields r__dataExtension_key, name, or eventDefinitionKey. Other fields are locked.

Create event definitions

When creating new events you can include a dataExtension in the deployment payload or pre-deploy it separately before you deploy the event, as long as you link it via r__dataExtension_key. If you do not set r__dataExtension_key or you did set it but you provide a schema section then that is used to auto-generate a dataExtension for you.

Valid deployments for create:

  • schema and dataExtension key set
{
    "type": "APIEvent",
    "name": "testNew_event_withSchema",
    "description": "updated on deploy",
    "eventDefinitionKey": "testNew_event_withSchema",
    "schema": {
        "fields": [
            {
                "name": "ContactId",
                "dataType": "Text",
                "maxLength": 18,
                "isNullable": false,
                "isPrimaryKey": true
            },
            {
                "name": "Type",
                "dataType": "Text",
                "maxLength": "50",
                "isNullable": false,
                "isPrimaryKey": false
            }
        ],
        "sendableCustomObjectField": "ContactId",
        "sendableSubscriberField": "_SubscriberKey"
    },
    "isVisibleInPicker": true,
    "r__dataExtension_key": "testNew_event_withSchema"
}
  • only dataExtension key set
{
    "type": "APIEvent",
    "name": "testNew_event_withSchema",
    "description": "updated on deploy",
    "eventDefinitionKey": "testNew_event_withSchema",
    "isVisibleInPicker": true,
    "r__dataExtension_key": "testNew_event_withSchema"
}
  • only schema set
{
    "type": "APIEvent",
    "name": "testNew_event_withSchema",
    "description": "updated on deploy",
    "eventDefinitionKey": "testNew_event_withSchema",
    "schema": {
        "fields": [
            {
                "name": "ContactId",
                "dataType": "Text",
                "maxLength": 18,
                "isNullable": false,
                "isPrimaryKey": true
            },
            {
                "name": "Type",
                "dataType": "Text",
                "maxLength": "50",
                "isNullable": false,
                "isPrimaryKey": false
            }
        ],
        "sendableCustomObjectField": "ContactId",
        "sendableSubscriberField": "_SubscriberKey"
    },
    "isVisibleInPicker": true,
}

Invalid deployments for create:

  • no schema and no dataExtension set
{
    "type": "APIEvent",
    "name": "testNew_event_withSchema",
    "description": "updated on deploy",
    "mode": "Production",
    "eventDefinitionKey": "testNew_event_withSchema",
    "isVisibleInPicker": true,
}

Mobile Push Application (transactionalPush)

When creating / updating transactionalPush you will notice a field for the Mobile Application ID (applicationId). This cannot be found in your downloaded BUs but only in SFMC Setup > Mobile Push. Mobile Applications can be created and maintained solely via GUI and not via API.

If you plan to deploy transactionalPush messages to another BU, ensure that as a pre-deployment step, you created the respective Mobile App on the target BU and that you replaced the application ID in your deployment package with that new ID. In a CI/CD environment where the deployment would happen automatically based on your development version, you will have to create a new variable in your source & target market, which stores the application Id for the respective BUs.

Journey (journey)

Retrieve & delete by ID

Journey is the only type that supports ids for retrieve and delete. The reason is that the key is not exposed in the interface. Meanwhile, the ID is directly visible in the URL (together with the version). To use this, prefix your ID with id: like in the following example, in which 'afjdskfsafkldsafklf' is the ID you found in the URL:

mcdev retrieve cred/bu journey id:afjdskfsafkldsafklf

Deploying from one BU to another BU

When deploying a Journey to a different BU, you will notice that not all IDs are automatically replaced, and the server might return error messages. This is because the Journey is not a standalone object but rather a collection of other objects. The current state of the mcdev only checks for triggeredSends and eventDefinitions and replaces those IDs. If you encounter other objects that are not replaced, please try to replace the IDs/Keys on your own.

Creating a new Draft version

When creating a new Journey, you will notice that its version is set to 1 and the status is set to Draft. Deploying changes to this Journey will update this particular version until you publish it via the GUI.

If you want to create a new draft version of a journey after publishing it, simply deploy it again. The tool will automatically create a new version based on the JSON you are deploying and set the status of that new version to Draft again. Note that the values in the version and status fields are automatically overridden during this deployment.

Note: Please make sure you retrieve the Journey again before creating a new draft version to ensure you are working off the latest version. Otherwise, mcdev might create a new version based on an outdated version of the Journey.

Roles (role)

Analyzing differences between roles

This one is a hack, but it works nicely nonetheless. We support document for roles; that method takes all roles downloaded in the retrieve/ folder and creates the md file for it. If you want to create such a report for chosen roles you need to make sure you downloaded a recent copy of them, then delete all other roles in your retrieve folder (this will not affect the server!) and afterwards run mcdev document cred/_ParentBU_ role. It will create a report just for the remaining roles! You can then go in and either check the rendered md file directly or copy it into an excel file for further analysis.

Users (user)

You can create/update most fields in the user JSONs, except for User Permissions. Roles and Business Units can be assigned and removed at will simply by adding or removing the role name or the MID, respectively.

Roles associated with Business Units

While the user interface of SFMC allows setting up roles that will be automatically assigned in the background to any user with access to that BU, we cannot access that information via API at this point (or at least we do not know how).

Roles associated with users only for a specific Business Unit

SFMC DevTools cannot show which roles were assigned to users just for given Business Units, nor can it update such an assignment.

Standard "Marketing Cloud *" roles

Due to a bug in SFMC's APIs, whenever we try to update a user that has one of these standard Marketing Cloud roles assigned, these roles will be unassigned. This means if you had such a role assigned to a user, and then you try to update that user, even if it does not concern the roles, these default roles will be removed afterwards. If, on the other hand, you try to add such a role, this attempt will be ignored.

Warning: To avoid issues, whenever you try to update a user with the expectation of that user (still) having such a role afterward, the update is blocked with an error. You can only update that user if you remove the role in question from your to-be-deployed JSON.

Unlock user account

Depending on your security settings, you might choose to automatically block access to unused user accounts or after several failed attempts to log in. In these cases you will notice that c__IsLocked_readOnly is set to true. To unlock them, you cannot just change this value (hence the custom "readOnly" suffix) but instead you need to use the field Unlock and set that to true. To make things easy, mcdev adds this particular field automatically whenever IsLocked is true. All you need to do is change it from false to true and then re-deploy that user to unlock him/her.

SSO & FederationID

While this information currently cannot be retrieved via API (that still worked back in 2019...), it can still be set. Therefore, if you use not only an external SSO provider but also a provisioning system, you can enable/disable the SSO checkbox and set the federation ID (your external user ID) in Marketing Cloud.

You have to add this to the end of your user-JSON file and run deploy:

    "SsoIdentity": {
        "IsActive": true, // true enables SSO, false disables SSO
        "FederatedID": "test-mcdev" // make sure to replace test-mcdev with the actual value from your SSO provider for this user
    }

Full example:

{
    "CreatedDate": "2021-06-22T14:49:02.99",
    "ModifiedDate": "2021-06-22T14:54:07.45",
    "CustomerKey": "0cea7619-8ea9-41e2-97a3-d9a33dc0847a",
    "UserID": "test-mcdev@accenture.com",
    "Name": "NameOfTheUser",
    "Email": "test-mcdev@accenture.com",
    "MustChangePassword": true,
    "ActiveFlag": true,
    "UserPermissions": [],
    "LastSuccessfulLogin": "2021-06-22T14:49:02.99",
    "IsAPIUser": true,
    "NotificationEmailAddress": "test-mcdev@accenture.com",
    "DefaultBusinessUnit": 7281698,
    "c__type": "User",
    "c__AccountUserID": 717142520,
    "c__IsLocked_readOnly": false,
    "c__AssociatedBusinessUnits": [
        7281698
    ],
    "c__RoleNamesGlobal": [
        "My super role",
    ],
    "c__TimeZoneName": "(GMT+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna *",
    "c__LocaleCode": "en-GB",
    "SsoIdentity": {
        "IsActive": true,
        "FederatedID": "test-mcdev"
    }
}

Verification Activity (verification)

See details above: Automation > Adding Verification Activities

Copyright (c) 2020-2024 Accenture. MIT licensed. Main contributors: Jörn Berkefeld, Doug Midgley

Clone this wiki locally