Skip to content

07. Advanced Configuration

Jump to bottom
Jörn Berkefeld edited this page Nov 18, 2024 · 11 revisions

The tool's configuration can be changed in the file .mcdevrc.json located in the root of your project folder.

It contains Market Configuration (markets: { ... }), Market List Configuration (marketList: { ... }) the list of usable Business Units per credentials, directories, as well as other options.

You will also find the configuration for what metadata shall be retrieved here in metaDataTypes.retrieve: [ ... ].

You will also find a secondary file named .mcdev-auth.json containing your credentials. Do not commit this to your repository! You should only commit .mcdevrc.json as this file contains project-wide settings that do not compromise security.

Config Options

.mcdev-auth.json

This file contains the credentials provided by the installed package you create for mcdev. It is generated upon running mcdev init and its name is automatically placed in the .gitignore to ensure it is not accidentally stored in your git repository.

{
    "MyProject": {
        "client_id": "0m0sdfkmq23zv0pebqlv91d1",
        "client_secret": "lSL5XVOEFU1A5QAorQOHLEMK",
        "auth_url": "https://mct0l7hsfq2rt1kxfy8sc47mq.auth.marketingcloudapis.com/",
        "account_id": "7281234"
    }
}

Please note that account_id is your Enterprise ID, the MID of your Parent BU.

.mcdevrc.json

The central config in .mcdevrc.json holds multiple adjustable settings:

{
  "options": {
    "deployment": {
      "commitHistory": 10,
      "sourceTargetMapping": {
        "deployment-source": "deployment-target"
      },
      "targetBranchBuMapping": {
        "release/*": "MySandbox/QA-DE",
        "master": ["MyProduction/PROD-DE", "MyProduction/PROD-NL"]
      }
    },
    "validation": {
            "retrieve": {
                "noAmpscriptHtmlTag": "warn",
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
                "overrides": [
                    {
                        "type": [
                            "journey"
                        ],
                        "options": {
                            "noGuidKeys": "off",
                        }
                    }
                ]
            },
            "buildDefinition": {
                "noAmpscriptHtmlTag": "warn",
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
                "overrides": [
                    {
                        "type": [
                            "journey"
                        ],
                        "options": {
                            "noGuidKeys": "off",
                        }
                    }
                ]
            },
            "deploy": {
                "noAmpscriptHtmlTag": "warn",
                "noGuidKeys": "error",
                "noRootFolder": "error",
                "overrides": [
                    {
                        "type": [
                            "journey"
                        ],
                        "options": {
                            "noGuidKeys": "off",
                        }
                    }
                ]
            }
    },
    "documentType": "md",
    "documentStandardRoles": true,
    "exclude": {
      "role": {
        "CustomerKey": ["excludedRoleKey","excludedOtherRoleKey"]
      }
    },
    "include": {
      "asset": {
        "r__folder_Path": ["Content Builder/only/assets/in/here"]
      },
    },
    "serverTimeOffset": -6
  },
  "directories": {
        "businessUnits": "businessUnits/",
        "deploy": "deploy/",
        "docs": "docs/",
        "retrieve": "retrieve/",
        "template": "template/",
        "templateBuilds": ["retrieve/", "deploy/"]
  },
  "metaDataTypes": {
    "documentOnRetrieve": ["user", "automation", "dataExtension", "role"],
    "retrieve": [...]
  }
}
Setting Default Description
options.deployment.commitHistory 10 Configures how many commits createDeltaPkg will display if no parameters are given
options.deployment.sourceTargetMapping {"deployment-source": "deployment-target"} Configuration of 1 or many source-target marketList combos for mcdev createDeltaPkg
options.deployment.targetBranchBuMapping {"release/*": "...","master": ["..."]} Can be used by CI/CD pipelines to know what BUs shall be deployed to upon a merge into one of the specified branches
options.validation current default and custom rules Allows setting validation rules to "off", "warn" or "error
options.documentType 'md' Defines the format for documentation ('md', 'html', 'both')
options.documentStandardRoles false Optionally skip standard role documentation by setting to false
options.exclude.type.field [] Allows you to filter out metadata on retrieve based on their field values, e.g. CustomerKey (previously options.filter)
options.include.type.field [] Allows you to filter out metadata on retrieve based on their field values, e.g. CustomerKey
options.serverTimeOffset -6 Used to work around quirks in how SFMC handles timezones; For stack4: set to -7 (US Mountain time); others: -6 (US Central)
options.formatOnSave true Allows disabling auto-formatting for all retrieve/deploy operations
directories.businessUnits 'businessUnits/' Directory to save BU base details in
directories.deploy 'deploy/' Where deploy searches for files to deploy
directories.docs 'docs/' Directory for document output
directories.retrieve 'retrieve/' Where retrieve stores downloaded files
directories.template 'template/' Where bt stores downloaded templates & bd retrieves them from
directories.templateBuilds ['retrieve/','deploy/'] Where bd saves final deployment versions in. This can hold multiple directories, e.g. ['retrieve/','deploy/']
metaDataTypes.documentOnRetrieve ['user', 'automation', 'dataExtension', 'role'] automatically executes document for selected types
metaDataTypes.retrieve changes with each release check Metadata Type Support for current list

Validation rules

You can define validation rules for retrieve, buildDefinition and deploy. The following rules exist out of the box:

  • noGuidKeys: test for metadata that has a GUID / UUID as key.
  • noRootFolder: test if metadata that does exist in a folder resides in the root for that particular type or in a subfolder.

Possible rule settings are:

  • not set, which implicitly turns off the rule
  • "off" explicitly turns off the rule
  • "warn" shows a log message of type warning
  • "error" shows a log message of type error AND blocks further execution
    • for retrieve that prevents download
    • for buildDefintion (and build) that prevents the creation of the corresponding file in the deploy folder
    • for deploy this will prevent deployment.

Please keep in mind that you can always skip validation rules all together at run-time by adding --skipValidation to your command (deploy / build / buildDefinition / buildDefinitionBulk)

Custom Validation rules - .mcdev-validation.js

Optionally one can create a file named .mcdev-validations.js in the root of your project to specify your own validation rules. That gives you full flexibility to test for whatever guidelines you might have as you can literally parse the JSON of the metadata yourself.

One example that depends on BU names and hence is not part of the standard set could look like this:

'use strict';

const buSuffixMap = {
    _ParentBU_: '',
    DEV: '_DEV',
    QA: '_QA',
    PROD: '',
};

/**
 *
 * @param {any} definition type defintiion
 * @param {any} item metadata json
 * @param {string} targetDir where the metadata is stored ("deploy/cred/bu")
 * @param {any} Util helper methods
 * @returns {Promise.<any>} validation rule
 */
export function validation(definition, item, targetDir, Util) {
    const bu = targetDir.includes('/') ? targetDir.split('/').pop() : targetDir.split('\\').pop();
    const suffix = buSuffixMap[bu];

    if (suffix === undefined) {
        Util.logger.error(
            `BU '${bu}' not defined for keySuffix validation in .mcdev-validations.js`
        );
    }
    return {
        keySuffix: {
            failedMsg: 'Key Suffix expected but not found: ' + suffix,
            /**
             * @returns {boolean} true=test passed
             */
            passed: function () {
                // exclude non-relevant items
                const relevantTypes = ['asset', 'dataExtension'];
                if (!relevantTypes.includes(definition.type)) {
                    return true;
                }
                if (
                    definition.type === 'dataExtension' &&
                    item.r__folder_ContentType !== 'shared_dataextension'
                ) {
                    // only shared DEs need a suffix
                    return true;
                }

                // actual test
                const key = item[definition.keyField] + '';
                if (key) {
                    return key.endsWith(suffix);
                } else {
                    Util.logger.debug('validation-keySuffix: key not found');
                    return true;
                }
            },
        },
    };
}

To control how a custom rule is applied, simply add its name (in the above example, keySuffix) to options.validation like you would for the default rules. Because custom rules are configured in the same way as default rules, you can also use the same override logic for them

    "validation": {
            "retrieve": {
                "keySuffix": "warn",
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
            },
            "buildDefinition": {
                "keySuffix": "warn",
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
                "overrides": [
                    {
                        "type": [
                            "asset"
                        ],
                        "options": {
                            "keySuffix": "error",
                        }
                    }
                ]
            },
            "deploy": {
                "keySuffix": "error",
                "noGuidKeys": "error",
                "noRootFolder": "error",
            }
    },

Market Configuration

You will want to set up configs for variable parts that change between Business Units. We advise starting this after you've first run the buildTemplate or build command. This might sound counterintuitive, but when you review what was copied into your template folder you will likely spot these variable parts the fastest and can then start setting up your market config. Please consider this an iterative process, as you will likely run bt/build followed by additional config updates until you get it right.

We advise clustering your logical approach into variable things on the instance parent (=_ParentBU_ in Accenture SFMC DevTools), the environment parent (under which you cluster your child Business Units for DEV, QA, and PROD respectively), and child Business Units. Ideally, the instance parent is only used to deploy Shared Data Extensions, the environment parent is used for integrations with external services and to separate incoming data via Automations into the respective Shared Data Extensions. The child Business Units are then reserved for everything that is run on a market-by-market basis.

Note: We do see it often that instance parent and environment parent are the same. This depends on your client's setup since Business Units are not for free, and clients sometimes decide to save the extra money. Sometimes, you even end up with only one BU for DEV activities, no QA environment - and share the instance parent between DEV and production... This is not the recommended approach for multiple reasons, including security, but it is the reality in some of our projects.

Here is a simple example with one DEV BU, 1 QA BU, and 2 PROD BUs:

// example market config in your .mcdevrc.json
"markets": {
        "DEV-NL": {
            "mid": "12345",
            "buName": "DEV - Child NL",
            "sharedFolder": "/Shared Data Extensions/DEV/NL",
            "suffix": "_DEV_NL",
            "countryCodeIn": "'NL'"
        },
        "QA-DE": {
            "mid": "12346",
            "buName": "QA - Child DE",
            "sharedFolder": "/Shared Data Extensions/QA/DE",
            "suffix": "_QA_DE",
            "countryCodeIn": "'DE'"
        },
        "PROD-DE": {
            "mid": "12349",
            "buName": "DE - Germany",
            "sharedFolder": "/Shared Data Extensions/DE - Germany",
            "suffix": "_DE",
            "countryCodeIn": "'DE'"
        },
        "PROD-NL": {
            "mid": "12351",
            "buName": "NL - Netherlands",
            "sharedFolder": "/Shared Data Extensions/NL - Netherlands",
            "suffix": "_NL",
            "countryCodeIn": "'NL'"
        }
}

Way more complex example with dedicated "Parent" BUs per environment (DEV, QA, PROD) and multiple country-specific BUs for QA and PROD:

// example market config in your .mcdevrc.json
"markets": {
    "DEV-Parent": {
        "Account_Salesforce": "Account_Salesforce_1",
        "suffix": "_DEV"
    },
    "DEV-NL": {
        "mid": "12345",
        "buName": "DEV - Child NL",
        "sharedFolder": "Shared Items/Shared Data Extensions/DEV/NL",
        "suffix": "_DEV_NL",
        "countryCodeIn": "'NL'"
    },
    "QA-Parent": {
        "Account_Salesforce": "Account_Salesforce_2",
        "suffix": "_QA"
    },
    "QA-DE": {
        "mid": "12346",
        "buName": "QA - Child DE",
        "sharedFolder": "Shared Items/Shared Data Extensions/QA/DE",
        "suffix": "_QA_DE",
        "countryCodeIn": "'DE'"
    },
    "QA-GULF": {
        "mid": "12347",
        "buName": "QA - Child GULF",
        "sharedFolder": "Shared Items/Shared Data Extensions/QA/GULF",
        "suffix": "_QA_GULF",
        "countryCodeIn": "'AE', 'BH', 'IQ', 'KW', 'OM', 'QA', 'SA'"
    },
    "PROD-Parent": {
        "Account_Salesforce": "Account_Salesforce",
        "suffix": ""
    },
    "PROD-AT": {
        "mid": "123458",
        "buName": "AT - Austria",
        "sharedFolder": "Shared Items/Shared Data Extensions/AT - Austria",
        "suffix": "_AT",
        "countryCodeIn": "'AT'"
    },
    "PROD-DE": {
        "mid": "12349",
        "buName": "DE - Germany",
        "sharedFolder": "Shared Items/Shared Data Extensions/DE - Germany",
        "suffix": "_DE",
        "countryCodeIn": "'DE'"
    },
    "PROD-CH": {
        "mid": "12352",
        "buName": "CH - Switzerland",
        "sharedFolder": "Shared Items/Shared Data Extensions/CH - Switzerland",
        "suffix": "_CH",
        "countryCodeIn": "'CH'"
    },
    "PROD-GULF": {
        "mid": "12350",
        "buName": "GULF - Arab states of the Persian Gulf",
        "sharedFolder": "Shared Items/Shared Data Extensions/GULF - Arab states of the Persian Gulf",
        "suffix": "_IC_GULF",
        "countryCodeIn": "'AE', 'BH', 'IQ', 'KW', 'OM', 'QA', 'SA'"
    },
    "PROD-NL": {
        "mid": "12351",
        "buName": "NL - Netherlands",
        "sharedFolder": "Shared Items/Shared Data Extensions/NL - Netherlands",
        "suffix": "_NL",
        "countryCodeIn": "'NL'"
    }
}

Market List Configuration

Market Lists are very powerful and you will quickly notice how much time they can save you during your deployment preparation. Let's first look at an example list config:

// this is a market-list-config based on the example market-config listed above!
"markets": {...},
"marketList": {
    "deployment-source": {
        "description": "Define one 1:1 BU-Market combo here to as source for automated creation of deployment packages; you can create more than one source market list",
        "MySandbox/DEV-NL": "DEV-NL"
    },
    "deployment-target": {
        "description": "Define n BU-Market combo here to as target for automated creation of deployment packages; you can create more than one target market list and they can be as complex as you like",
        "MySandbox/QA-DE": "QA-DE",
        "MyProduction/PROD-DE": "PROD-DE",
        "MyProduction/PROD-NL": "PROD-NL"
    }
}

The above can be used together with createDeltaPkg command.

Way more complex example:

// this is a market-list-config based on the example market-config listed above!
"markets": {...},
"marketList": {
    "Parent-shared": {
        "description": "used to deploy shared data extensions",
        "MySandbox/_ParentBU_": ["QA-DE", "QA-GULF"],
        "MyProduction/_ParentBU_": ["PROD-AT", "PROD-CH", "PROD-DE", "PROD-GULF", "PROD-NL"]
    },
    "Parent-medium": {
        "description": "if you use the instance's parent BU also for imports",
        "MySandbox/_ParentBU_": "QA-Parent",
        "MyProduction/_ParentBU_": "PROD-Parent"
    },
    "Parent-large": {
        "description": "very large projects often decide against using the instance parent to get more order into the chaos and define their own 'parents' instead",
        "MySandbox/QA-_Parent_": "QA-Parent",
        "MyProduction/ProjectX-_Parent_": "PROD-Parent"
    },
    "Parent-medium-multi": {
        "description": "equal to Parent-shared"
    },
    "Parent-large-multi": {
        "description": "use to deploy market-adapted queries to your parent BUs",
        "MySandbox/QA-_Parent_": ["QA-DE", "QA-GULF"],
        "MyProduction/ProjectX-_Parent_": [
            "PROD-AT",
            "PROD-CH",
            "PROD-DE",
            "PROD-GULF",
            "PROD-NL"
        ]
    },
    "Children": {
        "description": "use this to deploy to your market BUs",
        "MySandbox/QA-DE": "QA-DE",
        "MySandbox/QA-GULF": "QA-GULF",
        "MyProduction/PROD-Child_AT": "PROD-AT",
        "MyProduction/PROD-Child_CH": "PROD-CH",
        "MyProduction/PROD-Child_DE": "PROD-DE",
        "MyProduction/PROD-Child_GULF": "PROD-GULF",
        "MyProduction/PROD-Child_NL": "PROD-NL"
    }
}

First off, we don't see DEV in here. If you have more than one market in DEV then this might deviate but in general, you don't want to bulk-deploy to DEV as this is your single source of truth. Apart from that, we can see 4 types of lists here:

  1. Parent-shared (instance parent): This would be used to deploy the Shared Data Extensions to the instance parent. The child-configs are listed in an array to ensure we end up with one file per child in our parent BU folder.
  2. Parent-medium/Parent-large (medium:instance parent; large:environment parent): A 1:1 config that handles automations and the part of your solution that only runs on the parent.
  3. Parent-medium-multi/Parent-large-multi (medium:instance parent; large:environment parent): Any scripts, queries, automations that are executed on the parent but require one per child (e.g. query to fill country-specific Shared Data Extensions)
  4. Children (child BUs): everything that is needed on the market-specific Business Units.

CLI Options

--api=[log|api]

Optionally, you can see API requests & responses in the logfile or directly in CLI.

Examples:

mcdev retrieve --api=log
mcdev deploy --api=cli

--noLogFile

Optionally, you can disable creating the 2 logfiles when running mcdev.

Examples:

mcdev retrieve --noLogFile
mcdev deploy --noLogFile

--noLogColors

Optionally, you can disable colors in CLI output. This is useful if you integrate mcdev into systems that cannot display color codes.

Examples:

mcdev retrieve --noLogColors
mcdev deploy --noLogColors

--errorLog

Optionally, mcdev can create a second log file ending on -error.log that only contains log entries of type "error" for quick parsing and potential visualization in a GUI. This file will not contain extended debug info but can make it easier to show errors to users when you don't want to parse the entire log file.

Note: This will not work in combination with --noLogFile

Examples:

mcdev retrieve --errorLog
mcdev deploy --errorLog

Copyright (c) 2020-2024 Accenture. MIT licensed. Main contributors: Jörn Berkefeld, Doug Midgley

Clone this wiki locally