Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7.4.0 #1642

Merged
merged 70 commits into from
Sep 4, 2024
Merged

7.4.0 #1642

merged 70 commits into from
Sep 4, 2024

Conversation

JoernBerkefeld
Copy link
Contributor

@JoernBerkefeld JoernBerkefeld commented Sep 2, 2024

Release details

Checklist

Before merge

  • Wiki updated with info in ticket listed under Documentation
  • ran npm run prepare-release (which runs npm audit fix, npm run lint-ts, npm run lint:fix, git add, git commit)
  • pushed potential changes made by prepare-release

After merge

  • merged all dependabot PRs that target main branch
  • updated bug template to include the new version
  • updated .mcdevrc for tests to the new version
  • ran npm run version:major/minor/patch
  • pushed version-prep commits
  • merged main branch into develop branch
  • closed GitHub milestone
  • created new GitHub Release

Documentation

#1642 (comment)

#1642 (comment)

#1642 (comment)

#1642 (comment)

#1642 (comment)

#1642 (comment)

Issues

JoernBerkefeld and others added 20 commits July 25, 2024 10:23
…deployed-for-inter-type-dependencies

# Conflicts:
#	@types/lib/index.d.ts.map
#	@types/lib/metadataTypes/Asset.d.ts.map
#	@types/lib/metadataTypes/MetadataType.d.ts.map
#	@types/lib/util/util.d.ts.map
…ighting-for-ssjs-files-in-gitlab

task/1637 enable syntax highlighting for SSJS files in gitlab
…-leaves-journeys-with-wrong-emails

bug/1505 fix bad asset links on journeys when retrieved together with triggeredSends
…deployed-for-inter-type-dependencies

# Conflicts:
#	lib/index.js
…-assets-are-deployed-for-inter-type-dependencies

BUG/1472 adapt order in which assets are deployed for inter-type dependencies
@JoernBerkefeld JoernBerkefeld added the bug Something isn't working label Sep 2, 2024
@JoernBerkefeld JoernBerkefeld added this to the 7.3.2 milestone Sep 2, 2024
@JoernBerkefeld JoernBerkefeld self-assigned this Sep 2, 2024
@JoernBerkefeld JoernBerkefeld added c/asset COMPONENT c/journey COMPONENT c/triggeredSend COMPONENT labels Sep 2, 2024
Copy link

github-actions bot commented Sep 2, 2024

Coverage Report

Commit:1094a21
Base: main@f365dba

Type Base This PR
Total Statements Coverage  70.09%  69.84%  (-0.25%)
Total Branches Coverage  68.85%  68.72%  (-0.13%)
Total Functions Coverage  82.11%  81.55%  (-0.56%)
Total Lines Coverage  70.09%  69.84%  (-0.25%)
Details (changed files):
File Statements Branches Functions Lines

@JoernBerkefeld JoernBerkefeld changed the title 7.4 7.4.0 Sep 2, 2024
JoernBerkefeld and others added 22 commits September 3, 2024 13:32
…ion-rule-overrides-for-selected-types

Feature/1666 optional validation rule overrides for selected types
…_checks_for_events_and_journeys_fail_if_multiple_entries_are_retrieved_by_key

# Conflicts:
#	@types/lib/metadataTypes/Journey.d.ts.map
#	lib/metadataTypes/Journey.js
…_events_and_journeys_fail_if_multiple_entries_are_retrieved_by_key

Bug/1668 sf object checks for events and journeys fail if multiple entries are retrieved by key
…locking-validation-errors-to-warnings

feature/1671 add --skipValidation option to deploy, build, buildDefinition, buildDefinitionBulk
…ven if the were omitted due to templating"

This reverts commit a6cb91f.
@JoernBerkefeld JoernBerkefeld merged commit 2720364 into main Sep 4, 2024
16 checks passed
@JoernBerkefeld
Copy link
Contributor Author

JoernBerkefeld commented Sep 16, 2024

https://github.com/Accenture/sfmc-devtools/wiki/06.b-~-Standard-Commands/_edit

Documentation

execute

Command: mcdev execute <business unit> [type] [key] [--like] [--schedule] [--metadata]

Alias: mcdev exec / mcdev start / mcdev resume

Currently supported types:

Name CLI Argument Effect
Automation automation RunOnce or Schedules automation according to already existing schedule
Query query Starts query execution
Journey journey Resumes a paused journey

[...]

publish

Command: mcdev publish <business unit> [metadata type] [metadata key] [--like] [--skipStatusCheck] [--metadata]

Alias: mcdev activate

Currently supported types:

Name CLI Argument Effect
Journey journey Activates a Draft-version of a journey (creates related transactionalEmail or event record)

@JoernBerkefeld
Copy link
Contributor Author

JoernBerkefeld commented Sep 16, 2024

https://github.com/Accenture/sfmc-devtools/wiki/06.b-~-Standard-Commands/_edit

Documentation

deploy

Command: mcdev deploy [business unit] [metadata type] [metadata key] [--metadata] [--fromRetrieve] [--refresh] [--keySuffix] [--noMidSuffix] [--changeKeyValue=yourNewKey] [--changeKeyField=otherFieldInJson] [--execute] [--schedule] [--fixShared] [--noUpdate] [--publish] [--skipStatusCheck] [--matchName] [--skipValidation]

deploy with --matchName:

Currently supported types:

Name CLI Argument
Asset asset
Data Extension dataExtension

@JoernBerkefeld
Copy link
Contributor Author

JoernBerkefeld commented Sep 16, 2024

Documentation - Advanced Configuration

Config options

.mcdevrc.json

The central config in .mcdevrc.json holds multiple adjustable settings:

{
  "options": {
    "deployment": {
      "commitHistory": 10,
      "sourceTargetMapping": {
        "deployment-source": "deployment-target"
      },
      "targetBranchBuMapping": {
        "release/*": "MySandbox/QA-DE",
        "master": ["MyProduction/PROD-DE", "MyProduction/PROD-NL"]
      }
    },
    "validation": {
            "retrieve": {
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
                "overrides": [
                    {
                        "type": [
                            "journey"
                        ],
                        "options": {
                            "noGuidKeys": "off",
                        }
                    }
                ]
            },
            "buildDefinition": {
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
                "overrides": [
                    {
                        "type": [
                            "journey"
                        ],
                        "options": {
                            "noGuidKeys": "off",
                        }
                    }
                ]
            },
            "deploy": {
                "noGuidKeys": "error",
                "noRootFolder": "error",
                "overrides": [
                    {
                        "type": [
                            "journey"
                        ],
                        "options": {
                            "noGuidKeys": "off",
                        }
                    }
                ]
            }
    },
    "documentType": "md",
    "documentStandardRoles": true,
    "exclude": {
      "role": {
        "CustomerKey": ["excludedRoleKey","excludedOtherRoleKey"]
      }
    },
    "include": {
      "asset": {
        "r__folder_Path": ["Content Builder/only/assets/in/here"]
      },
    },
    "serverTimeOffset": -6
  },
  "directories": {
        "businessUnits": "businessUnits/",
        "deploy": "deploy/",
        "docs": "docs/",
        "retrieve": "retrieve/",
        "template": "template/",
        "templateBuilds": ["retrieve/", "deploy/"]
  },
  "metaDataTypes": {
    "documentOnRetrieve": ["user", "automation", "dataExtension", "role"],
    "retrieve": [...]
  }
}
Setting Default Description
options.deployment.commitHistory 10 Configures how many commits createDeltaPkg will display if no parameters are given
options.deployment.sourceTargetMapping {"deployment-source": "deployment-target"} Configuration of 1 or many source-target marketList combos for mcdev createDeltaPkg
options.deployment.targetBranchBuMapping {"release/*": "...","master": ["..."]} Can be used by CI/CD pipelines to know what BUs shall be deployed to upon a merge into one of the specified branches
options.validation current default and custom rules Allows setting validation rules to "off", "warn" or "error
options.documentType 'md' Defines the format for documentation ('md', 'html', 'both')
options.documentStandardRoles false Optionally skip standard role documentation by setting to false
options.exclude.type.field [] Allows you to filter out metadata on retrieve based on their field values, e.g. CustomerKey (previously options.filter)
options.include.type.field [] Allows you to filter out metadata on retrieve based on their field values, e.g. CustomerKey
options.serverTimeOffset -6 Used to work around quirks in how SFMC handles timezones; For stack4: set to -7 (US Mountain time); others: -6 (US Central)
directories.businessUnits 'businessUnits/' Directory to save BU base details in
directories.deploy 'deploy/' Where deploy searches for files to deploy
directories.docs 'docs/' Directory for document output
directories.retrieve 'retrieve/' Where retrieve stores downloaded files
directories.template 'template/' Where rt stores downloaded templates & bd retrieves them from
directories.templateBuilds ['retrieve/','deploy/'] Where bd saves final deployment versions in. This can hold multiple directories, e.g. ['retrieve/','deploy/']
metaDataTypes.documentOnRetrieve ['user', 'automation', 'dataExtension', 'role'] automatically executes document for selected types
metaDataTypes.retrieve changes with each release check Metadata Type Support for current list

Validation rules

You can define validation rules for retrieve, buildDefinition and deploy.
The following rules exist out of the box:

  • noGuidKeys: test for metadata that has a GUID / UUID as key.
  • noRootFolder: test if metadata that does exist in a folder resides in the root for that particular type or in a subfolder.

Possible rule settings are:

  • not set, which implicitly turns off the rule
  • "off" explicitly turns off the rule
  • "warn" shows a log message of type warning
  • "error" shows a log message of type error AND blocks further execution
    • for retrieve that prevents download
    • for buildDefintion (and build) that prevents the creation of the corresponding file in the deploy folder
    • for deploy this will prevent deployment.

Please keep in mind that you can always skip validation rules all together at run-time by adding --skipValidation to your command (deploy / build / buildDefinition / buildDefinitionBulk)

Custom Validation rules - .mcdev-validation.js

Optionally one can create a file named .mcdev-validations.js in the root of your project to specify your own validation rules. That gives you full flexibility to test for whatever guidelines you might have as you can literally parse the JSON of the metadata yourself.

One example that depends on BU names and hence is not part of the standard set could look like this:

'use strict';

const buSuffixMap = {
    _ParentBU_: '',
    DEV: '_DEV',
    QA: '_QA',
    PROD: '',
};

/**
 *
 * @param {any} definition type defintiion
 * @param {any} item metadata json
 * @param {string} targetDir where the metadata is stored ("deploy/cred/bu")
 * @param {any} Util helper methods
 * @returns {Promise.<any>} validation rule
 */
export function validation(definition, item, targetDir, Util) {
    const bu = targetDir.includes('/') ? targetDir.split('/').pop() : targetDir.split('\\').pop();
    const suffix = buSuffixMap[bu];

    if (suffix === undefined) {
        Util.logger.error(
            `BU '${bu}' not defined for keySuffix validation in .mcdev-validations.js`
        );
    }
    return {
        keySuffix: {
            failedMsg: 'Key Suffix expected but not found: ' + suffix,
            /**
             * @returns {boolean} true=test passed
             */
            passed: function () {
                // exclude non-relevant items
                const relevantTypes = ['asset', 'dataExtension'];
                if (!relevantTypes.includes(definition.type)) {
                    return true;
                }
                if (
                    definition.type === 'dataExtension' &&
                    item.r__folder_ContentType !== 'shared_dataextension'
                ) {
                    // only shared DEs need a suffix
                    return true;
                }

                // actual test
                const key = item[definition.keyField] + '';
                if (key) {
                    return key.endsWith(suffix);
                } else {
                    Util.logger.debug('validation-keySuffix: key not found');
                    return true;
                }
            },
        },
    };
}

To control how a custom rule is applied, simply add its name (in the above example, keySuffix) to options.validation like you would for the default rules. Because custom rules are configured in the same way as default rules, you can also use the same override logic for them

    "validation": {
            "retrieve": {
                "keySuffix": "warn",
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
            },
            "buildDefinition": {
                "keySuffix": "warn",
                "noGuidKeys": "warn",
                "noRootFolder": "warn",
                "overrides": [
                    {
                        "type": [
                            "asset"
                        ],
                        "options": {
                            "keySuffix": "error",
                        }
                    }
                ]
            },
            "deploy": {
                "keySuffix": "error",
                "noGuidKeys": "error",
                "noRootFolder": "error",
            }
    },

@JoernBerkefeld
Copy link
Contributor Author

https://github.com/Accenture/sfmc-devtools/wiki/06.c-~-Templating-Commands

Documentation

Templating Commands

build

Command: mcdev build <--buFrom> <--marketFrom> <--buTo> <--marketTo> <--metadata> [--bulk] [--dependencies] [--retrieve] [--skipValidation]

buildDefinition

Command: mcdev buildDefinition <business unit> [type] [template name] [market] [--metadata] [--market] [--skipValidation]

buildDefinitionBulk

Command: mcdev buildDefinitionBulk <market list name> <type> <template name> [--metadata] [--skipValidation]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment