feature/#711: add support to update and refresh attributeGroup

docs/dist/documentation.md

@@ -17,6 +17,9 @@ Source and target business units are also compared before the deployment to appl
 <dt><a href="#AttributeGroup">AttributeGroup</a> ⇐ <code><a href="#MetadataType">MetadataType</a></code></dt>
 <dd><p>AttributeGroup MetadataType</p>
 </dd>
+<dt><a href="#AttributeSet">AttributeSet</a> ⇐ <code><a href="#MetadataType">MetadataType</a></code></dt>
+<dd><p>AttributeSet MetadataType</p>
+</dd>
 <dt><a href="#Automation">Automation</a> ⇐ <code><a href="#MetadataType">MetadataType</a></code></dt>
 <dd><p>Automation MetadataType</p>
 </dd>
@@ -105,9 +108,6 @@ Provides default functionality that can be overwritten by child metadata type cl
 <dt><a href="#SendClassification">SendClassification</a> ⇐ <code><a href="#MetadataType">MetadataType</a></code></dt>
 <dd><p>SendClassification MetadataType</p>
 </dd>
-<dt><a href="#SetDefinition">SetDefinition</a> ⇐ <code><a href="#MetadataType">MetadataType</a></code></dt>
-<dd><p>SetDefinition MetadataType</p>
-</dd>
 <dt><a href="#TransactionalEmail">TransactionalEmail</a> ⇐ <code><a href="#TransactionalMessage">TransactionalMessage</a></code></dt>
 <dd><p>TransactionalEmail MetadataType</p>
 </dd>
@@ -1219,6 +1219,7 @@ AttributeGroup MetadataType
 * [AttributeGroup](#AttributeGroup) ⇐ [<code>MetadataType</code>](#MetadataType)
     * [.retrieve(retrieveDir, [_], [__], [key])](#AttributeGroup.retrieve) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
     * [.retrieveForCache()](#AttributeGroup.retrieveForCache) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
+    * [.postRetrieveTasks(metadata)](#AttributeGroup.postRetrieveTasks) ⇒ <code>TYPE.MetadataTypeItem</code>
 
 <a name="AttributeGroup.retrieve"></a>
 
@@ -1242,6 +1243,87 @@ Retrieves Metadata of schema attribute groups for caching.
 
 **Kind**: static method of [<code>AttributeGroup</code>](#AttributeGroup)  
 **Returns**: <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code> - Promise of metadata  
+<a name="AttributeGroup.postRetrieveTasks"></a>
+
+### AttributeGroup.postRetrieveTasks(metadata) ⇒ <code>TYPE.MetadataTypeItem</code>
+manages post retrieve steps
+
+**Kind**: static method of [<code>AttributeGroup</code>](#AttributeGroup)  
+**Returns**: <code>TYPE.MetadataTypeItem</code> - metadata  
+
+| Param | Type | Description |
+| --- | --- | --- |
+| metadata | <code>TYPE.MetadataTypeItem</code> | a single metadata |
+
+<a name="AttributeSet"></a>
+
+## AttributeSet ⇐ [<code>MetadataType</code>](#MetadataType)
+AttributeSet MetadataType
+
+**Kind**: global class  
+**Extends**: [<code>MetadataType</code>](#MetadataType)  
+
+* [AttributeSet](#AttributeSet) ⇐ [<code>MetadataType</code>](#MetadataType)
+    * [.retrieve(retrieveDir, [_], [__], [key])](#AttributeSet.retrieve) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
+    * [.retrieveForCache()](#AttributeSet.retrieveForCache) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
+    * [.parseResponseBody(body, [singleRetrieve])](#AttributeSet.parseResponseBody) ⇒ <code>TYPE.MetadataTypeMap</code>
+    * [.postRetrieveTasks(metadata)](#AttributeSet.postRetrieveTasks) ⇒ <code>TYPE.MetadataTypeItem</code>
+    * [._getSystemValueDefinitions()](#AttributeSet._getSystemValueDefinitions) ⇒ <code>Array.&lt;object&gt;</code>
+
+<a name="AttributeSet.retrieve"></a>
+
+### AttributeSet.retrieve(retrieveDir, [_], [__], [key]) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
+Retrieves Metadata of schema set Definitions.
+
+**Kind**: static method of [<code>AttributeSet</code>](#AttributeSet)  
+**Returns**: <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code> - Promise  
+
+| Param | Type | Description |
+| --- | --- | --- |
+| retrieveDir | <code>string</code> | Directory where retrieved metadata directory will be saved |
+| [_] | <code>void</code> | unused parameter |
+| [__] | <code>void</code> | unused parameter |
+| [key] | <code>string</code> | customer key of single item to retrieve |
+
+<a name="AttributeSet.retrieveForCache"></a>
+
+### AttributeSet.retrieveForCache() ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
+Retrieves Metadata of schema set definitions for caching.
+
+**Kind**: static method of [<code>AttributeSet</code>](#AttributeSet)  
+**Returns**: <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code> - Promise  
+<a name="AttributeSet.parseResponseBody"></a>
+
+### AttributeSet.parseResponseBody(body, [singleRetrieve]) ⇒ <code>TYPE.MetadataTypeMap</code>
+Builds map of metadata entries mapped to their keyfields
+
+**Kind**: static method of [<code>AttributeSet</code>](#AttributeSet)  
+**Returns**: <code>TYPE.MetadataTypeMap</code> - keyField => metadata map  
+
+| Param | Type | Description |
+| --- | --- | --- |
+| body | <code>object</code> | json of response body |
+| [singleRetrieve] | <code>string</code> \| <code>number</code> | key of single item to filter by |
+
+<a name="AttributeSet.postRetrieveTasks"></a>
+
+### AttributeSet.postRetrieveTasks(metadata) ⇒ <code>TYPE.MetadataTypeItem</code>
+manages post retrieve steps
+
+**Kind**: static method of [<code>AttributeSet</code>](#AttributeSet)  
+**Returns**: <code>TYPE.MetadataTypeItem</code> - metadata  
+
+| Param | Type | Description |
+| --- | --- | --- |
+| metadata | <code>TYPE.MetadataTypeItem</code> | a single metadata |
+
+<a name="AttributeSet._getSystemValueDefinitions"></a>
+
+### AttributeSet.\_getSystemValueDefinitions() ⇒ <code>Array.&lt;object&gt;</code>
+helper for [postRetrieveTasks](#AttributeSet.postRetrieveTasks)
+
+**Kind**: static method of [<code>AttributeSet</code>](#AttributeSet)  
+**Returns**: <code>Array.&lt;object&gt;</code> - all system value definitions  
 <a name="Automation"></a>
 
 ## Automation ⇐ [<code>MetadataType</code>](#MetadataType)
@@ -5189,40 +5271,6 @@ Retrieves SOAP based metadata of metadata type into local filesystem. executes c
 | [__] | <code>void</code> | unused parameter |
 | [key] | <code>string</code> | customer key of single item to retrieve |
 
-<a name="SetDefinition"></a>
-
-## SetDefinition ⇐ [<code>MetadataType</code>](#MetadataType)
-SetDefinition MetadataType
-
-**Kind**: global class  
-**Extends**: [<code>MetadataType</code>](#MetadataType)  
-
-* [SetDefinition](#SetDefinition) ⇐ [<code>MetadataType</code>](#MetadataType)
-    * [.retrieve(retrieveDir, [_], [__], [key])](#SetDefinition.retrieve) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
-    * [.retrieveForCache()](#SetDefinition.retrieveForCache) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
-
-<a name="SetDefinition.retrieve"></a>
-
-### SetDefinition.retrieve(retrieveDir, [_], [__], [key]) ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
-Retrieves Metadata of schema set Definitions.
-
-**Kind**: static method of [<code>SetDefinition</code>](#SetDefinition)  
-**Returns**: <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code> - Promise  
-
-| Param | Type | Description |
-| --- | --- | --- |
-| retrieveDir | <code>string</code> | Directory where retrieved metadata directory will be saved |
-| [_] | <code>void</code> | unused parameter |
-| [__] | <code>void</code> | unused parameter |
-| [key] | <code>string</code> | customer key of single item to retrieve |
-
-<a name="SetDefinition.retrieveForCache"></a>
-
-### SetDefinition.retrieveForCache() ⇒ <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code>
-Retrieves Metadata of schema set definitions for caching.
-
-**Kind**: static method of [<code>SetDefinition</code>](#SetDefinition)  
-**Returns**: <code>Promise.&lt;TYPE.MetadataTypeMapObj&gt;</code> - Promise  
 <a name="TransactionalEmail"></a>
 
 ## TransactionalEmail ⇐ [<code>TransactionalMessage</code>](#TransactionalMessage)

lib/MetadataTypeDefinitions.js

@@ -6,6 +6,7 @@
 const MetadataTypeDefinitions = {
     asset: require('./metadataTypes/definitions/Asset.definition'),
     attributeGroup: require('./metadataTypes/definitions/AttributeGroup.definition'),
+    attributeSet: require('./metadataTypes/definitions/AttributeSet.definition'),
     automation: require('./metadataTypes/definitions/Automation.definition'),
     campaign: require('./metadataTypes/definitions/Campaign.definition'),
     contentArea: require('./metadataTypes/definitions/ContentArea.definition'),
@@ -32,7 +33,6 @@ const MetadataTypeDefinitions = {
     role: require('./metadataTypes/definitions/Role.definition'),
     script: require('./metadataTypes/definitions/Script.definition'),
     sendClassification: require('./metadataTypes/definitions/SendClassification.definition'),
-    setDefinition: require('./metadataTypes/definitions/SetDefinition.definition'),
     transactionalEmail: require('./metadataTypes/definitions/TransactionalEmail.definition'),
     transactionalPush: require('./metadataTypes/definitions/TransactionalPush.definition'),
     transactionalSMS: require('./metadataTypes/definitions/TransactionalSMS.definition'),

lib/MetadataTypeInfo.js

@@ -6,6 +6,7 @@
 const MetadataTypeInfo = {
     asset: require('./metadataTypes/Asset'),
     attributeGroup: require('./metadataTypes/AttributeGroup'),
+    attributeSet: require('./metadataTypes/AttributeSet'),
     automation: require('./metadataTypes/Automation'),
     campaign: require('./metadataTypes/Campaign'),
     contentArea: require('./metadataTypes/ContentArea'),
@@ -32,7 +33,6 @@ const MetadataTypeInfo = {
     role: require('./metadataTypes/Role'),
     script: require('./metadataTypes/Script'),
     sendClassification: require('./metadataTypes/SendClassification'),
-    setDefinition: require('./metadataTypes/SetDefinition'),
     transactionalEmail: require('./metadataTypes/TransactionalEmail'),
     transactionalPush: require('./metadataTypes/TransactionalPush'),
     transactionalSMS: require('./metadataTypes/TransactionalSMS'),

lib/index.js

@@ -173,15 +173,20 @@ class Mcdev {
                 }
             }
         }
-
+        const resultsObj = {};
         if (businessUnit === '*') {
             Util.logger.info(':: Retrieving all BUs for all credentials');
             let counter_credTotal = 0;
             for (const cred in properties.credentials) {
                 Util.logger.info(`:: Retrieving all BUs for ${cred}`);
                 let counter_credBu = 0;
                 for (const bu in properties.credentials[cred].businessUnits) {
-                    await this.#retrieveBU(cred, bu, selectedTypesArr, keys);
+                    resultsObj[`${cred}/${bu}`] = await this.#retrieveBU(
+                        cred,
+                        bu,
+                        selectedTypesArr,
+                        keys
+                    );
                     counter_credBu++;
                     Util.startLogger(true);
                 }
@@ -215,7 +220,12 @@ class Mcdev {
                 Util.logger.info(`:: Retrieving all BUs for ${cred}`);
                 let counter_credBu = 0;
                 for (const bu in properties.credentials[cred].businessUnits) {
-                    await this.#retrieveBU(cred, bu, selectedTypesArr, keys);
+                    resultsObj[`${cred}/${bu}`] = await this.#retrieveBU(
+                        cred,
+                        bu,
+                        selectedTypesArr,
+                        keys
+                    );
                     counter_credBu++;
                     Util.startLogger(true);
                 }
@@ -231,10 +241,27 @@ class Mcdev {
                 );
                 if (changelogOnly) {
                     return retrieveChangelog;
+                } else {
+                    resultsObj[`${cred}/${bu}`] = retrieveChangelog;
                 }
                 Util.logger.info(`:: Done\n`);
             }
         }
+
+        // merge all results into one object
+        for (const credBu in resultsObj) {
+            for (const type in resultsObj[credBu]) {
+                const base = resultsObj[credBu][type][0];
+
+                for (let i = 1; i < resultsObj[credBu][type].length; i++) {
+                    // merge all items into the first array
+                    Object.assign(base, resultsObj[credBu][type][i]);
+                }
+                resultsObj[credBu][type] = resultsObj[credBu][type][0];
+            }
+        }
+
+        return resultsObj;
     }
     /**
      * helper for {@link Mcdev.retrieve}
@@ -265,6 +292,7 @@ class Mcdev {
             // clean up old folders after types were renamed
             // TODO: Remove renamedTypes-logic 6 months after version 5 release
             const renamedTypes = {
+                attributeSet: 'setDefinition',
                 emailSend: 'emailSendDefinition',
                 event: 'eventDefinition',
                 fileLocation: 'ftpLocation',
@@ -339,9 +367,7 @@ class Mcdev {
                     null,
                     changelogOnly
                 );
-                if (changelogOnly) {
-                    return retrieveChangelog;
-                }
+                return retrieveChangelog;
             } catch (ex) {
                 Util.logger.errorStack(ex, 'mcdev.retrieve failed');
             }

lib/metadataTypes/AttributeGroup.js

@@ -1,7 +1,9 @@
 'use strict';
 
-const MetadataType = require('./MetadataType');
 const TYPE = require('../../types/mcdev.d');
+const MetadataType = require('./MetadataType');
+const Util = require('../util/util');
+const cache = require('../util/cache');
 
 /**
  * AttributeGroup MetadataType
@@ -18,7 +20,7 @@ class AttributeGroup extends MetadataType {
      * @param {string} [key] customer key of single item to retrieve
      * @returns {Promise.<TYPE.MetadataTypeMapObj>} Promise of metadata
      */
-    static retrieve(retrieveDir, _, __, key) {
+    static async retrieve(retrieveDir, _, __, key) {
         return super.retrieveREST(
             retrieveDir,
             '/hub/v1/contacts/schema/attributeGroups',
@@ -34,6 +36,78 @@ class AttributeGroup extends MetadataType {
     static retrieveForCache() {
         return super.retrieveREST(null, '/hub/v1/contacts/schema/attributeGroups');
     }
+
+    /**
+     * manages post retrieve steps
+     *
+     * @param {TYPE.MetadataTypeItem} metadata a single metadata
+     * @returns {TYPE.MetadataTypeItem} metadata
+     */
+    static postRetrieveTasks(metadata) {
+        // Member ID
+        delete metadata.mID;
+
+        // attributeSet
+        metadata.attributeSetIdentifiers = metadata.attributeSetIdentifiers.map((attributeSet) => {
+            try {
+                const key = cache.searchForField(
+                    'attributeSet',
+                    attributeSet.definitionID,
+                    'definitionID',
+                    'definitionKey'
+                );
+                if (key !== attributeSet.definitionKey) {
+                    throw new Error(
+                        `AttributeSet key mismatch. Found ${key} instead of ${attributeSet.definitionKey}`
+                    );
+                }
+                return key;
+            } catch (ex) {
+                Util.logger.warn(
+                    ` - ${this.definition.type} ${metadata[this.definition.keyField]} (for ${
+                        attributeSet.definitionKey
+                    }): ${ex.message}`
+                );
+                return attributeSet;
+            }
+        });
+
+        // requiredRelationships
+        // TODO: implement
+
+        // description is not returned by API when empty. Set to empty string to propose the field as an option to users
+        metadata.description ||= '';
+
+        // applicationKey is only used by system generated attribute groups and otherwise it's empty.
+        if (metadata.applicationKey === '') {
+            // remove useless field
+            delete metadata.applicationKey;
+        }
+
+        // connectingID.identifierType seems to be always set to 'FullyQualifiedName' - to be sure we check it here and remove it if it's the case
+        if (metadata.connectingID?.identifierType === 'FullyQualifiedName') {
+            // remove useless field
+            delete metadata.connectingID;
+        }
+
+        // containsSchemaAttributes is only true for system generated attribute groups and otherwise it's false.
+        if (!metadata.containsSchemaAttributes) {
+            delete metadata.containsSchemaAttributes;
+        }
+
+        // isSystemDefined is only true for system generated attribute groups and cannot be deployed
+        if (!metadata.isSystemDefined) {
+            delete metadata.isSystemDefined;
+        }
+
+        return metadata;
+    }
+    /**
+     * prepares for deployment
+     *
+     * @param {TYPE.MetadataTypeItem} metadata a single item
+     * @returns {TYPE.MetadataTypeItem} Promise
+     */
 }
 
 // Assign definition to static attributes