Merge remote-tracking branch 'origin/v1.2.0'

README.md

@@ -45,6 +45,7 @@ The tool needs to be configured before using it. The following steps describe th
     - `enableReferencedItemCheck` - This setting is used to enable or disable the check existence of a referenced item in any form fields value. When enabled, the tool will not populate fields that refers an item that doesn't exist in the destination Sitecore master database.
     - `analysis_ExcludeBaseStandardFields` - This setting is used to exclude base standard fields of Sitecore items from the list of fields that are reported about in the *conversion analysis report* output. The exclusion of the base standard fields from the analysis reporting process doesn't exclude them from the conversion process.
     - `includeOnlyFormIds` - This setting is used to select a subset of source forms to convert, specifying an array of source form IDs. This selection applies to both forms items conversion process and forms data migration process.
+    - `excludeFormIds` - This setting is used to exclude a subset of source forms from the conversion process, specifying an array of source form IDs. This selection applies to both forms items conversion process and forms data migration process.
 
 ## How to Use the Tool
 The tool should be executed in a Command Prompt window application in order to control its input parameters and visualize the execution progress. 
@@ -58,6 +59,7 @@ In the context of a Sitecore upgrade project, if all Sitecore content items have
     - `WFFM.ConversionTool.exe help` or `WFFM.ConversionTool.exe ?` - Use this option to get a list of available input parameters.
     - `WFFM.ConversionTool.exe -convert` - Use this option to execute the conversion and migration of both forms items and their stored data to the destination databases.
     - `WFFM.ConversionTool.exe -convert -nodata` - Use this option to execute the conversion and migration of forms items only. Stored forms data will not be converted and migrated.
+    - `WFFM.ConversionTool.exe -convert -onlydata` - Use this option to execute the conversion and migration of forms data only. Only data of forms previously converted and migrated on the destination instance will be converted and migrated by this process. Data of forms which items don't exist on the destination instance will not be converted and migrated.
 4) After the tool execution is finished successfully, if the destination Sitecore instance was running in IIS while the tool was executed, recycle the app pool of the destination Sitecore web application to clear the Sitecore Items cache.
 5) Once you login in the destination Sitecore instance, the migrated forms will not be listed in the Forms section (accessible from the Sitecore Desktop) until the `sitecore_master_index` index is rebuilt. Use the Indexing Manager in the Sitecore Control Panel to rebuild it.
 
@@ -113,6 +115,8 @@ The *analysis convertion report* contains the following columns for each record:
 - `ItemLanguage` - Language of the source item.
 - `ItemTemplateId` - Template Id of the source item.
 - `ItemTemplateName` - Template Name of the source item.
+- `FormId` - Id of the related source form item.
+- `FormName` - Name of the related source form item.
 - `FieldId` - Id of the field of the souce item.
 - `FieldName` - Name of the field of the source item.
 - `FieldType` - Type of the field of the source item.
@@ -127,7 +131,7 @@ The *analysis convertion report* contains the following columns for each record:
     - *Source Field Element Value Not Mapped* - The source field XML element value is not mapped.
     - *Form Field Item Not Mapped - Form Field Type Name = field-type-name* - The form field type is not mapped, because the field is a custom field type. Form fields that are not mapped are still migrated and converted using the default destination *Input* form field type. 
 
-## WFFM Conversion Tool Extensions - NEW !!!
+## WFFM Conversion Tool Extensions
 The v1.1.0 release of the WFFM Conversion Tool introduces the availability of the *WFFM Conversion Tool Extensions*, a group of plugins that add support for the conversion of WFFM forms items that don't exist in the Sitecore Forms out-of-the-box solution, but that are extended using popular Sitecore modules available on the Sitecore Marketplace.
 
 The WFFM Conversion Tool Extensions plugins are available in the `Extensions` folder in the tool root folder. Each *extension* plugin has its own subfolder that contains the plugin files and a `readme_<module_name>.txt` file that describes step-by-step instructions to install each plugin. 
@@ -155,7 +159,7 @@ The `baseTemplateMetadataFileName` property should contain the name of the metad
 A useful online resource to validate the content of a metadata file is the [JSON Schema Validator](https://www.jsonschemavalidator.net/). The JSON schema of the metadata file is included in the tool `Schemas` folder and also available [here](https://github.com/afaniuolo/WFFM-Conversion-Tool/blob/master/src/WFFM.ConversionTool/Schemas/metadata-schema.json).
 
 ### How to Customize the Conversion of a Field Value
-The *fieldConverter* property of *convertedField* mapping objects in metadata files allows to specify the name of a converter to process the source value of a field. Converters are defined in the `AppSettings.json` configuratio file in the *converters* property. This property is an array of *converter* objects that have the following properties:
+The *fieldConverter* property of *convertedField* mapping objects in metadata files allows to specify the name of a converter to process the source value of a field. Converters are defined in the `AppSettings.json` configuration file in the *converters* property. This property is an array of *converter* objects that have the following properties:
 - `name` - Name of the converter, used as value in the *fieldConverter* property in metadata files.
 - `converterType` - Type of the class object that defines the converter.
 

src/WFFM.ConversionTool.Library/Converters/ItemConverter.cs

@@ -234,10 +234,13 @@ private List<SCItem> ConvertFields(SCItem destItem, SCItem lastDescendantItem)
 				}
 			}
 
-			// Create new fields
-			foreach (var newField in _itemMetadataTemplate.fields.newFields)
+			if (_itemMetadataTemplate.fields.newFields != null)
 			{
-				destFields.AddRange(_fieldFactory.CreateFields(newField, itemId, langVersions, languages));
+				// Create new fields
+				foreach (var newField in _itemMetadataTemplate.fields.newFields)
+				{
+					destFields.AddRange(_fieldFactory.CreateFields(newField, itemId, langVersions, languages));
+				}
 			}
 
 			destItem.Fields = destFields;

src/WFFM.ConversionTool.Library/Converters/SubmitConverter.cs

@@ -48,7 +48,7 @@ public void Convert(SCItem form, SCItem pageItem)
 			string tracking = form.Fields.FirstOrDefault(field => field.FieldId == new Guid(FormConstants.FormTrackingFieldId))?.Value;
 
 			// Save Data Action
-			ConvertSaveDataAction(form, buttonItem);
+			bool saveToDatabaseActionCreated = ConvertSaveDataAction(form, buttonItem);
 
 			// Convert Submit Mode
 			ConvertSubmitMode(form, buttonItem);
@@ -57,7 +57,7 @@ public void Convert(SCItem form, SCItem pageItem)
 			ConvertTriggerGoal(form, tracking, buttonItem);
 
 			// Convert Other Save Actions
-			ConvertSaveActions(form, tracking, buttonItem);
+			ConvertSaveActions(form, tracking, buttonItem, saveToDatabaseActionCreated);
 		}
 
 		private SCItem ConvertSubmitButton(SCItem form, SCItem pageItem)
@@ -92,12 +92,12 @@ private SCItem ConvertSubmitButton(SCItem form, SCItem pageItem)
 			return buttonItem;
 		}
 
-		private void ConvertSaveDataAction(SCItem form, SCItem buttonItem)
+		private bool ConvertSaveDataAction(SCItem form, SCItem buttonItem)
 		{
 			var saveDataValues = new Dictionary<Guid, string>();
 			saveDataValues.Add(new Guid(SubmitActionConstants.SubmitActionFieldId), SubmitActionConstants.SubmitActionField_SaveActionValue);
 			saveDataValues.Add(new Guid(BaseTemplateConstants.SortOrderFieldId), "0");
-			ConvertSourceFieldToSubmitActionItem(form, new Guid(FormConstants.FormSaveToDatabaseFieldId), "1", "Save Data", saveDataValues, buttonItem);
+			return ConvertSourceFieldToSubmitActionItem(form, new Guid(FormConstants.FormSaveToDatabaseFieldId), "1", "Save Data", saveDataValues, buttonItem);
 		}
 
 		private void ConvertSubmitMode(SCItem form, SCItem buttonItem)
@@ -196,7 +196,7 @@ private void ConvertTriggerGoal(SCItem form, string tracking, SCItem buttonItem)
 			}
 		}
 
-		private void ConvertSaveActions(SCItem form, string tracking, SCItem buttonItem)
+		private void ConvertSaveActions(SCItem form, string tracking, SCItem buttonItem, bool saveToDatabaseActionCreated)
 		{
 			var formSaveActionField = form.Fields
 				.FirstOrDefault(field => field.FieldId == new Guid(FormConstants.FormSaveActionFieldId));
@@ -246,7 +246,7 @@ private void ConvertSaveActions(SCItem form, string tracking, SCItem buttonItem)
 								}
 							}
 						}
-						else if (saveActionItem.Id == FormConstants.FormSaveAction_SaveToDatabaseValue)
+						else if (!saveToDatabaseActionCreated && saveActionItem.Id == FormConstants.FormSaveAction_SaveToDatabaseValue)
 						{
 							ConvertSaveDataAction(form, buttonItem);
 						}
@@ -328,7 +328,7 @@ private void CreateItem(string metadataTemplateName, string destItemName, Dictio
 			WriteNewItem(metadataTemplate.destTemplateId, submitActionsFolder, destItemName, metadataTemplate);
 		}
 
-		private void ConvertSourceFieldToSubmitActionItem(SCItem form, Guid sourceFieldId,
+		private bool ConvertSourceFieldToSubmitActionItem(SCItem form, Guid sourceFieldId,
 			string sourceFieldValue, string destItemName, Dictionary<Guid, string> destFieldValues, SCItem buttonItem)
 		{
 			SCField sourceFieldToConvert = null;
@@ -340,7 +340,10 @@ private void ConvertSourceFieldToSubmitActionItem(SCItem form, Guid sourceFieldI
 			if (sourceFieldToConvert == null || sourceFieldToConvert.Value == sourceFieldValue)
 			{
 				CreateItem("SubmitActionDefinition", destItemName, destFieldValues, buttonItem);
+				return true;
 			}
+
+			return false;
 		}
 
 		private void ConvertFieldsToSubmitActionItem(string destItemName, Dictionary<Guid, string> destFieldValues, SCItem buttonItem)

src/WFFM.ConversionTool.Library/Helpers/XmlHelper.cs

@@ -150,7 +150,7 @@ private static string AddParentNodeAndEncodeElementValue(string fieldValue)
 				// Add parent xml element to value
 				fieldValue = string.Format("<ParentNode>{0}</ParentNode>", fieldValue);
 				// Escape special chars in text value
-				fieldValue = fieldValue.Replace(" &", " &amp;");
+				fieldValue = fieldValue.Replace(" & ", " &amp; ").Replace(" &<"," &amp;<");
 			}
 
 			return fieldValue;
@@ -163,7 +163,10 @@ private static string SanitizeFieldValue(string fieldValue)
 				.Replace("</b<","</b><")
 				.Replace("</a<","</a><")
 				.Replace("</strong<","</strong><")
-				.Replace("</i<","</i><");
+				.Replace("</i<","</i><")
+				.Replace("&rsquo;", "’")
+				.Replace("&lsquo;", "‘")
+				.Replace("&nbsp;"," ");
 		}
 	}
 }

src/WFFM.ConversionTool.Library/Migrators/DataMigrator.cs

@@ -89,6 +89,11 @@ private void MigrateFormData(Guid formId)
 				// Get Field Data records data from forms data provider
 				var fieldDataRecords = _dataProvider.GetFieldDataRecords(formDataRecord.Id);
 
+				if (fieldDataRecords == null || !fieldDataRecords.Any())
+				{
+					continue;
+				}
+
 				// Convert the source field data records
 				List<FieldData> fieldDataFormsRecords = fieldDataRecords.Select(data => ConvertFieldData(data, fieldValueTypeCollection)).ToList();
 
@@ -112,8 +117,8 @@ private FieldData ConvertFieldData(Database.WFFM.FieldData wffmFieldData, List<F
 				FieldName = wffmFieldData.FieldName,
 				FormEntryID = wffmFieldData.FormId,
 				ID = wffmFieldData.Id,
-				Value = ConvertFieldDataValue(wffmFieldData.Value, wffmFieldData.Data, collection.First(f => f.fieldId == wffmFieldData.FieldItemId)?.dataValueConverter),
-				ValueType = collection.First(f => f.fieldId == wffmFieldData.FieldItemId)?.dataValueType ?? "System.String"
+				Value = ConvertFieldDataValue(wffmFieldData.Value, wffmFieldData.Data, collection.FirstOrDefault(f => f.fieldId == wffmFieldData.FieldItemId)?.dataValueConverter),
+				ValueType = collection.FirstOrDefault(f => f.fieldId == wffmFieldData.FieldItemId)?.dataValueType ?? "System.String"
 			};
 		}
 

src/WFFM.ConversionTool.Library/Models/Metadata/AppSettings.cs

@@ -22,6 +22,7 @@ public class AppSettings
 		public bool analysis_ExcludeBaseStandardFields { get; set; }
 		public bool excludeSampleWffmForms { get; set; }
 		public List<Guid> includeOnlyFormIds { get; set; }
+		public List<Guid> excludeFormIds { get; set; }
 
 		public Dictionary<string, string> extensions { get; set; }
 

src/WFFM.ConversionTool.Library/Models/Reporting/ReportingRecord.cs

@@ -17,6 +17,8 @@ public class ReportingRecord
 		public string ItemLanguage { get; set; }
 		public string ItemTemplateId { get; set; }
 		public string ItemTemplateName { get; set; }
+		public string FormId { get; set; }
+		public string FormName { get; set; }
 		public string FieldId { get; set; }
 		public string FieldName { get; set; }
 		public string FieldType { get; set; }

src/WFFM.ConversionTool.Library/Processors/FormProcessor.cs

@@ -42,7 +42,7 @@ public class FormProcessor : ItemProcessor, IFormProcessor
 		private readonly string ButtonTemplateName = "button";
 
 		public FormProcessor(ILogger logger, ISourceMasterRepository sourceMasterRepository, AppSettings appSettings, IMetadataProvider metadataProvider,
-			IDestMasterRepository destMasterRepository, IItemConverter itemConverter, IItemFactory itemFactory, SubmitConverter submitConverter, 
+			IDestMasterRepository destMasterRepository, IItemConverter itemConverter, IItemFactory itemFactory, SubmitConverter submitConverter,
 			FormAppearanceConverter formAppearanceConverter, SectionAppearanceConverter sectionAppearanceConverter, IReporter conversionReporter)
 			: base(destMasterRepository, itemConverter, itemFactory, appSettings)
 		{
@@ -97,6 +97,12 @@ public List<Guid> ConvertForms()
 					forms = forms.Where(form => _appSettings.includeOnlyFormIds.Contains(form.ID)).ToList();
 				}
 
+				// Filter forms to exclude forms listed in appSettings "excludeFormIds" parameter
+				if (_appSettings.excludeFormIds != null && _appSettings.excludeFormIds.Any())
+				{
+					forms = forms.Where(form => !_appSettings.excludeFormIds.Contains(form.ID)).ToList();
+				}
+
 				// Filter sample forms out
 				if (_appSettings.excludeSampleWffmForms)
 				{
@@ -125,60 +131,72 @@ public List<Guid> ConvertForms()
 
 				foreach (var form in forms)
 				{
-					// Convert and Migrate Form items
-					ConvertAndWriteItem(form, _appSettings.itemReferences["destFormFolderId"]);
-
-					// Create Page item
-					var pageId = Guid.Empty;
-					SCItem pageItem = null;
-					if (!_destMasterRepository.ItemHasChildrenOfTemplate((Guid)destPageTemplateId, form))
-					{
-						// Create Page items for each form (only once)
-						pageId = WriteNewItem((Guid)destPageTemplateId, form, "Page");
-					}
-					else
-					{
-						// Get Page Item Id
-						pageItem = _destMasterRepository.GetSitecoreChildrenItems((Guid)destPageTemplateId, form.ID).FirstOrDefault(item => string.Equals(item.Name, "Page", StringComparison.InvariantCultureIgnoreCase));
-						pageId = pageItem?.ID ?? form.ID;
-					}
-					if (pageItem == null) pageItem = _destMasterRepository.GetSitecoreItem(pageId);
-
-					// Convert and Migrate Section items
-					var sections = _sourceMasterRepository.GetSitecoreChildrenItems((Guid)sourceSectionTemplateId, form.ID);
-					foreach (var section in sections)
-					{
-						ConvertAndWriteItem(section, pageId);
-						_sectionAppearanceConverter.ConvertTitle(section);
-						_sectionAppearanceConverter.ConvertInformation(section);
-					}
-
-					// Convert and Migrate Form Field items
-					List<SCItem> formFields = new List<SCItem>();
-					formFields.AddRange(_sourceMasterRepository.GetSitecoreChildrenItems((Guid)sourceFieldTemplateId, form.ID));
-					foreach (var section in sections)
+					try
 					{
-						formFields.AddRange(_sourceMasterRepository.GetSitecoreChildrenItems((Guid)sourceFieldTemplateId, section.ID));
+						// Set current form being processed in a global variable for reporting
+						_conversionReporter.CurrentFormId = form.ID.ToString("B").ToUpper();
+						_conversionReporter.CurrentFormName = form.Name;
+
+						// Convert and Migrate Form items
+						ConvertAndWriteItem(form, _appSettings.itemReferences["destFormFolderId"]);
+
+						// Create Page item
+						var pageId = Guid.Empty;
+						SCItem pageItem = null;
+						if (!_destMasterRepository.ItemHasChildrenOfTemplate((Guid)destPageTemplateId, form))
+						{
+							// Create Page items for each form (only once)
+							pageId = WriteNewItem((Guid)destPageTemplateId, form, "Page");
+						}
+						else
+						{
+							// Get Page Item Id
+							pageItem = _destMasterRepository.GetSitecoreChildrenItems((Guid)destPageTemplateId, form.ID).FirstOrDefault(item => string.Equals(item.Name, "Page", StringComparison.InvariantCultureIgnoreCase));
+							pageId = pageItem?.ID ?? form.ID;
+						}
+						if (pageItem == null) pageItem = _destMasterRepository.GetSitecoreItem(pageId);
+
+						// Convert and Migrate Section items
+						var sections = _sourceMasterRepository.GetSitecoreChildrenItems((Guid)sourceSectionTemplateId, form.ID);
+						foreach (var section in sections)
+						{
+							ConvertAndWriteItem(section, pageId);
+							_sectionAppearanceConverter.ConvertTitle(section);
+							_sectionAppearanceConverter.ConvertInformation(section);
+						}
+
+						// Convert and Migrate Form Field items
+						List<SCItem> formFields = new List<SCItem>();
+						formFields.AddRange(_sourceMasterRepository.GetSitecoreChildrenItems((Guid)sourceFieldTemplateId, form.ID));
+						foreach (var section in sections)
+						{
+							formFields.AddRange(_sourceMasterRepository.GetSitecoreChildrenItems((Guid)sourceFieldTemplateId, section.ID));
+						}
+
+						foreach (var formField in formFields)
+						{
+							var parentItem = _sourceMasterRepository.GetSitecoreItem(formField.ParentID);
+							var destParentId = parentItem.TemplateID == sourceFormTemplateId ? pageId : parentItem.ID;
+							ConvertAndWriteItem(formField, destParentId);
+						}
+
+						// Convert Submit form section fields
+						_submitConverter.Convert(form, pageItem);
+
+						// Convert Form Appearance fields
+						_formAppearanceConverter.ConvertTitle(form, pageItem);
+						_formAppearanceConverter.ConvertIntroduction(form, pageItem);
+						_formAppearanceConverter.ConvertFooter(form, pageItem);
+
+						formCounter++;
+						// Update progress bar
+						ProgressBar.DrawTextProgressBar(formCounter, forms.Count, $"forms {formAction}");
 					}
-
-					foreach (var formField in formFields)
+					catch (Exception ex)
 					{
-						var parentItem = _sourceMasterRepository.GetSitecoreItem(formField.ParentID);
-						var destParentId = parentItem.TemplateID == sourceFormTemplateId ? pageId : parentItem.ID;
-						ConvertAndWriteItem(formField, destParentId);
+						_logger.Log(new LogEntry(LoggingEventType.Error, string.Format("Error processing form ItemID = {0}", form.ID), ex));
+						throw;
 					}
-
-					// Convert Submit form section fields
-					_submitConverter.Convert(form, pageItem);
-
-					// Convert Form Appearance fields
-					_formAppearanceConverter.ConvertTitle(form, pageItem);
-					_formAppearanceConverter.ConvertIntroduction(form, pageItem);
-					_formAppearanceConverter.ConvertFooter(form, pageItem);
-
-					formCounter++;
-					// Update progress bar
-					ProgressBar.DrawTextProgressBar(formCounter, forms.Count, $"forms {formAction}");
 				}
 
 				if (_appSettings.enableOnlyAnalysisByDefault)

src/WFFM.ConversionTool.Library/Properties/AssemblyInfo.cs

@@ -32,5 +32,5 @@
 // You can specify all the values or you can default the Build and Revision Numbers
 // by using the '*' as shown below:
 // [assembly: AssemblyVersion("1.0.*")]
-[assembly: AssemblyVersion("1.1.1.0")]
-[assembly: AssemblyFileVersion("1.1.1.0")]
+[assembly: AssemblyVersion("1.2.0.0")]
+[assembly: AssemblyFileVersion("1.2.0.0")]