Merged PR 56335: Update Order Sync to send data via API v3

## What's being changed

We've updated our batch processing and exporting system for enhanced flexibility and modularity. New interfaces and classes have been introduced, existing exporter classes have been aligned with new definitions, and the handling of transactional data imports has been improved.

## Why it's being changed

The reason for these changes is to improve the performance and efficiency of the batch processing and exporting system. By introducing new interfaces and refactoring existing classes, we aim to support new data processing strategies and enhance the order synchronization process.

These changes also ensure better logging and error handling during the sync process, which is crucial for maintaining data integrity and debugging issues. Additionally, the new structures provide a more modular and flexible architecture, allowing for easier maintenance and future enhancements.

## How to review/test this change

### Test Sync

Ensure you are using the branch on https://dev.azure.com/dotdigital/ec/_git/dd-php/pullrequest/57314 for the SDK models required for the importer sync progress checks.

1. Verify the new interfaces and their implementations.
2. Check the integration of `SdkOrderRecordCollectionBuilder` in the export process.
3. Smoke test `GuestExporter`, `SubscriberExporter`, and `Dotdigitalgroup\Email\Model\Sync\Customer\Exporter`.
4. Test the `ImporterQueueManager` with `TransactionalBulkJsonFactory`.
5. Validate the refactored `Order` class for correct order synchronization.
6. Confirm the `Exporter` class functionality with `SdkOrderRecordCollectionBuilderFactory`.
7. Test the `BulkJson` class for `TransactionalData` processing.
8. Test using multiple stores and ensure the loop by website works as expected, including orders disabled for one website even after it has orders in the `email_order` table.
9. Run importer sync. Ensure the BULK_JSON type gets handled correctly by the Progress report handler.
10. Observe a status indicator of 2 within the database record for your test sync.

### Test UI

1. Navigate to the import reports page.
2. Ensure the BULK_JSON records are visible as BULK.
3. Test the BULK filter option; ensure it includes the BULK_JSON records displayed as BULK.

**Notes**
- Review the logging outputs for any potential issues during the sync process.
- Pay special attention to the new `SdkOrderRecordCollectionBuilder` class and its integration, as it plays a crucial role in the export process.
- Verify that the refactored `Order` class correctly handles order synchronization with the new factories and interfaces.

Related work items: #238504

Api/Model/Sync/Batch/BatchMergerInterface.php

@@ -0,0 +1,17 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Batch;
+
+interface BatchMergerInterface
+{
+    /**
+     * Merge two batches of data.
+     *
+     * @param array $batch
+     * @param array $megaBatch
+     * @return array
+     */
+    public function mergeBatch(array $batch, array $megaBatch);
+}

Api/Model/Sync/Batch/BatchProcessorInterface.php

@@ -0,0 +1,18 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Batch;
+
+interface BatchProcessorInterface
+{
+    /**
+     * Processes a batch of data for a specific import type and website.
+     *
+     * @param array $batch An array of data to be processed.
+     * @param int $websiteId The ID of the website the batch is associated with.
+     * @param string $importType The type of import, which determines the processing logic to be applied.
+     * @param string $bulkImportMode The type of import mode.
+     */
+    public function process(array $batch, int $websiteId, string $importType, string $bulkImportMode);
+}

Api/Model/Sync/Batch/BatchStrategyFactoryInterface.php

@@ -0,0 +1,16 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Batch;
+
+interface BatchStrategyFactoryInterface
+{
+    /**
+     * Creates a batch strategy instance based on the specified import type.
+     *
+     * @param string $importType The type of import for which the strategy is created.
+     * @return BatchStrategyInterface
+     */
+    public function create(string $importType): BatchStrategyInterface;
+}

Api/Model/Sync/Batch/BatchStrategyInterface.php

@@ -0,0 +1,19 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Batch;
+
+interface BatchStrategyInterface
+{
+    /**
+     * Processes the data set by setData method.
+     *
+     * This method is the core of the strategy, where the actual processing of the data happens.
+     * It should contain the logic specific to the strategy's purpose, such as sending emails,
+     * importing records, or any other task that the strategy is designed to perform.
+     *
+     * @return mixed
+     */
+    public function process();
+}

Api/Model/Sync/Batch/Record/RecordImportedStrategyInterface.php

@@ -0,0 +1,25 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Batch\Record;
+
+use Dotdigitalgroup\Email\Api\Model\Sync\Batch\BatchStrategyInterface;
+
+interface RecordImportedStrategyInterface extends BatchStrategyInterface
+{
+    /**
+     * Sets the data to be processed by the strategy.
+     *
+     * @param array $records
+     * @return RecordImportedStrategyInterface
+     */
+    public function setRecords(array $records): RecordImportedStrategyInterface;
+
+    /**
+     * Processes a batch of records.
+     *
+     * @return void
+     */
+    public function process(): void;
+}

Api/Model/Sync/Batch/Sender/SenderStrategyInterface.php

@@ -0,0 +1,33 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Batch\Sender;
+
+use Dotdigitalgroup\Email\Api\Model\Sync\Batch\BatchStrategyInterface;
+
+interface SenderStrategyInterface extends BatchStrategyInterface
+{
+    /**
+     * Sets the batch of data to be processed.
+     *
+     * @param array $batch
+     * @return SenderStrategyInterface
+     */
+    public function setBatch(array $batch): SenderStrategyInterface;
+
+    /**
+     * Sets the website ID associated with the data batch.
+     *
+     * @param int $websiteId
+     * @return SenderStrategyInterface
+     */
+    public function setWebsiteId(int $websiteId): SenderStrategyInterface;
+
+    /**
+     * Processes a batch of records.
+     *
+     * @return string An import ID, or an empty string.
+     */
+    public function process(): string;
+}

Api/Model/Sync/Export/ContactExporterInterface.php

@@ -0,0 +1,45 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Export;
+
+use Exception;
+use Magento\Framework\Exception\LocalizedException;
+use Magento\Store\Api\Data\WebsiteInterface;
+
+/**
+ * Interface ContactExporterInterface
+ *
+ * This interface extends the ExporterInterface and provides methods to export data and manage field mappings.
+ */
+interface ContactExporterInterface extends ExporterInterface
+{
+    /**
+     * Export data.
+     *
+     * @param array $data
+     * @param WebsiteInterface $website
+     * @param int $listId
+     *
+     * @return array
+     * @throws LocalizedException|Exception
+     */
+    public function export(array $data, WebsiteInterface $website, int $listId);
+
+    /**
+     * Set field mapping.
+     *
+     * @param WebsiteInterface $website
+     *
+     * @return void
+     */
+    public function setFieldMapping(WebsiteInterface $website): void;
+
+    /**
+     * Get field mapping.
+     *
+     * @return array
+     */
+    public function getFieldMapping(): array;
+}

Api/Model/Sync/Export/ExporterInterface.php

@@ -0,0 +1,10 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Export;
+
+interface ExporterInterface
+{
+
+}

Api/Model/Sync/Export/InsightDataExporterInterface.php

@@ -0,0 +1,24 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Export;
+
+use Dotdigital\V3\Models\InsightData\Record;
+use Magento\Store\Api\Data\WebsiteInterface;
+
+/**
+ * Interface InsightDataExporterInterface
+ *
+ * This interface extends the ExporterInterface and provides a method to build a collection of insight data records.
+ */
+interface InsightDataExporterInterface extends ExporterInterface
+{
+    /**
+     * Build a collection of insight data records.
+     *
+     * @param array $data
+     * @return array<string, Record>
+     */
+    public function export(array $data): array;
+}

Api/Model/Sync/Export/SdkBuilderInterface.php

@@ -0,0 +1,28 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Export;
+
+/**
+ * Interface SyncBuilderInterface
+ *
+ * This interface defines the methods required for building and exporting sync data.
+ */
+interface SdkBuilderInterface
+{
+    /**
+     * Set the data to be built.
+     *
+     * @param mixed $data The data to be built.
+     * @return SdkBuilderInterface Returns the instance of the builder.
+     */
+    public function setBuildableData($data): SdkBuilderInterface;
+
+    /**
+     * Build the data.
+     *
+     * @return mixed The built data.
+     */
+    public function build();
+}

Api/Model/Sync/Export/SdkCollectionBuilderInterface.php

@@ -0,0 +1,17 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Export;
+
+use Dotdigital\V3\Models\InsightData\RecordsCollection;
+
+interface SdkCollectionBuilderInterface extends SdkBuilderInterface
+{
+    /**
+     * Build a RecordsCollection.
+     *
+     * @return RecordsCollection
+     */
+    public function build(): RecordsCollection;
+}

Api/Model/Sync/Importer/BulkSyncInterface.php

@@ -0,0 +1,18 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Dotdigitalgroup\Email\Api\Model\Sync\Importer;
+
+use Dotdigitalgroup\Email\Model\Importer as ImporterModel;
+
+interface BulkSyncInterface
+{
+    /**
+     * Process a single importer item.
+     *
+     * @param ImporterModel $item
+     * @return mixed
+     */
+    public function process(ImporterModel $item);
+}

Model/Importer.php

@@ -26,6 +26,7 @@ class Importer extends AbstractModel
 
     //import mode
     public const MODE_BULK = 'Bulk';
+    public const MODE_BULK_JSON = 'Bulk_JSON';
     public const MODE_SINGLE = 'Single';
     public const MODE_SINGLE_DELETE = 'Single_Delete';
     public const MODE_CONTACT_DELETE = 'Contact_Delete';

Model/ResourceModel/Importer/Collection.php

@@ -2,6 +2,9 @@
 
 namespace Dotdigitalgroup\Email\Model\ResourceModel\Importer;
 
+use Dotdigitalgroup\Email\Model\Importer as ImporterModel;
+use Dotdigitalgroup\Email\Model\ResourceModel\Importer;
+
 class Collection extends \Magento\Framework\Model\ResourceModel\Db\Collection\AbstractCollection
 {
     /**
@@ -38,15 +41,20 @@ public function reset()
      * @param array $websiteIds
      * @param array $types
      * @param bool $useFile
-     *
+     * @param string $mode
      * @return $this|boolean
      */
-    public function getItemsWithImportingStatus($websiteIds, array $types, bool $useFile = false)
-    {
+    public function getItemsWithImportingStatus(
+        $websiteIds,
+        array $types,
+        bool $useFile = false,
+        string $mode = ImporterModel::MODE_BULK
+    ) {
         $collection = $this->addFieldToFilter(
             'import_status',
             ['eq' => \Dotdigitalgroup\Email\Model\Importer::IMPORTING]
         )
+            ->addFieldToFilter('import_mode', ['eq'=> $mode])
             ->addFieldToFilter('import_id', ['neq' => ''])
             ->addFieldToFilter(
                 'website_id',