Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add data warehouse batch export #1010

Merged
merged 257 commits into from
Aug 22, 2019
Merged
Changes from 1 commit
Commits
Show all changes
257 commits
Select commit Hold shift + click to select a range
c464faa
Add batch export requests table
jtpalmer Feb 27, 2019
403c9fa
Merge remote-tracking branch 'upstream/xdmod8.1' into data-warehouse-…
jtpalmer Feb 27, 2019
a5e10fb
Merge remote-tracking branch 'upstream/xdmod8.1' into data-warehouse-…
jtpalmer Mar 20, 2019
c37510c
Merge remote-tracking branch 'upstream/xdmod8.1' into data-warehouse-…
jtpalmer Apr 15, 2019
52a9743
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Apr 29, 2019
b339146
Stubs for Export batch script classes.
Apr 12, 2019
ed40bdd
Current early state of QueryHandler class for batch export script.
jsperhac Apr 19, 2019
00ccd7e
Current status of class to handle querying for export.
jsperhac Apr 29, 2019
2b6617e
Being modern, we are now binding variables...
jsperhac Apr 29, 2019
ad244d1
Strings like realm name need to be quoted...
jsperhac Apr 29, 2019
7dbd26f
Pass your array of params, please.
jsperhac Apr 29, 2019
c6ac461
Public constructor.
jsperhac Apr 30, 2019
936fe6e
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 1, 2019
0124c16
Add expiration columns
jtpalmer May 1, 2019
b204ac3
Added component tests for the DB Export query handler class. Rebased to
jsperhac Apr 30, 2019
f776e87
Modified component tests for DB Export query handler to accommodate
jsperhac May 2, 2019
0cf39b2
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 13, 2019
dfbe59b
Add migration and test
jtpalmer May 14, 2019
d82ef26
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 15, 2019
b82071f
thing
chakrabortyr Apr 2, 2019
f91a33a
more stuff
chakrabortyr Apr 2, 2019
afcba76
things
chakrabortyr Apr 2, 2019
f2a00ae
thing
chakrabortyr Apr 3, 2019
e73b552
the stuffs
chakrabortyr Apr 5, 2019
b79b632
things
chakrabortyr Apr 5, 2019
839922d
Thing
chakrabortyr Apr 5, 2019
a142e5a
thing
chakrabortyr Apr 10, 2019
975be8a
thing
chakrabortyr Apr 10, 2019
39c4f5f
thing
chakrabortyr Apr 11, 2019
70d36e3
thing
chakrabortyr Apr 11, 2019
624f480
Replace everything
jtpalmer May 16, 2019
f01602c
Replace tabs with spaces
jtpalmer May 16, 2019
84dfd4e
Change style and add fieldset
jtpalmer May 16, 2019
41424b1
Change data store, add empty text and more columns
jtpalmer May 16, 2019
c381032
Add controller provider
jtpalmer May 16, 2019
52f96d4
Add grid title
jtpalmer May 16, 2019
ddcdc93
Change text and add help tips
jtpalmer May 17, 2019
aa75f83
Add REST endpoint for batch export realms
jtpalmer May 17, 2019
4c67ff2
Fix indentation and spacing
jtpalmer May 17, 2019
08e817a
Add paging toolbar
jtpalmer May 17, 2019
2bf991f
Move data store out of component
jtpalmer May 17, 2019
4d15ed5
Refactor to separate components
jtpalmer May 17, 2019
bba9322
Change margins, etc.
jtpalmer May 17, 2019
54f024c
Fix style issues and add more example data
jtpalmer May 17, 2019
ade14cf
Remove trailing comma
jtpalmer May 17, 2019
f7e7275
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 20, 2019
26d2ca2
Merge branch 'data-warehouse-export-rest' into data-warehouse-export-dev
jtpalmer May 20, 2019
d0f24b7
Add REST configuration and fix controller
jtpalmer May 20, 2019
9ea097e
Use remote realm list, update various options
jtpalmer May 20, 2019
cc15f78
Fix typo
jtpalmer May 20, 2019
e4555f0
Add ID field and types to data store
jtpalmer May 21, 2019
9da63e8
Rearrange columns and add button to bottom bar
jtpalmer May 24, 2019
1262706
Adjust combo options
jtpalmer May 24, 2019
7f26e49
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 24, 2019
5e2426f
Update get_tabs test output
jtpalmer May 24, 2019
496a97c
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 24, 2019
cbd1a06
Fix actions
jtpalmer May 24, 2019
29401cd
Fix test output
jtpalmer May 24, 2019
1cbe6c8
Fix typos
jtpalmer May 24, 2019
7d3a1d4
Fix return values
jtpalmer May 24, 2019
9d07d19
Add JSON store and update fields
jtpalmer May 24, 2019
53a5165
Fix comparison
jtpalmer May 24, 2019
97f9b80
Removed call to pdo quote function on insert of new record.
jsperhac May 24, 2019
5318215
Refactor data store
jtpalmer May 28, 2019
c5e3f25
Getting tests running cleanly for arbitrary user (must exist in Users!)
jsperhac May 28, 2019
534becb
Add request date and format to query output from export requests table.
jsperhac May 28, 2019
981e349
Restructure panels and add validation
jtpalmer May 28, 2019
0df25f5
Current working state, added state requirements to transitions, and
jsperhac May 28, 2019
9f343e1
Fix style issues and improve error reporting
jtpalmer May 29, 2019
244b079
Add missing variable declarations and fix logic
jtpalmer May 29, 2019
985842d
Added state requirements to all transitions in DB access class.
jsperhac May 29, 2019
ef6d9d7
Merge remote-tracking branch 'jsperhac/batch-export-classes' into dat…
jtpalmer May 30, 2019
4515059
Style fixes for Export database class tests.
jsperhac May 30, 2019
1493108
Updated name of data_warehouse_export config field indicating
jsperhac May 30, 2019
bbacf76
Merge remote-tracking branch 'jsperhac/batch-export-classes' into dat…
jtpalmer May 30, 2019
2e2bf1d
Added default path to data warehouse export file storage.
jsperhac May 30, 2019
ae96ece
Remove unnecessary mask functions
jtpalmer May 30, 2019
d320e8c
Improve field and data store configurations
jtpalmer May 30, 2019
39bb62e
Add some validation and update controller
jtpalmer May 30, 2019
b0cff72
Merge remote-tracking branch 'jsperhac/batch-export-classes' into dat…
jtpalmer May 31, 2019
f155380
Fix style issue
jtpalmer May 31, 2019
6aaf03e
Update template and comments
jtpalmer May 31, 2019
d23970a
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 31, 2019
b0a62eb
Merge pull request #1 from jtpalmer/batch-export-classes
jsperhac May 31, 2019
b5d040b
Ensure TRUE and FALSE are 1 and 0 in SQL for tests
jsperhac May 31, 2019
9068d38
Merge branch 'batch-export-classes' of github.com:jsperhac/xdmod into…
jsperhac May 31, 2019
60c0d45
doing some stuff to my tests...
jsperhac May 31, 2019
fa08b80
Use request state data from internal API
jtpalmer May 31, 2019
1b10024
Change state column colors and action buttons
jtpalmer May 31, 2019
8ab56f1
Refactor error handler and "Delete all ..." button
jtpalmer May 31, 2019
61f2b9d
Add styles
jtpalmer May 31, 2019
0820727
Remove unused arguments
jtpalmer May 31, 2019
20931db
Merge remote-tracking branch 'jsperhac/batch-export-classes' into dat…
jtpalmer May 31, 2019
4ac325e
Add missing break statements
jtpalmer May 31, 2019
e16d780
Fix style
jtpalmer May 31, 2019
76e90e8
Remove console debugging
jtpalmer May 31, 2019
f2b0e9f
Refactor realm store usage
jtpalmer May 31, 2019
d0a9429
Add eslint exceptions
jtpalmer May 31, 2019
67343a0
Implement more REST endpoints
jtpalmer May 31, 2019
751b94b
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer May 31, 2019
69a3860
Rearrange and simplify tests for data warehouse export query class
jsperhac May 31, 2019
490bcc2
Fix file download REST
jtpalmer Jun 2, 2019
18b1347
Fix and rearrange route setup
jtpalmer May 31, 2019
28357c8
Implement delete buttons
jtpalmer May 31, 2019
1ac43ab
Fix icons and change green color
jtpalmer Jun 3, 2019
49e4215
Change variable and property names
jtpalmer Jun 3, 2019
e3a1fa4
Fix bug
jtpalmer Jun 3, 2019
4cde143
Improve exceptions
jtpalmer Jun 3, 2019
a518de7
Enable/disable "Delete all ..." button
jtpalmer Jun 3, 2019
b3d0fb9
Change field name style
jtpalmer Jun 3, 2019
66487cf
Implement request resubmission
jtpalmer Jun 3, 2019
ad38a63
Merge remote-tracking branch 'jsperhac/batch-export-classes' into dat…
jtpalmer Jun 3, 2019
39fa80c
Add more start/end date validation
jtpalmer Jun 3, 2019
2f8e495
Add mask when no requests exist
jtpalmer Jun 3, 2019
3a3d327
Fix data index
jtpalmer Jun 3, 2019
39d85f2
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jun 10, 2019
b735c66
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jun 10, 2019
84501c3
Merge branch 'data-warehouse-export-dev' into data-warehouse-export
jtpalmer Jun 12, 2019
f8cc984
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jun 12, 2019
0ab3c68
Fix style issues
jtpalmer Jun 12, 2019
6f6f1b2
First attempt at integrating ACLs
jtpalmer Jun 14, 2019
5f11e2e
Add more data to REST responses
jtpalmer Jun 14, 2019
82bbe6a
Add REST tests
jtpalmer Jun 14, 2019
ba78709
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jun 15, 2019
f8c9482
Change ordering
jtpalmer Jun 21, 2019
1f083c8
Implement more tests
jtpalmer Jun 21, 2019
7091787
More assertions
jtpalmer Jun 21, 2019
c939d2b
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jun 21, 2019
3cb9c8a
Rearrange and comment-out code
jtpalmer Jun 24, 2019
3ac94e5
Add function
jtpalmer Jun 28, 2019
3ce898d
Update comments and formatting
jtpalmer Jun 28, 2019
f47c6bc
Rename and improve function
jtpalmer Jun 28, 2019
8d34c89
Add (mostly empty) email templates
jtpalmer Jun 28, 2019
f5c2799
Add class for realm related export functions
jtpalmer Jun 28, 2019
c74cda1
Refactor, etc.
jtpalmer Jun 28, 2019
d1de151
Add batch script, etc.
jtpalmer Jun 28, 2019
d4aef00
Fix typo and add missing code
jtpalmer Jun 28, 2019
6fb3441
Mostly documentation changes
jtpalmer Jun 30, 2019
f4ff0e7
Fix namespace
jtpalmer Jul 1, 2019
a12ed84
Fix typo
jtpalmer Jul 1, 2019
4b4b9cb
Implement function
jtpalmer Jul 2, 2019
2e65848
Add function
jtpalmer Jul 2, 2019
da101b1
Add tests
jtpalmer Jul 3, 2019
412be8d
More batch export changes
jtpalmer Jul 3, 2019
a6108e9
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jul 3, 2019
3f78735
Fix syntax, etc.
jtpalmer Jul 3, 2019
7b4e3d3
Replace tabs with spaces and improve help text
jtpalmer Jul 7, 2019
2ab6b16
More implementation work and tests
jtpalmer Jul 7, 2019
e8d5e71
Use unused variable
jtpalmer Jul 7, 2019
374f979
Fix namespaces and base class
jtpalmer Jul 7, 2019
d83d63d
Fix use of xd_utilities
jtpalmer Jul 7, 2019
44dca68
Call parent constructor
jtpalmer Jul 8, 2019
8a8819b
Change logging
jtpalmer Jul 8, 2019
28fd650
Change function name
jtpalmer Jul 8, 2019
d954882
Change failure reporting emails
jtpalmer Jul 8, 2019
cff8563
Fix bug and make some functions static
jtpalmer Jul 8, 2019
5d9f349
Remove unnecessary namespace
jtpalmer Jul 8, 2019
1cfa4f6
Fix typo
jtpalmer Jul 8, 2019
1f46815
Implement test helper functions
jtpalmer Jul 8, 2019
c560c47
Add batch export directory
jtpalmer Jul 9, 2019
bd8622e
Add data warehouse config migration
jtpalmer Jul 9, 2019
2ab79c9
Add data warehouse export file writing classes
jtpalmer Jul 10, 2019
b348279
Add expiration date to email
jtpalmer Jul 10, 2019
827ca59
Refactor archive error handling
jtpalmer Jul 10, 2019
92b5b9a
More file writing
jtpalmer Jul 10, 2019
d969c40
Use file writers
jtpalmer Jul 10, 2019
b3dfc6c
Add newline to end of help text
jtpalmer Jul 10, 2019
8a81fdd
Add missing function call
jtpalmer Jul 10, 2019
188e058
Order "use" statements
jtpalmer Jul 10, 2019
68a0f09
Add missing argument
jtpalmer Jul 10, 2019
33ad92c
Add missing database update for failures
jtpalmer Jul 10, 2019
9d4fff9
Add missing class property
jtpalmer Jul 10, 2019
05309ce
Fix typos
jtpalmer Jul 10, 2019
d396730
Add/change exception logging
jtpalmer Jul 10, 2019
307c5c6
Change fields selected in some queries
jtpalmer Jul 10, 2019
b4674c8
Remove old code
jtpalmer Jul 10, 2019
b89e0f0
Fix record format
jtpalmer Jul 10, 2019
f79783c
Refactor loggers
jtpalmer Jul 10, 2019
fcf8dbe
Don't update database records during dry run
jtpalmer Jul 10, 2019
eb6631c
Fix typo
jtpalmer Jul 10, 2019
2dc1869
Make format check case insensitive
jtpalmer Jul 10, 2019
14da63a
Change log message
jtpalmer Jul 10, 2019
76bf8bd
Fix querying
jtpalmer Jul 10, 2019
9a05cb1
Change file naming
jtpalmer Jul 10, 2019
5bce86a
Fix typo
jtpalmer Jul 10, 2019
3ccb235
Add default constructor argument
jtpalmer Jul 10, 2019
662b980
Update expected keys
jtpalmer Jul 10, 2019
80764e7
Update assertions, move comments into messages
jtpalmer Jul 10, 2019
00fed1e
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jul 11, 2019
7524388
Refactor export file related functions
jtpalmer Jul 11, 2019
1e53430
Remove unused variable
jtpalmer Jul 11, 2019
f646b93
Remove role parameters from query
jtpalmer Jul 11, 2019
318a2fe
Fix typo
jtpalmer Jul 12, 2019
8616a82
Change download URL and implement test
jtpalmer Jul 12, 2019
b35343c
Add email details
jtpalmer Jul 12, 2019
792146b
Fix test
jtpalmer Jul 12, 2019
6f48211
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jul 15, 2019
988d616
Ensure realms are returned in numeric array
jtpalmer Jul 15, 2019
3fe3f2a
Include messages for previous stack traces
jtpalmer Jul 15, 2019
dfdc4f5
Disable memory and time limits
jtpalmer Jul 15, 2019
320bcc2
Add debug logging statement
jtpalmer Jul 16, 2019
3d10ce7
WIP
jtpalmer Jul 19, 2019
eca46fe
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jul 22, 2019
63f9664
Merge branch 'data-warehouse-export-wip' into data-warehouse-export
jtpalmer Jul 22, 2019
1ea58a3
Add missing semicolon
jtpalmer Jul 23, 2019
b95ecdd
Comment-out paging until implemented
jtpalmer Jul 23, 2019
5b93ceb
Refactor batch data set
jtpalmer Jul 24, 2019
4d4f755
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Jul 24, 2019
dc0f7c0
Add message for successful request submission
jtpalmer Jul 26, 2019
a5b1098
Change batch export option
jtpalmer Jul 26, 2019
a3b2a5d
Replace switch statement with if statements
jtpalmer Jul 26, 2019
f87e41b
Update batch export configuration
jtpalmer Jul 26, 2019
4ca708d
Update ChartPortlet for UI consistency.
jpwhite4 Jul 26, 2019
4002479
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Aug 5, 2019
2c508bb
Change data export download URL
jtpalmer Aug 6, 2019
c682a57
Add hash salt
jtpalmer Aug 6, 2019
82788e7
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Aug 7, 2019
3dfc000
Change debugging output
jtpalmer Aug 7, 2019
a491691
Quote field alias
jtpalmer Aug 9, 2019
26a32bd
Refactor configuration file format
jtpalmer Aug 12, 2019
a52bca2
Make function protected
jtpalmer Aug 13, 2019
2efd387
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Aug 13, 2019
251a9a1
Fix style
jtpalmer Aug 13, 2019
01b08c9
Fix logging
jtpalmer Aug 15, 2019
c395aa4
Change realm name field
jtpalmer Aug 15, 2019
0b7fcb0
Add batch export option to ETL profile
jtpalmer Aug 16, 2019
0acc805
Add data warehouse export configuration
jtpalmer Aug 18, 2019
61ecd66
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Aug 18, 2019
3cd72e7
Replace tabs with spaces
jtpalmer Aug 18, 2019
a6ec73a
Update role configuration
jtpalmer Aug 20, 2019
e642739
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Aug 20, 2019
8353e03
Add error checking
jtpalmer Aug 21, 2019
c5ad365
Fix comment grammar and add more error checking
jtpalmer Aug 21, 2019
5c206e3
Remove commented-out code
jtpalmer Aug 21, 2019
9247ec2
Change raw query stat name
jtpalmer Aug 21, 2019
4634939
Change comments
jtpalmer Aug 21, 2019
701c3ad
Add debugging
jtpalmer Aug 21, 2019
1b3281b
Use explicit time zone and start time
jtpalmer Aug 21, 2019
248d9dd
Merge remote-tracking branch 'upstream/xdmod8.5' into data-warehouse-…
jtpalmer Aug 21, 2019
90b6597
Improve comment
jtpalmer Aug 21, 2019
2faf124
More debugging
jtpalmer Aug 21, 2019
54160d6
Add cron job
jtpalmer Aug 22, 2019
ecc3f9a
Update test artifacts
jtpalmer Aug 22, 2019
94f6cb1
Update test artifacts
jtpalmer Aug 22, 2019
6b888f6
Remove incorrect namespace
jtpalmer Aug 22, 2019
0e3f616
Add default configuration values
jtpalmer Aug 22, 2019
78802ec
Update setup and test scripts
jtpalmer Aug 22, 2019
f27288d
Update test scripts
jtpalmer Aug 22, 2019
5478352
Update test script
jtpalmer Aug 22, 2019
4324f92
Update test artifacts
jtpalmer Aug 22, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
More implementation work and tests
jtpalmer committed Jul 7, 2019
commit 2ab6b16da7ae60c7fc5c787068428aca30572209
82 changes: 68 additions & 14 deletions classes/DataWarehouse/Export/BatchProcessor.php
Original file line number Diff line number Diff line change
@@ -34,13 +34,23 @@ class BatchProcessor extends Loggable
*/
private $queryHandler;

/**
* Path of directory containing export zip files.
* @var string
*/
private $exportDirectory;

/**
* Construct a new batch processor.
*/
public function __construct()
{
$this->dbh = DB::factory('database');
$this->queryHandler = new QueryHandler();
$this->exportDirectory = xd_utilities::getConfiguration(
'data_warehouse_export',
'export_directory'
);
}

/**
@@ -105,12 +115,9 @@ private function processSubmittedRequest(array $request)
try {
$this->dbh->beginTransaction();
$this->queryHandler->submittedToAvailable($request['id']);

$dataSet = $this->getDataSet($request);

$dataFile = $this->writeDataToFile($dataSet, $request['format']);

$zipFile = 'TODO';
$zipFile = $this->getExportZipFilePath($request['id']);
$this->createZipFile($dataFile, $zipFile);

// Query for same record to get expiration date.
@@ -161,9 +168,8 @@ private function processExpiringRequest(array $request)

try {
$this->dbh->beginTransaction();
// TODO
// Delete file
$this->queryHandler->availableToExpired($request['id']);
$this->removeExportFile($request['id']);
$this->dbh->commit();
} catch (Exception $e) {
$this->dbh->rollback();
@@ -190,6 +196,7 @@ private function getDataSet(array $request, XDUser $user)
'user_email' => $user->getEmailAddress(),
'user_first_name' => $user->getFirstName(),
'user_last_name' => $user->getLastName(),
'batch_export_request.id' => $request['id'],
'realm' => $request['realm'],
'start_date' => $request['start_date'],
'end_date' => $request['end_date']
@@ -223,16 +230,36 @@ private function getDataSet(array $request, XDUser $user)
}

/**
* Write data set to file.
*
* @param \DataWarehouse\Data\RawDataset $dataSet
* @param string $format
*/
private function generateExportFile()
private function writeDataToFile(RawDataset $dataSet, $format)
{
if ($this->dryRun) {
$this->logger->notice('dry run: Not generating export file');
$this->logger->notice('dry run: Not writing data to file');
return;
}

$this->logger->info('Generating export file');
// TODO
$tmpFile = tempnam(sys_get_temp_dir(), 'batch-export-');

$this->logger->info([
'message' => 'Writing data to file',
'tmp_file' => $tmpFile
]);

// The `export` function returns the first result along with the
// necessary metadata.
foreach ($dataSet->export() as $datum) {
// TODO
}

foreach ($dataSet->getResults() as $result) {
// TODO
}

return $tmpFile;
}

/**
@@ -249,7 +276,11 @@ private function createZipFile($dataFile, $zipFile)
return;
}

$this->logger->info('Creating zip file');
$this->logger->info([
'message' => 'Creating zip file',
'data_file' => $dataFile,
'zip_file' => $zipFile
]);

$zip = new ZipArchive();
$zipOpenCode = $zip->open($zipFile, ZipArchive::CREATE);
@@ -278,16 +309,39 @@ private function createZipFile($dataFile, $zipFile)
}

/**
* Remove an export data file.
*
* @param int $id Export request primary key.
*/
private function removeExportFile()
private function removeExportFile($id)
{
if ($this->dryRun) {
$this->logger->notice('dry run: Not removing export file');
return;
}

$this->logger->info('Removing export file');
// TODO
$zipFile = $this->getExportZipFilePath($id);

$this->logger->info([
'message' => 'Removing export file',
'batch_export_request.id' => $id,
'zip_file' => $zipFile
]);

if (!unlink($zipFile)) {
throw new Exception(sprintf('Failed to delete "%s"', $zipFile));
}
}

/**
* Get the full path for the export data file.
*
* @param int $id Export request primary key.
* @return string
*/
private function getExportZipFilePath($id)
{
return $this->exportDirectory . DIRECTORY_SEPARATOR . $id . '.zip';
}

/**
105 changes: 105 additions & 0 deletions tests/component/lib/Export/BatchProcessTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
<?php

namespace ComponentTests\Roles;

use DataWarehouse\Export\BatchProcessor;
use DataWarehouse\Export\QueryHandler;
use xd_utilities;

/**
* Test batch processing of export requests.
*
* @coversDefaultClass \DataWarehouse\Export\BatchProcessor
*/
class BatchProcessTest extends BaseTest
{
/**
* Test artifacts path.
*/
const TEST_GROUP = 'component/export/batch_process';

/**
* @var \DataWarehouse\Export\QueryHandler
*/
private static $queryHandler;

/**
* The file containing email data.
* @var string
*/
private static $mailSpool = '/var/mail/root';

/**
* The directory containing batch export zip files.
* @var string.
*/
private static $exportDirectory;

/**
*/
public static function setUpBeforeClass()
{
parent::setUpBeforeClass();
self::$queryHandler = new QueryHandler();
self::$exportDirectory = xd_utilities::getConfiguration(
'data_warehouse_export',
'export_directory'
);
}

/**
*/
private function getEmails()
{
// TODO
return [];
}

/**
*/
private function getExportFiles()
{
// TODO
return [];
}

/**
* Test the batch processor dry run option.
*/
public function testDryRun()
{
$batchProcessor = new BatchProcessor();
$batchProcessor->setDryRun(true);

$emails = $this->getEmails();
$files = $this->getExportFiles();
$submittedRequests = $this->queryHandler->listSubmittedRecords();
$expiringRequests = $this->queryHandler->listExpiringRecords();

$batchProcessor->processRequests();

$this->assertEquals($emails, $this->getEmails(), 'No new emails');
$this->assertEquals($files, $this->getFiles(), 'No new export files');
$this->assertEquals(
$submittedRequests,
$this->queryHandler->listSubmittedRecords(),
'No submitted requests changed'
);
$this->assertEquals(
$expiringRequests,
$this->queryHandler->listExpiringRecords(),
'No expiring requests changed'
);

$this->markTestIncomplete('This test has not been implemented yet.');
}

/**
* Test processing batch export requests.
*/
public function testRequestProcessing()
{
$batchProcessor = new BatchProcessor();
$this->markTestIncomplete('This test has not been implemented yet.');
}
}