Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import Licence versions #1195

Merged
merged 96 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
85c92d0
Import Licence versions
jonathangoulding Jul 18, 2024
502daf9
chore: remove unused comment
jonathangoulding Jul 18, 2024
06b2e2b
feat: get purpose from db and map to rough format
jonathangoulding Jul 22, 2024
a714ad7
feat: add licence version purposes
jonathangoulding Jul 22, 2024
a228676
feat: add licence version purposes
jonathangoulding Jul 22, 2024
731b99a
chore: update js docs
jonathangoulding Jul 22, 2024
afe6f11
feat: add licence version and purpose validator
jonathangoulding Jul 22, 2024
3dc605b
feat: add licence version and purpose validator
jonathangoulding Jul 22, 2024
d829fbf
test: licence versions mapper
jonathangoulding Jul 23, 2024
1b822a0
test: licence versions mapper
jonathangoulding Jul 23, 2024
54eafa6
test: legacy service
jonathangoulding Jul 23, 2024
15f0c73
Merge branch 'main' into import-licence-versions
jonathangoulding Jul 25, 2024
c931526
test: legacy import service versions and purposes
jonathangoulding Jul 25, 2024
af6bd12
test: legacy import service versions and purposes
jonathangoulding Jul 25, 2024
9eb8d6e
chore: fix sql linting
jonathangoulding Jul 26, 2024
1e8fffa
chore: fix sql linting
jonathangoulding Jul 26, 2024
df13ff7
chore: fix sql linting
jonathangoulding Jul 26, 2024
2970e71
chore: fix sql linting
jonathangoulding Jul 26, 2024
1c154d3
chore: fix sql linting
jonathangoulding Jul 26, 2024
11f25e1
chore: fix sonar cloud issues
jonathangoulding Jul 26, 2024
32667da
test: licence versions validator
jonathangoulding Jul 26, 2024
7f6c958
Merge branch 'main' into import-licence-versions
jonathangoulding Jul 26, 2024
23974f0
test: licence versions validator
jonathangoulding Jul 26, 2024
8bb3824
feat: add licence ref to error log
jonathangoulding Jul 26, 2024
e869e72
Update app/services/import/legacy-import/fetch-licence-versions.servi…
jonathangoulding Jul 26, 2024
7f8c397
Update app/services/import/legacy-import/fetch-licence-versions.servi…
jonathangoulding Jul 26, 2024
a9cfc9f
Update app/services/import/legacy-import/fetch-licence-versions.servi…
jonathangoulding Jul 26, 2024
7f50242
Update app/services/import/legacy-import/fetch-licence-versions.servi…
jonathangoulding Jul 26, 2024
4bc5899
Update app/services/import/legacy-import/fetch-licence-versions.servi…
jonathangoulding Jul 26, 2024
b51422d
Update app/validators/import/licence-versions.validator.js
jonathangoulding Jul 26, 2024
03530c2
Update test/services/import/legacy-licence.service.test.js
jonathangoulding Jul 26, 2024
a615973
Update app/services/import/legacy-import/licence-versions.mapper.js
jonathangoulding Jul 26, 2024
f201df8
Update app/services/import/legacy-import/licence-versions.mapper.js
jonathangoulding Jul 26, 2024
83857e7
Update app/services/import/legacy-import/licence-versions.mapper.js
jonathangoulding Jul 26, 2024
619ceac
Update app/services/import/legacy-import/licence-versions.mapper.js
jonathangoulding Jul 26, 2024
cb49d30
Update app/services/import/legacy-import/licence-versions.mapper.js
jonathangoulding Jul 26, 2024
e04100f
Update app/services/import/persist-licence-versions.service.js
jonathangoulding Jul 26, 2024
275095c
fix: licence date validation
jonathangoulding Jul 26, 2024
6f2a7ef
Merge branch 'import-licence-versions' of https://github.com/DEFRA/wa…
jonathangoulding Jul 26, 2024
5ba1387
fix: licence date validation
jonathangoulding Jul 26, 2024
73da354
fix: licence date validation
jonathangoulding Jul 26, 2024
941db0f
Merge branch 'main' into import-licence-versions
jonathangoulding Jul 29, 2024
01d97a6
test: add licence versions persist tests
jonathangoulding Jul 29, 2024
a310bf8
test: licence versions purposes
jonathangoulding Jul 29, 2024
d71e2a4
test: licence versions purposes
jonathangoulding Jul 29, 2024
fdfedc8
test: licence versions purposes
jonathangoulding Jul 29, 2024
86f8a63
chore: pre pr checks
jonathangoulding Jul 30, 2024
1634fad
Merge branch 'main' into import-licence-versions
jonathangoulding Jul 30, 2024
d75f1dd
feat: regions are required
jonathangoulding Jul 30, 2024
dca16ca
feat: licence version and purposes are required fields
jonathangoulding Jul 30, 2024
e7157a8
feat: licence version and purposes are required fields
jonathangoulding Jul 30, 2024
0849920
chore: add regions key to legacy fetch licence
jonathangoulding Jul 30, 2024
582cd3c
refactor: fixtures to use func to generate random data
jonathangoulding Jul 30, 2024
285a830
refactor: add nald regions to static lib
jonathangoulding Jul 30, 2024
a950db4
refactor: date lib to throw an error
jonathangoulding Jul 30, 2024
cc73896
Merge branch 'main' into import-licence-versions
jonathangoulding Jul 31, 2024
dbfe0d7
refactor: import controller test
jonathangoulding Jul 31, 2024
5391102
Merge branch 'main' into import-licence-versions
jonathangoulding Jul 31, 2024
928c7e8
Merge branch 'main' into import-licence-versions
jonathangoulding Jul 31, 2024
3f142dd
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 2, 2024
b003330
Merge branch 'main' into import-licence-versions
robertparkinson Aug 2, 2024
2a74947
Merge remote-tracking branch 'origin/main' into import-licence-versions
jonathangoulding Aug 8, 2024
e5f9033
chore: merge main pr and fix
jonathangoulding Aug 8, 2024
08cdc00
fix: remove expectation of includeInSrocTptBilling in licence default…
jonathangoulding Aug 8, 2024
a17ea4d
fix: sonar consistent return
jonathangoulding Aug 8, 2024
6e900c8
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 8, 2024
17d419d
fix: data binding
jonathangoulding Aug 9, 2024
5dd0e74
fix: controller logic - follow convention
jonathangoulding Aug 9, 2024
28c6347
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 9, 2024
e93fc73
fix: error handling for import controller test
jonathangoulding Aug 9, 2024
c9d8f44
chore: update js docs
jonathangoulding Aug 9, 2024
209a01f
fix: persist licence version does not need to return anything
jonathangoulding Aug 9, 2024
29d1d85
fix: persist licence version does not need to return anything
jonathangoulding Aug 9, 2024
60930dc
chore: fix lint
jonathangoulding Aug 9, 2024
2d5a6d7
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 9, 2024
77f8e76
fix: persist licence test - persist should return the licence
jonathangoulding Aug 9, 2024
67a9ab9
chore: lint / fix js docs
jonathangoulding Aug 9, 2024
3103cab
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 12, 2024
d0e6276
refactor: add specific date sort for licence mapper start date
jonathangoulding Aug 12, 2024
ba900be
refactor: pair programming refactor
jonathangoulding Aug 13, 2024
acc129b
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 13, 2024
752c950
refactor: use data object in persist licence
jonathangoulding Aug 14, 2024
6b836cc
refactor: move licence versions created and update at into presenter
jonathangoulding Aug 14, 2024
6b76ade
refactor: move licence versions purposes created and update at into p…
jonathangoulding Aug 14, 2024
a404c84
fix: sonar assert thrown error
jonathangoulding Aug 14, 2024
dffb80f
Revert "fix: sonar assert thrown error"
jonathangoulding Aug 14, 2024
3f89561
Revert "refactor: move licence versions purposes created and update a…
jonathangoulding Aug 14, 2024
a6993bd
Revert "refactor: move licence versions created and update at into pr…
jonathangoulding Aug 14, 2024
28bc42a
refactor: add migration for updated and created at to be handled by db
jonathangoulding Aug 14, 2024
6e7e83e
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 19, 2024
52eb04e
Merge remote-tracking branch 'refs/remotes/origin/main' into import-l…
jonathangoulding Aug 20, 2024
915cc6e
Merge branch 'main' into import-licence-versions
jonathangoulding Aug 20, 2024
40f96d3
Merge remote-tracking branch 'refs/remotes/origin/main' into import-l…
jonathangoulding Aug 22, 2024
0b96829
chore: fix lint
jonathangoulding Aug 22, 2024
3e9a7a6
Alans suggested import licence version changes (#1286)
Cruikshanks Aug 27, 2024
94f5a8c
Merge branch 'main' into import-licence-versions
Cruikshanks Aug 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions app/controllers/import.controller.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,16 @@ const Boom = require('@hapi/boom')
* @module ImportController
*/
async function licence (request, h) {
try {
const { licenceRef } = request.payload
// try {
const { licenceRef } = request.payload

await LegacyImportLicenceService.go(licenceRef)
await LegacyImportLicenceService.go(licenceRef)

return h.response().code(204)
} catch (error) {
return Boom.badImplementation(error.message)
}
return h.response().code(204)
// } catch (error) {
// log licence ref error in logs ?
// return Boom.badImplementation(error.message)
// }
}

module.exports = {
Expand Down
95 changes: 74 additions & 21 deletions app/services/import/legacy-import/fetch-licence-versions.service.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@
*/
const { db } = require('../../../../db/db.js')

/**
* Gets the licence versions
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
*
* Returns the licence version purposes
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
*
* @param {LegacyLicenceType} licenceData - the licence
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
* @returns {Promise<LegacyLicenceVersionsArray>}
*/
async function go (licenceData) {
const { ID: licenceId, FGAC_REGION_CODE: regionCode } = licenceData

Expand All @@ -14,29 +22,47 @@ async function go (licenceData) {

async function _getLicenceVersions (licenceId, regionCode) {
const query = `
SELECT *
FROM import."NALD_ABS_LIC_VERSIONS" v
WHERE v."FGAC_REGION_CODE" = '${regionCode}'
AND v."AABL_ID" = '${licenceId}'
AND v."STATUS" <> 'DRAFT';`
SELECT
versions."EFF_END_DATE",
versions."EFF_ST_DATE",
versions."INCR_NO",
versions."ISSUE_NO",
versions."STATUS",
versions."FGAC_REGION_CODE",
versions."AABL_ID",
(SELECT json_agg(json_build_object(
'PERIOD_END_DAY', purposes."PERIOD_END_DAY",
'PERIOD_END_MONTH', purposes."PERIOD_END_MONTH",
'PERIOD_ST_DAY', purposes."PERIOD_ST_DAY",
'PERIOD_ST_MONTH', purposes."PERIOD_ST_MONTH",
'ANNUAL_QTY', purposes."ANNUAL_QTY",
'DAILY_QTY', purposes."DAILY_QTY",
'FGAC_REGION_CODE', purposes."FGAC_REGION_CODE",
'ID', purposes."ID",
'HOURLY_QTY', purposes."HOURLY_QTY",
'INST_QTY', purposes."INST_QTY",
'NOTES', purposes."NOTES",
'APUR_APPR_CODE', purposes."APUR_APPR_CODE",
'APUR_APSE_CODE', purposes."APUR_APSE_CODE",
'APUR_APUS_CODE', purposes."APUR_APUS_CODE",
'TIMELTD_END_DATE', purposes."TIMELTD_END_DATE",
'TIMELTD_ST_DATE', purposes."TIMELTD_ST_DATE"
)
)
FROM import."NALD_ABS_LIC_PURPOSES" purposes
WHERE purposes."AABV_AABL_ID" = versions."AABL_ID"
AND purposes."AABV_ISSUE_NO" = versions."ISSUE_NO"
AND purposes."AABV_INCR_NO" = versions."INCR_NO"
AND purposes."FGAC_REGION_CODE" = versions."FGAC_REGION_CODE") AS purposes
FROM import."NALD_ABS_LIC_VERSIONS" versions
WHERE versions."FGAC_REGION_CODE" = '${regionCode}'
AND versions."AABL_ID" = '${licenceId}'
AND versions."STATUS" <> 'DRAFT';
`

const { rows } = await db.raw(query)

return select(rows)
}

/**
* Maps licence versions to the required format / data
*
* @param {{}} rows - the licence version columns
* @returns {LegacyLicenceVersionsArray}
*/
function select (rows) {
return rows.map((row) => {
return {
EFF_ST_DATE: row.EFF_ST_DATE
}
})
return rows
}

module.exports = {
Expand All @@ -47,10 +73,37 @@ module.exports = {
* A legacy licence version
* @typedef {Object} LegacyLicenceVersionsType
*
* @property {string} EFF_ST_DATE - date in UK format | 'null'
* @property {string} EFF_END_DATE - date in UK format - can be 'null'
* @property {string} EFF_ST_DATE - date in UK format
* @property {string} INCR_NO - a number between 1 - 5
* @property {string} ISSUE_NO - a number - linked to the purpose id ?
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
* @property {string} STATUS - enum - 'DRAFT', 'SUPER', 'CURR' (Draft will not be selected)
* @property {string} FGAC_REGION_CODE
* @property {string} AABL_ID
* @property {LegacyLicenceVersionsPurposesType} purposes
*/

/**
* An array of legacy licence versions.
* @typedef {LegacyLicenceVersionsType[]} LegacyLicenceVersionsArray
*/

/**
* @typedef {Object} LegacyLicenceVersionsPurposesType
* @property {string} PERIOD_END_DAY - The end day of the period.
* @property {string} PERIOD_END_MONTH - The end month of the period.
* @property {string} PERIOD_ST_DAY - The start day of the period.
* @property {string} PERIOD_ST_MONTH - The start month of the period.
* @property {string} ANNUAL_QTY - The annual quantity.
* @property {string} DAILY_QTY - The daily quantity.
* @property {string} FGAC_REGION_CODE - The FGAC region code.
* @property {string} ID - The identifier.
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
* @property {string} HOURLY_QTY - The hourly quantity.
* @property {string} INST_QTY - The instant quantity.
* @property {string} NOTES - Additional notes.
* @property {string} APUR_APPR_CODE - The APUR approval code.
* @property {string} APUR_APSE_CODE - The APUR secondary code.
* @property {string} APUR_APUS_CODE - The APUR usage code.
* @property {string} TIMELTD_END_DATE - The time-limited end date.
* @property {string} TIMELTD_ST_DATE - The time-limited start date.
*/
42 changes: 15 additions & 27 deletions app/services/import/legacy-import/fetch-licence.service.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,38 +17,26 @@ async function go (licenceRef) {
}

async function _getLicenceByRef (licenceRef) {
// TODO: granular select ? map data to simplify if needs be
const query = `
SELECT *
FROM import."NALD_ABS_LICENCES" l
WHERE l."LIC_NO" = '${licenceRef}';
SELECT
licence.AREP_AREA_CODE,
licence.AREP_EIUC_CODE,
licence.AREP_LEAP_CODE,
licence.AREP_SUC_CODE,
licence.EXPIRY_DATE,
licence.FGAC_REGION_CODE,
licence.ID,
licence.LAPSED_DATE,
licence.LIC_NO,
licence.ORIG_EFF_DATE,
licence.REV_DATE
FROM import."NALD_ABS_LICENCES" licence
WHERE licence."LIC_NO" = '${licenceRef}';
`

const { rows: [row] } = await db.raw(query)

return select(row)
}

/**
* Fetches the licence data for the licence ref from the import.NALD_ABS_LICENCES table
*
* @param {any} licence
* @returns {LegacyLicenceType}
*/
function select (licence) {
return {
AREP_AREA_CODE: licence.AREP_AREA_CODE,
AREP_EIUC_CODE: licence.AREP_EIUC_CODE,
AREP_LEAP_CODE: licence.AREP_LEAP_CODE,
AREP_SUC_CODE: licence.AREP_SUC_CODE,
EXPIRY_DATE: licence.EXPIRY_DATE,
FGAC_REGION_CODE: licence.FGAC_REGION_CODE,
ID: licence.ID,
LAPSED_DATE: licence.LAPSED_DATE,
LIC_NO: licence.LIC_NO,
ORIG_EFF_DATE: licence.ORIG_EFF_DATE,
REV_DATE: licence.REV_DATE
}
return row
}

module.exports = {
Expand Down
95 changes: 95 additions & 0 deletions app/services/import/legacy-import/licence-versions.mapper.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
'use strict'

/**
* Maps the import licence versions data to the desired format
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
* @module LegacyImportLicenceVersionMapper
*/

const { formatStandardDateToISO } = require('../../../lib/dates.lib.js')

const statuses = {
CURR: 'current',
SUPER: 'superseded',
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
DRAFT: 'draft' // todo: check this can be removed as it the sql does not get draft status
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
}

const createExternalId = (licenceVersion) => {
const { FGAC_REGION_CODE, AABL_ID, ISSUE_NO, INCR_NO } = licenceVersion

return `${FGAC_REGION_CODE}:${AABL_ID}:${ISSUE_NO}:${INCR_NO}`
}

/**
* Maps the import licence versions data
*
* @param {LegacyLicenceVersionsArray} licenceVersions
* @returns {ImportLicenceVersionType[]}
*/
function go (licenceVersions) {
return _mapLicenceVersions(licenceVersions)
}

/**
* Iterates the import licence versions
*
* @param {LegacyLicenceVersionsArray} licenceVersions
* @returns {ImportLicenceVersionType[]}
*/
function _mapLicenceVersions (licenceVersions) {
return licenceVersions.map((licenceVersion) => {
const issue = licenceVersion.ISSUE_NO // mapped to the legacy purpose id -
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
const increment = licenceVersion.INCR_NO

return {
endDate: formatStandardDateToISO(licenceVersion.EFF_END_DATE),
externalId: createExternalId(licenceVersion),
increment: Number(increment),
issue: Number(issue),
startDate: formatStandardDateToISO(licenceVersion.EFF_ST_DATE),
status: statuses[licenceVersion.STATUS],
purposes: _mapPurposes(licenceVersion)
}
})
}

/**
* Iterates the import licence versions purposes
*
* @param {LegacyLicenceVersionsType} licenceVersion
* @returns {ImportLicenceVersionPurposeType}
*/
function _mapPurposes (licenceVersion) {
return licenceVersion.purposes.map((purpose) => {
return _mapPurpose(purpose)
})
}

/**
* Maps the import licence versions purposes data to the desired format
jonathangoulding marked this conversation as resolved.
Show resolved Hide resolved
*
* @param {LegacyLicenceVersionsPurposesType} purpose
* @returns {ImportLicenceVersionPurposeType}
*/
const _mapPurpose = (purpose) => {
return {
abstractionPeriodEndDay: Number(purpose.PERIOD_END_DAY),
abstractionPeriodEndMonth: Number(purpose.PERIOD_END_MONTH),
abstractionPeriodStartDay: Number(purpose.PERIOD_ST_DAY),
abstractionPeriodStartMonth: Number(purpose.PERIOD_ST_MONTH),
annualQuantity: purpose.ANNUAL_QTY === 'null' ? null : Number(purpose.ANNUAL_QTY),
dailyQuantity: purpose.DAILY_QTY === 'null' ? null : Number(purpose.DAILY_QTY),
externalId: `${purpose.FGAC_REGION_CODE}:${purpose.ID}`,
hourlyQuantity: purpose.HOURLY_QTY === 'null' ? null : Number(purpose.HOURLY_QTY),
instantQuantity: purpose.INST_QTY === 'null' ? null : Number(purpose.INST_QTY),
notes: purpose.NOTES === 'null' ? null : purpose.NOTES,
primaryPurposeId: purpose.APUR_APPR_CODE,
secondaryPurposeId: purpose.APUR_APSE_CODE,
purposeId: purpose.APUR_APUS_CODE,
timeLimitedEndDate: formatStandardDateToISO(purpose.TIMELTD_END_DATE),
timeLimitedStartDate: formatStandardDateToISO(purpose.TIMELTD_ST_DATE)
}
}

module.exports = {
go
}
2 changes: 1 addition & 1 deletion app/services/import/legacy-import/licence.mapper.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ function _mapLicence (licence, licenceVersions) {
* Creates a JSON object of the region data
*
* @param {Object} licenceData
* @return {RegionsType}
* @return {ImportRegionType}
*/
const _regions = (licenceData) => {
const historicalAreaCode = licenceData.AREP_AREA_CODE
Expand Down
17 changes: 7 additions & 10 deletions app/services/import/legacy-licence.service.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@

const FetchLegacyImportLicenceService = require('./legacy-import/fetch-licence.service.js')
const FetchLegacyImportLicenceVersionsService = require('./legacy-import/fetch-licence-versions.service.js')
const LegacyImportLicenceMapper = require('./legacy-import/licence.mapper.js')
const ImportLicenceValidatorService = require('./licence-validator.service.js')
const LegacyImportLicenceMapper = require('./legacy-import/licence.mapper.js')
const LegacyImportLicenceVersionMapper = require('./legacy-import/licence-versions.mapper.js')
const PersistLicenceService = require('./persist-licence.service.js')
const PersistLicenceVersionsService = require('./persist-licence-versions.service.js')

/**
* Imports a licence from the legacy import tables. Maps and validates the data and then saves to the database.
Expand All @@ -21,22 +23,17 @@ async function go (licenceRef) {
console.debug('Importing licence ref: ', licenceRef)
const licenceData = await FetchLegacyImportLicenceService.go(licenceRef)

console.debug('Imported licence data: ', licenceData)
const licenceVersionsData = await FetchLegacyImportLicenceVersionsService.go(licenceData)

console.debug('Imported licence versions data: ', licenceVersionsData)
const mappedLicenceData = LegacyImportLicenceMapper.go(licenceData, licenceVersionsData)

const mappedLicenceData = await LegacyImportLicenceMapper.go(licenceData, licenceVersionsData)
const mappedLicenceVersionsData = LegacyImportLicenceVersionMapper.go(licenceVersionsData)

console.debug('Mapped imported licence data: ', mappedLicenceData)

ImportLicenceValidatorService.go(mappedLicenceData)
ImportLicenceValidatorService.go(mappedLicenceData, mappedLicenceVersionsData)

const savedLicence = await PersistLicenceService.go(mappedLicenceData)

console.debug('Saved Licence: ', savedLicence)

return savedLicence
await PersistLicenceVersionsService.go(mappedLicenceVersionsData, savedLicence.id)
}

module.exports = {
Expand Down
Loading
Loading