Trigger new licence end date change endpoints #1049
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://eaflood.atlassian.net/browse/WATER-4546 > Part of the work to migrate management of return versions to WRLS from NALD During the import from NALD, if a licence end date is changed, for example, if it has been revoked in NALD, our return version functionality needs to know about it. This is so we can reissue the return logs for the licence to match the changed end date. We are actively trying to move away from the legacy code base, so this work was always going to be done in [water-abstraction-system](https://github.com/DEFRA/water-abstraction-system). Initially, it would be triggered by our [own import process](https://eaflood.atlassian.net/browse/WATER-4535), which was due to replace the legacy one. But that was when we thought ReSP would be taking over from NALD. Now, the plan is for WRLS to encompass the final abstraction leg and take over from NALD. So, there is little point in replacing a complex import we intend to switch off in the next year. But, knowing we'd need something in the interim, we created the [/jobs/licence-changes job](DEFRA/water-abstraction-system#1593). The intent was to schedule this after the first [NALD import job](https://github.com/DEFRA/water-abstraction-team/blob/main/jobs/import.md#nald-import) had completed (the one that downloads and extracts the NALD data) but before the main [licence import job](https://github.com/DEFRA/water-abstraction-team/blob/main/jobs/import.md#licence-import). That way, our job could compare the NALD and WRLS licence records to determine if an end date has changed. But then we had our 'doh!' moment. Our reissue return logs engine expects to be given a licence ID and the date the change applies. However, the WRLS licence record needs to have been updated for it to determine the start and end dates of the new return logs. If we schedule `/jobs/licence-changes` before the licence import job, it will see the change but won't reissue anything because the WRLS record won't have been updated. If we schedule it after, it won't see any difference and won't trigger the reissue. Doh! So, we've needed to change tact on this completely. Now, we intend to - _trigger_ `/licences/end-dates/check` from the [NALD import job](https://github.com/DEFRA/water-abstraction-team/blob/main/jobs/import.md#nald-import) rather than schedule it - refactor `/jobs/licence-changes` as `/licences/end-dates`, but instead of processing changed licences, it [stores the details of the change in a table](DEFRA/water-abstraction-service#2666) - add a new `/licences/end-dates/process` endpoint that processes these 'licence change' records, which will be triggered from the [licence import job](https://github.com/DEFRA/water-abstraction-team/blob/main/jobs/import.md#licence-import) This change adds new steps to the NALD and Licence import jobs to trigger these new endpoints.
That endpoint is defunct. So, we replace the existing call with one to our new check endpoint.
All this needs to do is make the call to water-abstraction-system. We do this after the download has been extracted and loaded into the `import` schema, but before any real work starts.
Have it queue up the trigger end date check rather than DeleteRemovedDocuments.
All this needs to do is make the call to water-abstraction-system. We do this after the clean job has completed, but before any real work starts.
Have it queue up the trigger end date process rather than import-purpose-condition-types.
Cruikshanks
added a commit
that referenced
this pull request
Jan 22, 2025
https://eaflood.atlassian.net/browse/WATER-4546 > Part of the work to migrate management of return versions to WRLS from NALD In [Trigger new licence end date change endpoints](#1049) we added new steps to the NALD and Licence import to allow us to trigger functionality in [water-abstraction-system](https://github.com/DEFRA/water-abstraction-system) to trigger functionality linked to changes to licence end dates. It needs two steps, because in the **NALD import** we need to be able to see the difference between NALD and WRLS end dates _before_ the WRLS licence record is updated. Then in **Licence import** we can process the changes _after_ the licence has been updated with the new end date. Sorry, I did say _after_ didn't I?! We realised in testing that we'd placed the second step in **Licence import** before it updates the licence. This means features that depend on both knowing what the change date was, and that the licence has been updated with the new end date are failing. This fix moves the step in **Licence import** to _after_ the steps that update the licence records.
Cruikshanks
added a commit
that referenced
this pull request
Jan 22, 2025
https://eaflood.atlassian.net/browse/WATER-4546 > Part of the work to migrate management of return versions to WRLS from NALD In [Trigger new licence end date change endpoints](#1049), we added new steps to the NALD and Licence import to allow us to trigger functionality in [water-abstraction-system](https://github.com/DEFRA/water-abstraction-system) to trigger functionality linked to changes to licence end dates. It needs two steps because in the **NALD import**, we need to see the difference between NALD and WRLS end dates _before_ the WRLS licence record is updated. Then, in **Licence import**, we can process the changes _after_ the licence has been updated with the new end date. Sorry, I did say _after_ didn't I?! 🤦 We realised in testing that we'd placed the second step in **Licence import** before it updates the licence. This means features that depend on knowing what the change date was and that the licence has been updated with the new end date are failing. This fix moves the step in **Licence import** to _after_ the steps that update the licence records.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://eaflood.atlassian.net/browse/WATER-4546
During the import from NALD, if a licence end date is changed, for example, if it has been revoked in NALD, our return version functionality needs to know about it. This is so we can reissue the return logs for the licence to match the changed end date.
We are actively trying to move away from the legacy code base, so this work was always going to be done in water-abstraction-system. Initially, it would be triggered by our own import process, which was due to replace the legacy one. But that was when we thought ReSP would be taking over from NALD.
Now, the plan is for WRLS to encompass the final abstraction leg and take over from NALD. So, there is little point in replacing a complex import we intend to switch off in the next year.
But, knowing we'd need something in the interim, we created the /jobs/licence-changes job. The intent was to schedule this after the first NALD import job had completed (the one that downloads and extracts the NALD data) but before the main licence import
job. That way, our job could compare the NALD and WRLS licence records to determine if an end date has changed.
But then we had our 'doh!' moment.
Our reissue return logs engine expects to be given a licence ID and the date the change applies. However, the WRLS licence record needs to have been updated for it to determine the start and end dates of the new return logs.
If we schedule
/jobs/licence-changesbefore the licence import job, it will see the change but won't reissue anything because the WRLS record won't have been updated. If we schedule it after, it won't see any difference and won't trigger the reissue.Doh!
So, we've needed to change tact on this completely. Now, we intend to
/licences/end-dates/checkfrom the NALD import job rather than schedule it/jobs/licence-changesas/licences/end-dates, but instead of processing changed licences, it stores the details of the change in a table/licences/end-dates/processendpoint that processes these 'licence change' records, which will be triggered from the licence import jobThis change adds new steps to the NALD and Licence import jobs to trigger these new endpoints.