Fix: Apply Legacy Reference Numbers from Source Data #21
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reorganise learner construction by splitting monolithic functions into focused factory modules. Move files from src/utils to src/factories to better reflect their purpose as data builders rather than general utilities. - Extract reference number, learner details, health details, prior attainment, and employment logic into separate factory modules - Simplify pushLearners by composing learner objects from factory functions - Update imports to reflect new factory structure - Clean up formatting in main.js and buildLearningDeliveryArray
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR fixes a critical data integrity bug where learner reference numbers were being auto-generated instead of using the actual legacy values from source data, and previous learner reference numbers were being completely omitted from ILR files.
Tip
No additional steps required after pulling. Existing ILR files generated with auto-generated reference numbers will need to be regenerated to include correct legacy references.
Changes
🔴 Critical Bug Fix: Reference Number Handling
The Problem:
LearnRefNumberwas being auto-generated from the loop index (0001, 0002, 0003...) instead of reading actual legacy reference numbers from the CSVPrevLearnRefNumberwas completely missing from learner records - not being included in ILR files at allThe Impact:
The Fix:
LearnRefNumberfrom column 223 of the source CSV (the actual legacy reference)PrevLearnRefNumberfrom column 222 and includes it in learner recordsRefactoring: Factory Module Pattern
Extracted learner-building logic into dedicated factory modules to prevent similar bugs and improve maintainability:
buildReferenceNumbers.js- Handles all learner reference numbers (LearnRefNumber, ULN, PrevLearnRefNumber)buildLearnerDetails.js- Core learner information (name, DOB, contact details)buildHealthDetails.js- LLDD and health problem databuildPriorAttainment.js- Prior attainment informationEach module has a single, clear responsibility, making it obvious where data comes from and easier to spot mistakes.
src/utils/pushLearners.jsSimplified from 50+ lines to 24 lines by composing learner objects from focused factory functions, making the data flow clearer and reducing the risk of field mapping errors.
src/factories/buildEmploymentArray.jsMoved from
src/utils/tosrc/factories/to align with new factory module pattern.docs/inputs/Updated schema documentation:
25_26 Export.csvto25-26 Export.csv(standardised naming)25_26 Example.csvwith current field mappings25_26 Properties.csvreference documenting CSV column structureSummary
Like discovering your filing cabinet has been using "File001, File002, File003" instead of the actual document reference numbers written on each file, then realising some files are missing their "Previous Reference" labels entirely - this PR restores the proper reference numbers so everything links up correctly with the historical record system.