Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transporter 3.0.0 #725

Merged
merged 355 commits into from
Nov 1, 2022
Merged
Changes from 1 commit
Commits
Show all changes
355 commits
Select commit Hold shift + click to select a range
1536a71
npm update, theoretically nonbreaking
axoplasm Jul 28, 2022
3cdaf5c
Update package-lock.json
axoplasm Jul 28, 2022
52f11ae
Silence babel notices re: “loose”
axoplasm Jul 29, 2022
b98cca3
update nonbreaking vulnerabilities
axoplasm Aug 2, 2022
1504a29
update package.json to match lockfile
axoplasm Aug 2, 2022
5cac3bc
update to node-sass v7
axoplasm Aug 2, 2022
5978dd0
Fixed gaitids errors from the program admin lite
blakelong Aug 4, 2022
7a59e1f
Updated validation to check for any changes. Update to add modal laun…
Aug 4, 2022
89c4218
Fixed failing test case
blakelong Aug 4, 2022
8dba369
Added utility function to check for duplicate gaitids on IDAA
blakelong Aug 4, 2022
cba83f5
Merge branch 'transporter-3.0.0' into 2764-discrepancy_report
blakelong Aug 4, 2022
e2efa55
Updated Overview tab to static text. Update discrepancy report test c…
blakelong Aug 4, 2022
6d857ef
For mercycorps/TolaActivity#2770. Fixed GaitID errors from the progra…
blakelong Aug 5, 2022
bbffbba
Refactor modal code.
Aug 5, 2022
c78e8eb
Reverting code change.
Aug 5, 2022
ab36009
Merge branch 'transporter-3.0.0' into 2770-admin-light-profile-tab-up…
Aug 5, 2022
95c7554
Updated gait ids and fund code field spacing. Updated disabling field…
Aug 5, 2022
7c16a8a
For mercycorps/TolaActivity#2764 Discrepancy report updates
blakelong Aug 8, 2022
d814fc1
For mercycorps/TolaActivity#2810 Check for duplicate gaitids
blakelong Aug 8, 2022
dbad7ee
Merge branch 'transporter-3.0.0' into 2770-admin-light-profile-tab-up…
Aug 8, 2022
d2ae915
Refactor and updates comments.
Aug 8, 2022
8f73fe8
Update for making form not editable.
Aug 8, 2022
2544070
Added disabling form after submission to prevent button smashing.
Aug 8, 2022
432e051
Update to display outcome theme names only and not ids
Aug 8, 2022
ec3c3ea
Updated API update failure handling. Added a waiting class. Added sty…
Aug 9, 2022
39104a6
Merge pull request #638 from mercycorps/2770-admin-light-profile-tab-…
andrethomas6 Aug 9, 2022
2d285b1
Refactor code structure and updated validation and alert messages. Ad…
Aug 9, 2022
3823915
Merge branch 'transporter-3.0.0' into 2771-program-period-modal-new
Aug 9, 2022
7b630ad
Added email notification to country admins when dates are updated
blakelong Aug 10, 2022
dd485ae
For mercycorps/TolaActivity#2819 Country admin email notification
blakelong Aug 10, 2022
21f22b1
Change search_fields in ResultAdmin class (#644)
kbarkemeyer Aug 11, 2022
8edc5f6
For mercycorps/TolaActivity#2771, updated validation text and transla…
Aug 11, 2022
9506975
Clean up existing workflow admin
axoplasm Aug 11, 2022
9d26256
Clean up existing Indicator admin
axoplasm Aug 12, 2022
5cf0956
Merge pull request #645 from mercycorps/2771-program-period-modal-new
andrethomas6 Aug 12, 2022
accd4d4
IDAASector (note verbose_name change)
axoplasm Aug 16, 2022
3abf208
Program Discrepancy
axoplasm Aug 16, 2022
062f682
GAIT ID
axoplasm Aug 16, 2022
05165cb
Fundcode
axoplasm Aug 16, 2022
b900948
IDAA outcome theme
axoplasm Aug 16, 2022
217b261
register & clean up some stray models with lousy admin
axoplasm Aug 16, 2022
21c1288
Clean up Program model admin
axoplasm Aug 16, 2022
e0aa8d5
Return IDAA program name for discrepancies w/out program names
axoplasm Aug 16, 2022
761ac63
Pinned Reports and Level Tier admins
axoplasm Aug 17, 2022
1b008c7
Level Tier templates, misc tweaks
axoplasm Aug 17, 2022
9ae92bc
Bulk indicator import files
axoplasm Aug 17, 2022
b5be589
User Management Audit Log
axoplasm Aug 17, 2022
97aec48
2766 new program email notification (#646)
kbarkemeyer Aug 17, 2022
d250eea
removed catch for SMTPRecipientsRefused
blakelong Aug 17, 2022
80e9d3b
ProgramAuditLog
axoplasm Aug 17, 2022
b5f9438
For mercycorps/TolaActivity#2819 Removed catch for SMTPRecipientsRefused
blakelong Aug 17, 2022
a6c578e
ProgramAdminAuditLog
axoplasm Aug 17, 2022
06b4ed9
OrganizationAdminAuditLog
axoplasm Aug 17, 2022
3ea63b6
CountryAdminAuditLog and DisaggregationType
axoplasm Aug 17, 2022
6c20dde
Remove LandTypeAdmin
axoplasm Aug 17, 2022
9ed792c
Uncommented out the editable fields.
Aug 18, 2022
75c99e7
Fixed country page date picker.
Aug 18, 2022
e9df869
Merge pull request #649 from mercycorps/2771-program-period-modal-new
andrethomas6 Aug 18, 2022
a8d47e7
Added new tab to discrepancy report for duplicate IDAA programs
blakelong Aug 18, 2022
f8744e8
For mercycorps/TolaActivity#2764 Added tab Duplicate IDAA Programs to…
blakelong Aug 18, 2022
f947d81
Merge pull request #648 from mercycorps/2822-django-admin-tables
axoplasm Aug 18, 2022
625316e
ResultsAdmin
axoplasm Aug 18, 2022
751d2ff
Updated fields for the ProgramAdminSerializer
blakelong Aug 18, 2022
ca57a7d
Merge branch 'transporter-3.0.0' into 2820-editable-program-admin-light
Aug 19, 2022
89bfe9e
Add pc indicator to test programs (#652)
kbarkemeyer Aug 22, 2022
0be6c2f
Fix for resolved idaa discrepancies
blakelong Aug 22, 2022
c0819a6
Fix for fund codes displaying as PK value on change log
blakelong Aug 22, 2022
0a479e0
Merge pull request #653 from mercycorps/program-upload-create-and-fun…
blakelong Aug 22, 2022
01ebcf2
Merge pull request #651 from mercycorps/2825-indicator-results-admin
axoplasm Aug 23, 2022
bd1c43d
For mercycorps/TolaActivity#2820 Updated fields for the ProgramAdminS…
blakelong Aug 24, 2022
a5715dc
Allowed numpad decimal to be entered for number fields
blakelong Aug 24, 2022
977adcf
Fix for countries and sectors incorrectly showing as updated in statu…
blakelong Aug 25, 2022
f988aff
Update create_qa_programs to reflect program model changes (#658)
kbarkemeyer Aug 25, 2022
4d5b771
Added idaa_sectors and idaa_outcome_themes to program_page_context en…
blakelong Aug 25, 2022
9db3ca1
For mercycorps/TolaActivity#2792 Fix for countries and sectors incorr…
blakelong Aug 25, 2022
182479f
For mercycorps/TolaActivity#2820 Added idaa_sectors and idaa_outcome_…
blakelong Aug 26, 2022
ec335c2
Added list of created and updated programs to IDAA upload report
blakelong Aug 26, 2022
f35e644
For mercycorps/TolaActivity#2823 Added list of created and updated pr…
blakelong Aug 29, 2022
4b9a5d7
added external_program_id to ProgramAdminSerializer
blakelong Aug 30, 2022
9486220
external_program_id formerrors
blakelong Aug 30, 2022
d26adb6
For mercycorps/TolaActivity#2770 added external_program_id to Program…
blakelong Aug 30, 2022
a527856
Updated required fields and labels, max char lengths. Gait row update…
Aug 30, 2022
e969e98
Merge transporter-3.0.0
Aug 30, 2022
04b709c
Merge transporter-3.1.0 and fix conflict and errors.
Aug 30, 2022
b7b9745
For mercycorps/TolaActivity#2770, display none for null idaa program …
Aug 30, 2022
7e33bed
fixed typo for updated_programs
blakelong Aug 31, 2022
f39bc12
added idaa_sectors to program profile page. Updated sector filters to…
blakelong Aug 31, 2022
b18dcbd
added donor and donor_dept to idaa_logged_fields
blakelong Aug 31, 2022
3d5836b
install django-admin-autocomplete-filter
axoplasm Aug 31, 2022
1b78093
Autocomplete filters for results
axoplasm Aug 31, 2022
f181a09
Autocomplete (and other) filters for Indicators
axoplasm Aug 31, 2022
232a2db
Autocomplete (and other) filters for Level Tier
axoplasm Aug 31, 2022
5eaab6d
Autocomplete filter for Level
axoplasm Aug 31, 2022
9390c3c
Merge branch 'transporter-3.0.0' into 2822-django-admin-tables
axoplasm Aug 31, 2022
40ceb0c
Autocomplete fields & list filters for Periodic targets
axoplasm Aug 31, 2022
189be8b
autocomplete fields & list filters for Program Objectives
axoplasm Aug 31, 2022
4ce497f
autocomplete list filter for Fund Code
axoplasm Aug 31, 2022
9670b18
autocomplete list filter for GAIT ID
axoplasm Aug 31, 2022
3d8607f
sector filter for Program admin
axoplasm Aug 31, 2022
f6887a7
Update list filters for audit log admin
axoplasm Aug 31, 2022
20cbbd3
Date hierarchy filters for audit log admin screens
axoplasm Aug 31, 2022
64bd7aa
Merge pull request #663 from mercycorps/2770-update-program-id
andrethomas6 Sep 1, 2022
7245f63
resetting the user form now resets the organization field
blakelong Sep 6, 2022
7cfb646
For mercycorps/TolaActivity#1445 resetting the user form now resets t…
blakelong Sep 6, 2022
86a8ae8
For mercycorps/TolaActivity#2823 Fixed typo for updated programs
blakelong Sep 6, 2022
cad77db
For mercycorps/TolaActivity#2770 Added IDAASectors to program page pr…
blakelong Sep 6, 2022
3c76529
For mercycorps/TolaActivity#2772 Added donor and donor_dept to idaa_l…
blakelong Sep 6, 2022
8b07f12
resolved merge conflict between transporter-3.0.0 and 3.1.0
blakelong Sep 7, 2022
38ea4df
Add “cancel changes” button to CSO
axoplasm Sep 7, 2022
e09a956
Merge pull request #672 from mercycorps/transporter-3.1.0
axoplasm Sep 8, 2022
820a0de
Merge pull request #667 from mercycorps/2822-django-admin-tables
axoplasm Sep 8, 2022
e89fcad
mismatching countries and multiple program matches now update
blakelong Sep 12, 2022
ed8aa8f
Installed React datepicker and added it for the start date.
Sep 13, 2022
8cabebb
For mercycorps/TolaActivity#2767 mismatching countries and multiple p…
blakelong Sep 13, 2022
7ff9810
Added additional logging to the upload IDAA programs command
blakelong Sep 13, 2022
fe9073d
Merge branch 'transporter-3.0.0' into 2842-program-upload-logging
blakelong Sep 13, 2022
e76e2b9
For mercycorps/TolaActivity#2842 Additional logging to the upload IDA…
blakelong Sep 14, 2022
1dc93bc
For mercycorps/TolaActivity#2870 Allow numpad decimal key for number …
blakelong Sep 14, 2022
4c42870
For mercycorps/TolaActivity#2771, fixed typo for the read only serial…
Sep 14, 2022
e3f0b58
both reset buttons read “Cancel changes”
axoplasm Sep 14, 2022
b8f2b86
Merge pull request #673 from mercycorps/1447-reset-button-for-csos
axoplasm Sep 15, 2022
1366b5b
Merge pull request #634 from mercycorps/2673-upgrade-js-redux
axoplasm Sep 15, 2022
623b8a5
removed legacy api calls to Gait when a program is saved from the admin
blakelong Sep 16, 2022
4c55224
Merge pull request #677 from mercycorps/2771-program-period-modal-new
andrethomas6 Sep 16, 2022
f1b6acb
Add style sheet for the react datepicker. added the end date. Added a…
Sep 19, 2022
c25ff8d
Fix PC Count datepicker by using the elements id instead of class.
Sep 19, 2022
c6c3784
Merge pull request #679 from mercycorps/pc-count-result-datepicker-fix
andrethomas6 Sep 19, 2022
7c60dd0
For mercycorps/TolaActivity#2770, Updated logic for creating donor an…
Sep 19, 2022
c013b2a
Merge pull request #680 from mercycorps/2770-admin-light-profile-tab-…
andrethomas6 Sep 19, 2022
56b03fc
Merged transporter-3.0.0
Sep 19, 2022
5264dfd
removed legacy api calls to Gait when a program is saved from the admin
blakelong Sep 20, 2022
588b9e3
one idaa to many tola programs will not update program name
blakelong Sep 20, 2022
abc177b
2821 update create qa programs (#681)
kbarkemeyer Sep 20, 2022
b704b44
delete fundcode if deleted in idaa
blakelong Sep 20, 2022
6beaafe
valid_tracking_dates now checks the range from the first and last of …
blakelong Sep 20, 2022
80a4e26
prevent multiple_program discrepancies from having other discrepancies
blakelong Sep 20, 2022
b671ce7
fixed issue where not all discrepancies are captured
blakelong Sep 20, 2022
7b4e7bd
updated text on discrepancy report
blakelong Sep 20, 2022
99bf035
updated test cases
blakelong Sep 20, 2022
e822c2c
fixed error when writing an invalid idaa program to the log
blakelong Sep 20, 2022
b31c146
skipped test cases
blakelong Sep 20, 2022
d86b2bc
Fixes issues related to updating an existing program
blakelong Sep 20, 2022
f695ec8
Xanadu management command (#683)
kbarkemeyer Sep 27, 2022
4d1f841
Auto-create missing countries in TolaData
blakelong Sep 27, 2022
5dab33f
search on indicator name
axoplasm Sep 27, 2022
bb59064
For mercycorps/TolaActivity#2876 Auto-create new countries in TolaData
blakelong Sep 27, 2022
509144e
Merge pull request #685 from mercycorps/2878-periodic-target-search-e…
axoplasm Sep 27, 2022
323b89f
fixed issue where sectors and outcome themes are deleted from the DB
blakelong Sep 28, 2022
b2df99b
For mercycorps/TolaActivity#2767 fixed issue where sectors and outcom…
blakelong Sep 28, 2022
0cb3cb5
Remove drilldown filters from audit log admin lists
axoplasm Sep 28, 2022
1f5981a
Change/edit date fields are read-only in workflow app
axoplasm Sep 28, 2022
53511ad
Change/edit date fields are read-only in indicators app
axoplasm Sep 28, 2022
ca9028f
change/edit fields read-only in proxy admin models
axoplasm Sep 28, 2022
189ab3f
pinned report creation date field is read-only
axoplasm Sep 28, 2022
0099224
Merge pull request #688 from mercycorps/2880-audit-log-drilldown-filter
axoplasm Sep 28, 2022
feca42e
For mercycorps/TolaActivity#2841 unique IDAASectors and OutcomeThemes…
kbarkemeyer Sep 28, 2022
dd656dd
Merge pull request #689 from mercycorps/2879-change-edit-dates
axoplasm Sep 29, 2022
58b8f2e
Management command to alias all country admin emails
blakelong Sep 29, 2022
3876ec5
legacy program fields are readonly
axoplasm Sep 29, 2022
1abad0f
organize program detail into fieldsets
axoplasm Sep 29, 2022
90270d6
show fundcodes, donors; autocomplete tolauser
axoplasm Sep 29, 2022
2b89c70
GAIT ID as inline form on program
axoplasm Sep 29, 2022
eb55e5d
(remove read-only GAIT fields)
axoplasm Sep 29, 2022
7281b09
Some program fields now required in django admin
axoplasm Sep 29, 2022
fbd710d
fixed server error on django admin program discrepancy page
blakelong Sep 29, 2022
2824d59
Add verbose names (plural) (#690)
kbarkemeyer Sep 29, 2022
c97195d
fixed server error on django admin program discrepancy page
blakelong Sep 29, 2022
42928bc
Remove discrepancies for IDAA programs switched from funded to not fu…
blakelong Sep 30, 2022
39a2b4d
Save program_upload and sharepoint_info logs at LOGFILE directory
blakelong Sep 30, 2022
ac38782
GAIT ID inline reads better stacked
axoplasm Oct 2, 2022
42c5f34
Log removing old sectors. Translations for IDAA changes history log
blakelong Oct 3, 2022
386a402
Moved command to tola directory. Alias user emails
blakelong Oct 3, 2022
8222fc0
Updated test case for alias user emails
blakelong Oct 3, 2022
61f3c62
Management command to alias all user emails
blakelong Oct 3, 2022
698a30c
2821 update create qa programs (#697)
kbarkemeyer Oct 3, 2022
0517f80
Merge pull request #694 from mercycorps/2875-program-admin-detail
axoplasm Oct 4, 2022
0df106f
remove fundcodes from the tola program
blakelong Oct 4, 2022
5b4e81d
convert idaa fundcodes to int
blakelong Oct 4, 2022
08a6f6c
Added/updated react datepicker, updated fund code field for multiple …
Oct 4, 2022
007b4e5
Merge branch 'transporter-3.0.0' into 2820-editable-program-admin-light
Oct 4, 2022
c5a39ae
Remove fundcodes from the tola program. For mercycorps/TolaActivity#2767
blakelong Oct 4, 2022
469cfa0
Return outcometheme id only (#699)
kbarkemeyer Oct 4, 2022
8573dc4
Save program_upload and sharepoint_info logs at LOGFILE directory
blakelong Oct 4, 2022
c1dbfe9
Remove discrepancies for IDAA programs switched from funded to not fu…
blakelong Oct 4, 2022
824bc2e
Log removing old sectors. Translations for IDAA changes in the histor…
blakelong Oct 4, 2022
9fda50d
Fixed issue where program name changes were logged twice
blakelong Oct 5, 2022
054156c
Fixed issue where program name changes were logged twice
blakelong Oct 5, 2022
a78253f
Added front end validation for required fields and gaitids.
Oct 6, 2022
dfbe66d
GaitID.fund_code() returns a list of related fundcodes
axoplasm Oct 6, 2022
acab064
Merge branch 'transporter-3.0.0' into 2820-editable-program-admin-light
Oct 6, 2022
7b6ff6c
donor, donor_dept are optional (WIP)
axoplasm Oct 6, 2022
b97e9d4
migrations for verbose names in PR #690
axoplasm Oct 6, 2022
17e0c60
Added a dash between donor and donor dept in profile tab.
Oct 6, 2022
40677d7
Merge pull request #702 from mercycorps/2770-update-donor-dept
andrethomas6 Oct 6, 2022
eb9bc91
Merge pull request #701 from mercycorps/654-verbose-name-migrations
axoplasm Oct 6, 2022
d18fb48
Merge branch 'transporter-3.0.0' into 2886-gaitid-form
axoplasm Oct 6, 2022
2aa7099
Migrations for optional GAIT ID fields
axoplasm Oct 6, 2022
70321ba
Seperate donor and donor_dept with a dash in history log
blakelong Oct 6, 2022
05008a2
Separate donor and donor_dept with a dash in the history log
blakelong Oct 6, 2022
fd9ca8f
Change max_length for IDAAOutcomeTheme.name to 255 (#704)
kbarkemeyer Oct 6, 2022
f7ddc64
translations for the country created history log
blakelong Oct 7, 2022
1adce56
Inline fundcode field
axoplasm Oct 7, 2022
49c3c10
translations for the country created history log
blakelong Oct 10, 2022
b63474c
fund_codes list in inline GaitIDAdmin (program edit)
axoplasm Oct 10, 2022
b0e3046
(tuples for consistency)
axoplasm Oct 10, 2022
a9cabd8
Merge pull request #706 from mercycorps/2886-gaitid-form
axoplasm Oct 10, 2022
6de1497
Merge branch 'master' into merge-main-transporter-3.0.0
blakelong Oct 11, 2022
44df446
fixed pc result update test case
blakelong Oct 11, 2022
82dda04
Merge main branch into transporter
blakelong Oct 11, 2022
aaa1344
Remove line breaks from program name. Prevent NoneType: None log
blakelong Oct 12, 2022
7a3c58e
Merge branch 'transporter-3.0.0' into 2842-program-upload-logging-fix
blakelong Oct 12, 2022
fd25fab
2884 allow empty fields programadmin serializer (#708)
kbarkemeyer Oct 12, 2022
f36c1a4
Updates to the program upload logging
blakelong Oct 12, 2022
fc84917
2884 allow empty fields programadmin serializer (#711)
kbarkemeyer Oct 12, 2022
ada1e72
Change fundcode for QA programs (#712)
kbarkemeyer Oct 13, 2022
ef28463
Added validation for all fields with error messages and field highlig…
Oct 13, 2022
0456657
Merged Transporter-3.0.0
Oct 13, 2022
311f9c6
Update save and update functions. Restructured component props. Added…
Oct 17, 2022
f4f465a
Removed console logs
Oct 17, 2022
6391ab4
Updated translations and fixed the update fund code logic.
Oct 17, 2022
530d554
Update to fix the fund code update function with field name.
Oct 17, 2022
f32c096
For mercycorps/TolaActivity#2820, refactured profile page code, updat…
Oct 17, 2022
ec52fb1
Remove (empty) from migrations (#714)
kbarkemeyer Oct 18, 2022
bdd953e
Fixed validation for required fields null values and updated start an…
Oct 18, 2022
091e7bf
Fixed translations on program admin lite history log
blakelong Oct 18, 2022
41b19f2
Fixed translations on program admin lite history log
blakelong Oct 19, 2022
43bc01e
updated deps for bootstrap-multiselect
axoplasm Oct 19, 2022
461b9b6
Update translations.
Oct 19, 2022
517d205
Update react multiselect to give invalid fields a red border and appl…
Oct 19, 2022
ad37cf0
Merge pull request #713 from mercycorps/2820-editable-program-admin-l…
andrethomas6 Oct 19, 2022
f492995
Fixed formatting for super admin updated dates email
blakelong Oct 20, 2022
92ea593
Removed React Datepicker and added old datepicker for start and end d…
Oct 20, 2022
6b1f9dc
Merge pull request #717 from mercycorps/2820-editable-program-admin-l…
andrethomas6 Oct 21, 2022
ac93173
Update add program button conditional logic.
Oct 21, 2022
316c64f
Merge pull request #718 from mercycorps/2820-editable-program-admin-l…
andrethomas6 Oct 21, 2022
1181082
Added a function to alpa sort the selected options for country, outco…
Oct 21, 2022
70fbad8
Added dev2 to editable list.
Oct 21, 2022
c16b25a
Update editable logic
Oct 21, 2022
53259a1
Merge pull request #719 from mercycorps/2820-editable-program-admin-l…
andrethomas6 Oct 21, 2022
227dd84
Update datepicker min max dates.
Oct 21, 2022
f7a9fdc
Merge pull request #720 from mercycorps/2820-editable-program-admin-l…
andrethomas6 Oct 21, 2022
12568e5
Fixed formatting for super admin updated dates email
blakelong Oct 24, 2022
4c0eb9f
Fixed duplicate gait id error handling to prevent submission.
Oct 24, 2022
c50f433
Update addErrorMessage to include making form invalid.
Oct 24, 2022
a077431
Merge pull request #721 from mercycorps/2820-editable-program-admin-l…
andrethomas6 Oct 24, 2022
300daf2
Update to fix translating funding status.
Oct 25, 2022
47a2051
Merge pull request #722 from mercycorps/2820-editable-program-admin-l…
andrethomas6 Oct 25, 2022
e6adfa7
Update README (#723)
kbarkemeyer Oct 27, 2022
f5c70d7
Moved email notification outside discrepancy report scope
blakelong Nov 1, 2022
21504cb
Moved email notification outside discrepancy report scope
blakelong Nov 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added new tab to discrepancy report for duplicate IDAA programs
  • Loading branch information
blakelong committed Aug 18, 2022
commit a8d47e734e5acee8a96a2cb69c691ed113baf9e1
60 changes: 56 additions & 4 deletions workflow/discrepancy_report.py
Original file line number Diff line number Diff line change
@@ -201,6 +201,46 @@ def populate(self, idaa_json, tola_program):
]


class DuplicateIDAAProgramsTab(DiscrepancyReportTab):
columns = [
"IDAA Program Name", "IDAA Program ID", "IDAA Gait IDs", "IDAA Countries", "IDAA Start Date",
"IDAA End Date", "IDAA Program Status", "Notes"
]
title = "Duplicate IDAA Programs"
discrepancies = ["duplicate_gaitid"]

def format_worksheet(self, worksheet, wide_cells=None, small_cells=None):
wide_cells = [self.columns.index('IDAA Program Name')]
small_cells = [self.columns.index('IDAA Program ID')]
return super().format_worksheet(worksheet, wide_cells, small_cells)

def sort_by_gaitids(self, programs):
"""
Takes a list of programs formatted from super().populate()
Sorts the programs to have duplicates show as the next list item
"""
sorted_programs = []
checked_ids = []
gaitid_key = self.columns.index('IDAA Gait IDs')
id_key = self.columns.index('IDAA Program ID')
for program in programs:
gaitids = [int(gaitid) for gaitid in program[gaitid_key].split(',')]
if program[id_key] in checked_ids:
continue
checked_ids.append(program[1])
sorted_programs.append(program)
for gaitid in gaitids:
for other_program in programs:
if other_program[id_key] in checked_ids:
continue
other_gaitids = [int(gaitid) for gaitid in other_program[gaitid_key].split(',')]
if gaitid in other_gaitids:
sorted_programs.append(other_program)
checked_ids.append(other_program[1])

return sorted_programs


class OverviewTab(DiscrepancyReportTab):
columns = range(2) # columns as a range to work with inherited formating methods
wide_cell_width = 120
@@ -209,11 +249,12 @@ class OverviewTab(DiscrepancyReportTab):
{
"header": "ORIENTATION",
"body": [
"The Discrepancy Report is divided into 4 tabs:",
"The Discrepancy Report is divided into 5 tabs:",
"The first tab, titled Discrepancy Report Overview, provides explanation and instruction on what this Discrepancy Report is and how to use this to improve program data quality and consistency in TolaData ( and IDAA)."
"The second tab, titled IDAA Program to Multiple TolaData, identifies duplicated programs in TolaData, where there is one program in IDAA but this single program is broken out into multiple programs in TolaData. This often happens with multi-country programs, where each individual country decided to make their own version of the program in TolaData.",
"The third tab, titled MisMatching Fields, identifies existing programs in TolaData whose countries, dates and/or funding status do not match the data and information in IDAA. These discrepancies are not automatically corrected by the system given their sensitive nature. As a result, these discrepancies must be dealt with manually on a case-by-case basis.",
"The fourth and last tab, titled IDAA Program Has Missing Data, identifies programs in IDAA, which cannot be added or updated in TolaData due to data quality issues. These issues need to be addressed directly in IDAA before these programs can be added or updated in TolaData."
"The fourth tab, titled IDAA Program Has Missing Data, identifies programs in IDAA, which cannot be added or updated in TolaData due to data quality issues. These issues need to be addressed directly in IDAA before these programs can be added or updated in TolaData.",
"The fifth tab, titled Duplicate IDAA Programs, identifies programs in IDAA that have the same GAIT ID(s) assigned to them. A GAIT ID should only ever be assigned to one program. A single program may have multiple GAIT IDs associated to it, but a single GAIT ID should never be assigned to multiple programs. This check also attempts to address or prevent two possible undesireable scenarios: 1) many IDAA programs to one TolaData program and 2) many IDAA programs to many TolaData programs."
]
},
{
@@ -255,7 +296,7 @@ def populate_static(self, worksheet):
class GenerateDiscrepancyReport:
file_path = path.join(path.dirname(path.abspath(__file__)), f'discrepancy_report_{date.today().isoformat()}.xlsx')
_worksheet_mapper = [
OverviewTab, MultipleProgramsTab, MismatchingFieldsTab, IDAAInvalidFieldsTab
OverviewTab, MultipleProgramsTab, MismatchingFieldsTab, IDAAInvalidFieldsTab, DuplicateIDAAProgramsTab
]
discrepancy_highlight = '40 % - Accent2'
discrepancy_border = Border(left=Side(style='thin', color='fff00000'),
@@ -322,9 +363,12 @@ def generate(self):
"""
Method to generate the discrepancy report
"""
duplicated_idaa_programs = []
program_discrepancies = ProgramDiscrepancy.objects.all()
overview_tab = OverviewTab()
overview_tab.populate_static(self.wb[overview_tab.title])
duplicate_idaa_tab = DuplicateIDAAProgramsTab()
duplicate_idaa_worksheet = self.wb[duplicate_idaa_tab.title]

for program_discrepancy in program_discrepancies:
for worksheet_object in self._worksheet_mapper:
@@ -340,11 +384,14 @@ def generate(self):

worksheet_tab = worksheet_object()

if worksheet_object is MultipleProgramsTab:
if isinstance(worksheet_tab, MultipleProgramsTab):
for tola_program in program_discrepancy.program.all():
row = worksheet_tab.populate(program_discrepancy.idaa_json, tola_program)

worksheet.append(row)
elif isinstance(worksheet_tab, DuplicateIDAAProgramsTab):
row = worksheet_tab.populate(program_discrepancy, worksheet_discrepancies)
duplicated_idaa_programs.append(row)
else:
row = worksheet_tab.populate(program_discrepancy, worksheet_discrepancies)

@@ -354,4 +401,9 @@ def generate(self):

worksheet_tab.after_populate_format(worksheet)

rows = duplicate_idaa_tab.sort_by_gaitids(duplicated_idaa_programs)

for row in rows:
duplicate_idaa_worksheet.append(row)

self.wb.save(self.file_path)
5 changes: 3 additions & 2 deletions workflow/management/commands/upload_IDAA_programs.py
Original file line number Diff line number Diff line change
@@ -38,7 +38,7 @@ def handle(self, *args, **options):
msr_gaitid_list = AccessMSR().gaitid_list()

# Logs any duplicated gaitids
check_IDAA_duplicates(idaa_programs)
duplicated_gaitids = check_IDAA_duplicates(idaa_programs)

counts = {
'created': 0,
@@ -49,8 +49,9 @@ def handle(self, *args, **options):

for program in idaa_programs:
upload_program = ProgramUpload(
program['fields'], msr_country_codes_list=msr_country_codes_list, msr_gaitid_list=msr_gaitid_list
program['fields'], msr_country_codes_list=msr_country_codes_list, msr_gaitid_list=msr_gaitid_list, duplicated_gaitids=duplicated_gaitids
)

if upload_program.is_valid():
if options['upload']:
upload_program.upload()
1 change: 1 addition & 0 deletions workflow/models.py
Original file line number Diff line number Diff line change
@@ -801,6 +801,7 @@ class ProgramDiscrepancy(models.Model):
"countries": "TolaData program countries does not match IDAA Country",
"multiple_programs": "Multiple TolaData programs retrieved from IDAA program",
"gaitid": "IDAA program has invalid Gait ID",
"duplicate_gaitid": "IDAA program has a duplicated gaitid",
"ProgramName": "IDAA program is missing ProgramName",
"id": "IDAA program is missing ID",
"ProgramStartDate": "IDAA program is missing ProgramStartDate",
32 changes: 28 additions & 4 deletions workflow/program.py
Original file line number Diff line number Diff line change
@@ -158,13 +158,15 @@ def create_discrepancies(self):
class ProgramValidation(ProgramDiscrepancies):
funded_str = 'Funded'

def __init__(self, idaa_program, msr_country_codes_list, msr_gaitid_list):
def __init__(self, idaa_program, msr_country_codes_list, msr_gaitid_list, duplicated_gaitids):
self.idaa_program = idaa_program
self._validated = False

self.msr_country_codes_list = msr_country_codes_list
self.msr_gaitid_list = msr_gaitid_list

self.duplicated_gaitids = [clean_idaa_gaitid(gaitid) for gaitid in duplicated_gaitids]

super().__init__()

@property
@@ -246,6 +248,20 @@ def valid_gaitids(self):

return valid

def has_duplicated_gaitids(self):
"""
Returns True if the program has a gaitid shared with another IDAA program else False
"""
has_duplicates = False
gaitids = self.compressed_idaa_gaitids()

for gaitid in gaitids:
if gaitid in self.duplicated_gaitids:
self.add_discrepancy('duplicate_gaitid')
has_duplicates = True

return has_duplicates

def missing_fields(self):
"""
Checks that idaa_program is not missing any fields
@@ -275,9 +291,13 @@ def valid_idaa_program(self):
return False

valid_gaitids = self.valid_gaitids()
# Check duplicated gaitids
if valid_gaitids:
has_duplicates = self.has_duplicated_gaitids()

matching_countries = self.matching_countries()

return not missing_fields and valid_gaitids and matching_countries
return not missing_fields and valid_gaitids and matching_countries and not has_duplicates

def matching_countries(self):
"""
@@ -356,10 +376,12 @@ def is_valid(self):
# There is a case when a non-funded program can already have discrepancies at this point.
# Clear all discrepancies just in case.
self.clear_discrepancies()
self.has_duplicated_gaitids()
return False

# These discrepancies can come up while trying to retrieve the Tola program
if self.has_discrepancy('multiple_programs') or self.has_discrepancy('gaitid'):
self.has_duplicated_gaitids()
return False

valid_idaa_program = self.valid_idaa_program()
@@ -379,13 +401,15 @@ def is_valid(self):

class ProgramUpload(ProgramValidation):

def __init__(self, idaa_program, msr_country_codes_list, msr_gaitid_list):
def __init__(self, idaa_program, msr_country_codes_list, msr_gaitid_list, duplicated_gaitids=None):
self.idaa_program = idaa_program

self.msr_country_codes_list = msr_country_codes_list
self.msr_gaitid_list = msr_gaitid_list

super().__init__(idaa_program, msr_country_codes_list=msr_country_codes_list, msr_gaitid_list=msr_gaitid_list)
self.duplicated_gaitids = duplicated_gaitids

super().__init__(idaa_program, msr_country_codes_list=msr_country_codes_list, msr_gaitid_list=msr_gaitid_list, duplicated_gaitids=duplicated_gaitids)

self.tola_program = self.get_tola_programs()
self.program_updated = False
31 changes: 31 additions & 0 deletions workflow/tests/idaa_sample_data/idaa_invalid_sample.json
Original file line number Diff line number Diff line change
@@ -120,6 +120,37 @@
"ProgramStartDate": "2022-05-01T01:00:00Z",
"ProgramEndDate": "2023-05-01T01:00:00Z"
}
},
{
"fields": {
"id": 8,
"ProgramName": "Test Duplicate IDAA Progam",
"GaitIDs": [
{"LookupValue": "1111.0000000000"}
],
"ProgramStatus": "Funded",
"Country": [
{"LookupValue": "Colombia"}
],
"ProgramStartDate": "2022-05-01T01:00:00Z",
"ProgramEndDate": "2023-05-01T01:00:00Z"
}
},
{
"fields": {
"id": 9,
"ProgramName": "Test Duplicate IDAA Progam2",
"GaitIDs": [
{"LookupValue": "1111.0000000000"},
{"LookupValue": "1222.0000000000"}
],
"ProgramStatus": "Funded",
"Country": [
{"LookupValue": "Colombia"}
],
"ProgramStartDate": "2022-05-01T01:00:00Z",
"ProgramEndDate": "2023-05-01T01:00:00Z"
}
}
]
}
18 changes: 18 additions & 0 deletions workflow/tests/idaa_sample_data/idaa_sample.json
Original file line number Diff line number Diff line change
@@ -98,6 +98,24 @@
"_x0032_030OutcomeTheme": ["new", "program"],
"Sector": []
}
},
{
"fields": {
"id": 1525,
"ProgramName": "Program Create Test ",
"GaitIDs": [
{"LookupValue": "99999.0000000000"}
],
"ProgramStatus": "Funded",
"Country": [
{"LookupValue": "HQ"}
],
"Region": "HQ Managed",
"ProgramStartDate": "2023-04-29T07:00:00Z",
"ProgramEndDate": "2025-09-29T07:00:00Z",
"_x0032_030OutcomeTheme": ["new", "program"],
"Sector": []
}
}
]
}
8 changes: 5 additions & 3 deletions workflow/tests/test_discrepancy_report.py
Original file line number Diff line number Diff line change
@@ -21,6 +21,7 @@ class TestDiscrepancyReport(test.TestCase):
idaa_sample_data_path = 'workflow/tests/idaa_sample_data/idaa_invalid_sample.json'
discrepancy_report_path = f'workflow/discrepancy_report_{date.today().isoformat()}.xlsx'
idaa_json = None
duplicated_gaitids = ["1111.0000000000"]

def setUp(self):
with open(self.idaa_sample_data_path) as file:
@@ -71,7 +72,7 @@ def setUp(self):
def test_discrepancy_report(self):
for idaa_program in self.idaa_json['value']:
upload_program = program.ProgramUpload(idaa_program=idaa_program['fields'],
msr_country_codes_list=msr_country_codes_list, msr_gaitid_list=msr_gaitid_list
msr_country_codes_list=msr_country_codes_list, msr_gaitid_list=msr_gaitid_list, duplicated_gaitids=self.duplicated_gaitids
)

self.assertFalse(upload_program.is_valid())
@@ -86,10 +87,11 @@ def test_discrepancy_report(self):

def assert_workbook(self, workbook):
worksheets = [
{'class': discrepancy_report.OverviewTab, 'expected_rows': 16, 'id_col': None, 'expected_ids': None},
{'class': discrepancy_report.OverviewTab, 'expected_rows': 18, 'id_col': None, 'expected_ids': None},
{'class': discrepancy_report.MultipleProgramsTab, 'expected_rows': 3, 'id_col': 'F', 'expected_ids': [3, 3]},
{'class': discrepancy_report.MismatchingFieldsTab, 'expected_rows': 3, 'id_col': 'K', 'expected_ids': [2, 4]},
{'class': discrepancy_report.IDAAInvalidFieldsTab, 'expected_rows': 4, 'id_col': 'C', 'expected_ids': [1, 5, 6]}
{'class': discrepancy_report.IDAAInvalidFieldsTab, 'expected_rows': 4, 'id_col': 'C', 'expected_ids': [1, 5, 6]},
{'class': discrepancy_report.DuplicateIDAAProgramsTab, 'expected_rows': 3, 'id_col': 'B', 'expected_ids': [8, 9]}
]

self.assertEquals(len(worksheets), len(workbook.worksheets))
Loading