Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Figures for Bingling's paper on Healthcare Workforce #436

Draft
wants to merge 145 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
145 commits
Select commit Hold shift + click to select a range
dd256ca
establish files and folder structure
tbhallett Dec 9, 2021
db526b4
renaming for brevity
tbhallett Dec 9, 2021
39813df
renaming for brevity
tbhallett Dec 9, 2021
18ab263
small pop size
tbhallett Dec 9, 2021
0ad2472
change file name for testing
tbhallett Dec 9, 2021
5c2d723
restore definition of scenario_hsi_in_typical_run.py
tbhallett Dec 9, 2021
397fa3f
produce skeleton script
tbhallett Dec 9, 2021
2812358
Create the .py that generate daily capabilities and staff allocation …
BinglingICL Dec 9, 2021
62b9047
Refactor
BinglingICL Dec 9, 2021
daa0e51
Refactor as much as I could
BinglingICL Dec 9, 2021
6e21721
reset date and pop
BinglingICL Dec 10, 2021
3270423
refactor
BinglingICL Dec 10, 2021
7dc9127
Update the file plotting health system daily capabilities
BinglingICL Dec 10, 2021
dfc77da
Update the file plotting health system daily capabilities
BinglingICL Dec 10, 2021
de5cf14
Reset date and pop
BinglingICL Dec 10, 2021
b7e6aef
Merge remote-tracking branch 'origin/hallett/figure-to-describe-hsi' …
BinglingICL Dec 10, 2021
9d718cb
The jupyter notebook that generate sankey diagrams mapping coarse off…
BinglingICL Dec 13, 2021
66879cb
Update annotation
BinglingICL Dec 13, 2021
1c68853
Update annotation
BinglingICL Dec 13, 2021
1db1057
Revert "Update annotation"
BinglingICL Dec 13, 2021
a65bec1
Refactor
BinglingICL Dec 13, 2021
f3373a9
Refactor
BinglingICL Dec 13, 2021
483d130
Updating existing plots
BinglingICL Dec 13, 2021
19c8b08
Update annotation
BinglingICL Dec 14, 2021
dab2f57
Update annotation
BinglingICL Dec 14, 2021
1bd78e1
Refactor
BinglingICL Dec 14, 2021
25e7440
Great a file to plot the sankey diagram that maps appointment type to…
BinglingICL Dec 14, 2021
0902f0a
Refactor
BinglingICL Dec 14, 2021
4319a79
Questions and todo
BinglingICL Dec 15, 2021
258421b
Questions and todo
BinglingICL Dec 15, 2021
efe9e75
Questions and todo deleted
BinglingICL Dec 15, 2021
8f5a3ee
Two sankeys mapping appt and hsi
BinglingICL Dec 15, 2021
b59d380
Raise an issue and todo
BinglingICL Dec 15, 2021
19aaf5b
Rename weird modules
BinglingICL Dec 15, 2021
649375f
Reorder the nodes to make clear sankey
BinglingICL Dec 16, 2021
48272f5
Todo: update input data and reset node order
BinglingICL Dec 16, 2021
50134d0
Fix the issue of missing HSI events
BinglingICL Dec 16, 2021
921336b
Correct a typo for a treatment_id in Joe's module
BinglingICL Dec 16, 2021
ab8923c
Do nor rename modules
BinglingICL Dec 16, 2021
eed1444
Refactor
BinglingICL Dec 16, 2021
bc100b3
Refactor
BinglingICL Dec 16, 2021
81def58
Note down a possible issue
BinglingICL Dec 17, 2021
6a78703
Refactor
BinglingICL Dec 18, 2021
57f6119
Fix nodes order in the sankey
BinglingICL Dec 19, 2021
cf51d93
Embark the update of healthsystem data
BinglingICL Feb 10, 2022
b19ea34
Generate a new file to process the new CHAI data
BinglingICL Feb 14, 2022
a8b80e9
Update to line 878: same 21 cadres; CHAI establishment staff has data…
BinglingICL Feb 15, 2022
d6a7461
Update to line 1035: CHAI current staff has no data for Likoma, there…
BinglingICL Feb 15, 2022
5dc1cca
Update to line 1376: mfl is the same; not consider DHO service time a…
BinglingICL Feb 15, 2022
f5fdd0d
Update to line 1511: PFT table is different -- facility types include…
BinglingICL Feb 16, 2022
9865760
Update to line 1696: because of the change of PFT and staff counts, t…
BinglingICL Feb 16, 2022
51eda1a
Update to the end: because of missing PFT data for some cadres, the f…
BinglingICL Feb 16, 2022
0815e12
Minor modification
BinglingICL Feb 16, 2022
80d2e0d
Correct typo and propose a fix to the issue of PFT
BinglingICL Feb 16, 2022
025c43e
Try fixing the missing entries in PFT table
BinglingICL Feb 17, 2022
fd177f8
Try fixing the issue of zero DCSA staff in Likoma
BinglingICL Feb 17, 2022
d79c725
Correcting ZMH service types and time requirements
BinglingICL Feb 17, 2022
b349b2d
Trying to fix inconsistency of mental health service demand and staff…
BinglingICL Feb 17, 2022
a634a0c
Move mental health staff to districts with no such staff from their r…
BinglingICL Feb 18, 2022
490a598
Check difference of new and previous PFT
BinglingICL Feb 18, 2022
c79fa43
Minor format to be consistent with original formatting_healthsystem_d…
BinglingICL Feb 18, 2022
9d5dcb9
Move old data path to the beginning and notice the need to upload the…
BinglingICL Feb 18, 2022
a64e796
Update the input and output path (need upload the CHAI data to dropbox)
BinglingICL Feb 21, 2022
6acad5e
Refactor; ready for review
BinglingICL Feb 21, 2022
90896f0
Overwrite the old formatting_healthsystem_data file
BinglingICL Feb 21, 2022
eac331e
Update the simulation period of scenario hsi
BinglingICL Feb 21, 2022
458bee4
Refactor (and see if all tests pass)
BinglingICL Feb 21, 2022
72dd1bb
Update the simulation period of scenario hsi
BinglingICL Feb 21, 2022
f4ab441
Update the simulation period of scenario hsi
BinglingICL Feb 21, 2022
5e07607
Merge branch 'bs/update_healthsystem_data' into hallett/figure-to-des…
BinglingICL Feb 22, 2022
202be13
Merge remote-tracking branch 'origin/master' into hallett/figure-to-d…
BinglingICL Feb 23, 2022
06371e8
Update figures
BinglingICL Feb 23, 2022
7010aec
Update fontsize in figures
BinglingICL Feb 23, 2022
2e691f2
get the appointment usage data of year 2019
BinglingICL Mar 28, 2022
35f451c
add log of demography module
BinglingICL Mar 29, 2022
35463d4
update simulation end_date
BinglingICL Jun 7, 2022
9de67b2
add year column to hsi event table
BinglingICL Jun 7, 2022
ae057a4
Re-run jupter notebook to check sankey diagram; need to re-generate o…
BinglingICL Nov 8, 2022
49b434f
change scenario_filename
BinglingICL Nov 21, 2022
340d0d8
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Jun 9, 2023
1cd7594
add 'PharmDispensing' to the diagrams
BinglingICL Jun 9, 2023
3eef04f
compare actual and funded plus scenarios
BinglingICL Jun 22, 2023
4b45c9e
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Jul 13, 2023
311db15
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Jul 14, 2023
81071ca
create the scale run script to get most recent simulation results
BinglingICL Jul 14, 2023
750778b
minor update
BinglingICL Jul 14, 2023
d9d7570
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Jul 17, 2023
cffd1ea
Revert "Merge HR resources and Consumables at facility levels 1b and …
BinglingICL Jul 17, 2023
51b820a
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Jul 18, 2023
14f418b
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Jul 21, 2023
5f73293
revert the changes of merging levels 1b and 2
BinglingICL Jul 21, 2023
d193348
trim rel diff (for new plots) to [0.1, 10.0]
BinglingICL Jul 21, 2023
4e7eae2
Revert "trim rel diff (for new plots) to [0.1, 10.0]"
BinglingICL Jul 21, 2023
25162a4
add todo for HCW paper plots
BinglingICL Jul 25, 2023
648e223
create a separate file for HCW paper plots
BinglingICL Jul 26, 2023
6b2e505
Revert "add todo for HCW paper plots"
BinglingICL Jul 26, 2023
192242e
formate hcw usage
BinglingICL Jul 26, 2023
6822950
further format the hcw usage
BinglingICL Jul 26, 2023
d71d4a3
try get hsi count by appt type and facility level
BinglingICL Jul 26, 2023
4c93e17
get hsi count by treatment id and appt type and facility level
BinglingICL Jul 26, 2023
a3e2419
omit level info (as hsi calibration on faciliy level is not yet donw)…
BinglingICL Jul 28, 2023
46c251b
plot simulated hcw time usage per cadre
BinglingICL Jul 29, 2023
8e27897
minor update
BinglingICL Jul 29, 2023
0025b1a
minor but crucial update
BinglingICL Jul 29, 2023
53fb641
fix the hsi count issue: should calculate average annual count
BinglingICL Jul 31, 2023
872555c
reorganise the script
BinglingICL Jul 31, 2023
5ebd88f
delete experimental coding
BinglingICL Jul 31, 2023
6dae067
comment on todo task of drawing plot for DHIS2 data
BinglingICL Jul 31, 2023
7033136
currently delete the todo task of drawing the plots for DHIS2 data
BinglingICL Jul 31, 2023
f8b106b
rename Nursing_and_Midwifery
BinglingICL Jul 31, 2023
133c71c
rename Nursing_and_Midwifery
BinglingICL Jul 31, 2023
883e2c3
plot hcw sankey flow via appt to module
BinglingICL Jul 31, 2023
fb2f9f0
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Jul 31, 2023
6bfacf1
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Aug 1, 2023
907d53a
fix failed checks
BinglingICL Aug 1, 2023
234df05
fix failed checks
BinglingICL Aug 2, 2023
6052f9b
temporary change for HCW paper
BinglingICL Aug 3, 2023
f7d6e3f
temporary change for HCW paper
BinglingICL Aug 3, 2023
4a65060
temporary change for HCW paper
BinglingICL Aug 3, 2023
c2390ac
plot stacked bar for total appts usage of Model vs Data
BinglingICL Aug 4, 2023
28088bf
modify time for dummy ConWithDCSA so that no overworking/underworking
BinglingICL Aug 4, 2023
ce2dd3f
drop dummy PharmDispensing for HCW paper results and plots
BinglingICL Aug 5, 2023
d295657
update plot title
BinglingICL Aug 5, 2023
4a49c7e
increase height for maximal health care seeking sankey
BinglingICL Aug 5, 2023
abdd680
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Aug 16, 2023
e81a163
adjust figure size to fix margin issues of the long sankey
BinglingICL Aug 16, 2023
7f708a8
fix failing checks
BinglingICL Aug 16, 2023
681dd72
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Aug 18, 2023
a954c6a
save data for sankey diagram of appt to hsi/treatment id
BinglingICL Aug 18, 2023
ea6a800
draft to plot sankey diagram from appt to hsi via facility level
BinglingICL Aug 18, 2023
28dd29f
plot sankey diagram from appt to hsi via facility level
BinglingICL Aug 18, 2023
32f0f74
fix failed check
BinglingICL Aug 18, 2023
7495e37
Merge branch 'master' into hallett/figure-to-describe-hsi
BinglingICL Aug 19, 2023
b249520
redo the merge of level 1b and level 2 in healthsystem module
BinglingICL Aug 19, 2023
889bdf5
undo the modification of test_healthsystem considering the merge of l…
BinglingICL Aug 20, 2023
75bc473
more sankeys for illustrating HSI and appt
BinglingICL Aug 21, 2023
65441c7
update sankey
BinglingICL Aug 23, 2023
3a8df13
update sankey
BinglingICL Aug 23, 2023
a26bab2
fix typo
BinglingICL Aug 23, 2023
32d317d
do not adjust Data MentalAll usage, considering very low reporting ra…
BinglingICL Aug 24, 2023
72b081a
update y scale for staffing plots
BinglingICL Aug 31, 2023
ad2343b
update figure according to BMC HRH review
BinglingICL Jan 10, 2024
7efa1cb
try update figure according to BMC HRH review
BinglingICL Jan 11, 2024
b31b216
try update figure according to BMC HRH review
BinglingICL Jan 15, 2024
1829ec6
todo: drop facility level 5 and all relevant according to BMC HRH review
BinglingICL Feb 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,53 +1,158 @@
"""
This file produces a nice plot of the capabilities of the healthsystem in terms of the hours available for
different cadres of healthcare workers.
This file produces histograms of the healthsystem capabilities \
in terms of staff allocation and daily capabilities in minutes per cadre per facility level.
"""

# %%

from pathlib import Path

import pandas as pd
from matplotlib import pyplot as plt
from matplotlib.ticker import ScalarFormatter

# Get the path of the folder that stores the data - three scenarios: actual, funded, funded_plus
workingpath = Path('./resources/healthsystem/human_resources/funded_plus')

# Define the path of output histograms - three scenarios: actual, funded, funded_plus
outputpath = Path('./outputs/healthsystem/human_resources/funded_plus')

# Read data
data = pd.read_csv(workingpath / 'ResourceFile_Daily_Capabilities.csv')

resourcefilepath = Path("./resources")

# %%
# MINUTES PER HEALTH OFFICER CATEGORY BY DISTRICT
data_districts = data.dropna(inplace=False)
dat = pd.DataFrame(data_districts.groupby(['District', 'Officer_Category'], as_index=False)['Total_Mins_Per_Day'].sum())
tab = dat.pivot(index='District', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True)
plt.ylabel('Minutes per day')
plt.xlabel('District')

outputpath = Path("./outputs") # folder for convenience of storing outputs
ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'health_officer_minutes_per_district.pdf', bbox_inches='tight')

data = pd.read_csv(
Path(resourcefilepath) / "ResourceFile_Daily_Capabilities.csv"
)
# STAFF COUNTS PER HEALTH OFFICER CATEGORY BY DISTRICT
data_districts = data.dropna(inplace=False)
dat = pd.DataFrame(data_districts.groupby(['District', 'Officer_Category'], as_index=False)['Staff_Count'].sum())
tab = dat.pivot(index='District', columns='Officer_Category', values='Staff_Count')
ax = tab.plot.bar(stacked=True)
plt.ylabel('Staff counts')
plt.xlabel('District')

# [['Total_Minutes_Per_Day','Officer_Type','District']]
ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

data = data.dropna()
# data['District'] = data['District'].fillna('National')
plt.savefig(outputpath / 'staff_allocation_per_district.pdf', bbox_inches='tight')

# do some re-grouping to make a more manageable number of health cadres:
data['Officer_Type'] = data['Officer_Type'].replace('DCSA', 'CHW')
data['Officer_Type'] = data['Officer_Type'].replace(['Lab Officer', 'Lab Technician', 'Lab Assistant'], 'Lab Support')
data['Officer_Type'] = data['Officer_Type'].replace(['Radiographer', 'Radiography Technician'], 'Radiography')
data['Officer_Type'] = data['Officer_Type'].replace(['Nurse Officer', 'Nutrition Staff', 'Med. Assistant'], 'Nurse')
data['Officer_Type'] = data['Officer_Type'].replace('Nurse Midwife Technician', 'MidWife')
data['Officer_Type'] = data['Officer_Type'].replace(['Pharmacist', 'Pharm Technician', 'Pharm Assistant'], 'Pharmacy')
data['Officer_Type'] = data['Officer_Type'].replace(['Medical Officer / Specialist', 'Clinical Officer / Technician'],
'Clinician')
data['Officer_Type'] = data['Officer_Type'].replace(['Dental Therapist'], 'Dentist')

# MINUTES PER HEALTH OFFICER TYPE BY DISTRICT:
dat = pd.DataFrame(data.groupby(['District', 'Officer_Type'], as_index=False)['Total_Minutes_Per_Day'].sum())
tab = dat.pivot(index='District', columns='Officer_Type', values='Total_Minutes_Per_Day')
# MINUTES PER HEALTH OFFICER CATEGORY BY LEVEL
dat = pd.DataFrame(data.groupby(['Facility_Level', 'Officer_Category'], as_index=False)['Total_Mins_Per_Day'].sum())
tab = dat.pivot(index='Facility_Level', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True)
# ax = tab.plot.bar(stacked=True, log=True)
plt.ylabel('Minutes per day')
plt.xlabel('Facility level')

ax.tick_params(axis='x', rotation=0)

formatter = ScalarFormatter()
formatter.set_scientific(False)
ax.yaxis.set_major_formatter(formatter)

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'health_officer_minutes_per_level.pdf', bbox_inches='tight')

# STAFF COUNTS PER HEALTH OFFICER CATEGORY BY LEVEL
dat = pd.DataFrame(data.groupby(['Facility_Level', 'Officer_Category'], as_index=False)['Staff_Count'].sum())
tab = dat.pivot(index='Facility_Level', columns='Officer_Category', values='Staff_Count')
ax = tab.plot.bar(stacked=True)
# ax = tab.plot.bar(stacked=True, log=True)
plt.ylabel('Staff counts')
plt.xlabel('Facility level')

ax.tick_params(axis='x', rotation=0)

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'staff_allocation_per_level.pdf', bbox_inches='tight')


# MINUTES PER HEALTH OFFICER CATEGORY BY LEVEL

# Level 0
data_level = data.loc[data['Facility_Level'] == '0', :]
tab = data_level.pivot(index='District', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True)
plt.ylabel('Minutes per day at level 0')
plt.xlabel('District')

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'health_officer_minutes_per_district.pdf', bbox_inches='tight')
plt.show()
plt.savefig(outputpath / 'health_officer_minutes_per_district_level_0.pdf', bbox_inches='tight')

# Level 1a
data_level = data.loc[data['Facility_Level'] == '1a', :]
tab = data_level.pivot(index='District', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True)
plt.ylabel('Minutes per day at level 1a')
plt.xlabel('District')

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'health_officer_minutes_per_district_level_1a.pdf', bbox_inches='tight')

# Level 1b
data_level = data.loc[data['Facility_Level'] == '1b', :]
tab = data_level.pivot(index='District', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True)
plt.ylabel('Minutes per day at level 1b')
plt.xlabel('District')

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'health_officer_minutes_per_district_level_1b.pdf', bbox_inches='tight')

# Level 2
data_level = data.loc[data['Facility_Level'] == '2', :]
tab = data_level.pivot(index='District', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True)
plt.ylabel('Minutes per day at level 2')
plt.xlabel('District')

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'health_officer_minutes_per_district_level_2.pdf', bbox_inches='tight')

# Level 3
data_level = data.loc[data['Facility_Level'] == '3', :]
tab = data_level.pivot(index='Region', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True)
plt.ylabel('Minutes per day at level 3')
plt.xlabel('Regional Referral Hospital')
ax.tick_params(axis='x', rotation=0)

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

plt.savefig(outputpath / 'health_officer_minutes_per_district_level_3.pdf', bbox_inches='tight')

# Level 4
data_level = data.loc[data['Facility_Level'] == '4', :]
tab = data_level.pivot(index='Facility_Name', columns='Officer_Category', values='Total_Mins_Per_Day')
ax = tab.plot.bar(stacked=True, width=0.1)
plt.ylabel('Minutes per day at level 4')
plt.xlabel('National resource hospital')
ax.tick_params(axis='x', rotation=0)

ax.legend(ncol=3, bbox_to_anchor=(0, 1),
loc='lower left', fontsize='small')

# %%
plt.savefig(outputpath / 'health_officer_minutes_per_district_level_4.pdf', bbox_inches='tight')
Loading