Merge pull request #1066 from cal-itp/cpb_analysis

bus procurement. cost per bus analysis update.

bus_procurement_cost/Makefile

@@ -0,0 +1,6 @@
+# runs all scripts for bus procurement cost
+all_bus_scripts:
+	python fta_data_cleaner.py
+	python tircp_data_cleaner.py
+	python cost_per_bus_cleaner.py
+	python cpb_analysis_scripts.py

bus_procurement_cost/README.md

@@ -3,7 +3,7 @@
 ## Datasets
 * FTA Bus and Low- and No-Emission Grant Awards
 * TIRCP Project List
-* (upcoming) DGS Usage Reports (via Rebel)
+* DGS Usage Reports (via Rebel)
 * (upcoming )Washington and/or Georgia Contract list (via Rebel)
 
 ## GH issue
@@ -13,14 +13,19 @@ Research Request - Bus Procurement Costs & Awards #897
 Identify federal awards to fund bus purchases and how much agencies pay for them.
 
 ## Methodology
-- Examine each dataset to determine if any columns need cleaning/validation
-- Identify projects that include bus purchases. Of which, note the following"
+- Examine each dataset to:
+    * understand table structure. Which dataset need to be aggregated or disaggregated?
+    * determine if any columns need cleaning/validation
+    * determine which columns are unnecessary
+    * understand project types (bus procurement, facility procurement, both)
+<br></br>
+- Identify projects that include bus purchases. Of which, note the following
     * Total cost of project
     * count of buses
     * propulsion type of buses (zero/non-zero emission, BEB, FCEB, CNG etc)
     * bus type (standard, cutaway, articulated etc)
 <br> </br>
-- Combine datasets together, aggregate up by transit agency, calculate a "cost_per_bus" (cpb) column.
+- Combine datasets together, aggregate up by different categories, calculate a "cost_per_bus" (cpb) column.
 - Aggregate cpb by:
     * transit agency
     * propulsion type
@@ -30,4 +35,7 @@ Identify federal awards to fund bus purchases and how much agencies pay for them
  - Calculate summary stats on cpb
      * calculate mean, standard deviation
      * calculate z-score. remove outliers
-     * plot distribution
+     * plot distribution
+ <br></br>
+ - Other
+     * review external reports/resources regarding bus procurement

bus_procurement_cost/cost_per_bus_cleaner.py

@@ -51,6 +51,7 @@ def prepare_data() ->pd.DataFrame:
             "total_cost",
             "bus_count",
             "source",
+            "new_project_type"
         ],
         how="outer",
     )