Stop segments refactor 2

[tiffanychu90]

segment speeds

• run april date after refactoring entire workflow and note what's brittle or time-consuming
  • next step is to move the most time-consuming steps into dask client. don't move everything, unsure about long-term solution still.
• add 20th and 80th percentile speeds within segment, not just the mean
• remove old debug notebooks: debug_losing_curve and debug_stop_metrics and road_segments
• add the needed steps to get this from processed data to open data portal...add those connections in shared_utils
  • definitely want: organization_name, base64_url, caltrans_district...do we need organization_source_record_id?
  • TODO: other columns, can include, but might be too confusing for users: any versioned key (gtfs_dataset_key or name), feed_url....decoded ASCII version of base64_url, uri
  • TODO: add the natural identifiers back instead of our internal modeling of versioned identifiers (switch shape_array_key for shape_id)
• Epic - Daskify RT segment speeds #592

shared_utils

• portfolio_utils: go from a GTFS quartet dataset and get organization name Research Request - crosswalk for keys (feed_key, gtfs_dataset_key, source_record_id, name, organization_name) when aggregating #750
• portfolio_utils: remove outdated v1 functions for adding organization name and caltrans district
• gtfs_utils_v2: remove is_future column and filtering condition, that column no longer exists
• TODO: get caltrans district....which comes from dim_organizations.
• TODO: Determine what's the full set of open data portal identifiers after tomorrow's publishing meeting and just wrap it all in one function

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• rt_segment_speeds/debug-07.ipynb
• rt_segment_speeds/debug_losing_curve.ipynb
• rt_segment_speeds/road-segments.ipynb