Modifying the UNITY_OUTPUT_DATA_BUCKET to recognize deployment & venue

[jpl-btlunsfo]

Purpose

• Updating the non-venue-independent references in sbg_L1_to_L2_e2e_cwl_step_by_step_dag.py

Proposed Changes

• CHANGE the UNITY_OUTPUT_DATA_BUCKET variable (path) to include deployment name, venue, and counter

Issues

• Update SBG e2e workflow to work across venues #86

Testing

• manually tested in one-off airflow installation

[jpl-btlunsfo]

Just to note, the DAG fails pretty quickly if the UNITY_OUTPUT_DATA_BUCKET ssm path doesn't exist. Should creation of that parameter also be a part of this PR?

After talking with Galen I went with the format /unity/ds/data/bucket/{deployment_name}/{venue}-{counter}/primary-data-bucket (ex /unity/ds/data/bucket/btl/test-b2d6/primary-data-bucket)

[LucaCinquini]

Hi Brad, we should probably not merge this PR until the DAG will succeed. I think this involves:

o Making sure the output data bucket exists. Who is creating this, us or DS?
o Also creating the proper SSM parameter

You can work on #2 above, I will raise the question about #1 with the team