Create resources:
- A GCP project
- A GCS bucket
- 2 BigQuery datasets: feast (conventionally for Feast temp tables), another dataset for project data tables
Set the envs
GOOGLE_CLOUD_PROJECT
BUCKET_NAME
BIGQUERY_DATASET_NAME
- Source table:
BigQuery Data Viewer
- Feast dataset:
roles/bigquery.dataEditor
- Firestore:
Cloud Datastore User
- BigQuery project-level:
bigquery.jobs.create
,bigquery.readsessions.create
,bigquery.readsessions.getData
- BigQuery Feast dataset:
roles/bigquery.dataEditor
- BigQuery Source table:
BigQuery Data Viewer
- BigQuery project-level:
bigquery.jobs.create
,bigquery.readsessions.create
,bigquery.readsessions.getData
- BigQuery Source table:
BigQuery Data Viewer
- Firestore:
Cloud Datastore User
- Firestore:
Cloud Datastore User
- GCS:
storage.bucket.get
,storage.objects.*