Ingestion should fail immediately when there are no valid stores

[zhilingc]

Expected Behavior

Starting a job with invalid stores (e.g. using redis as a warehouse store) should not be allowed, and should fail quickly - ideally at the graph building step of ingestion.

Current Behavior

Starting a job with invalid stores will successfully send the job to the runner, which will run to completion (or indefinitely, in the case of streaming jobs). The errors will be logged, but the pipeline will run with no problems.

Steps to reproduce

• Register a feature with its warehouse sink pointing to a serving store (e.g. redis)
• Run a job (direct runner is the best way to view errors)
• Pipeline runs successfully, a successful response is returned to the caller

Possible Solution

PR #11 is a band-aid solution to this problem: it checks the store types at registration, ensuring that a feature is unable to use a serving store for warehousing, but ideally ingestion error out properly during graph building.

[tims]

@zhilingc I think you misunderstood the code it's not a "bandaid" :)

It does fail at graph build time if the feature references a store that is not found in the specs service.
See Specs.validate(). Am I missing something?

[tims]

Sorry I misread this issue.

The fix is in PR #15.

[zhilingc]

It's a band-aid because if someone alters the DB the job will fail since the checks are only at feature registration time.

And nice! If it works on remote flink we're golden :)

[tims]

This should stop those self imposed DoS events every 10 minutes too.

[tims]

We can close this now?

[zhilingc]

Yep. Thanks for the hard work :)