Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Data Source specific format options with DataFormat message #1049

Merged
merged 24 commits into from
Oct 19, 2020
Merged

Replace Data Source specific format options with DataFormat message #1049

merged 24 commits into from
Oct 19, 2020

Conversation

mrzzy
Copy link
Collaborator

@mrzzy mrzzy commented Oct 13, 2020

What this PR does / why we need it:
Add data format messages to represent as data formats:

  • FileFormat will replace file_format in FileOptions
  • StreamFormat class_path` in Kafka and KinesisOptions.
  • This would allow us to support multiple data formats for each SourceType (ie AvroFormat and ProtoFormat for Kafka).
  • Added AvroFormat Protobuf message

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Add `FileFormat` to replace `file_format` in FileOptions and `StreamFormat` to replace `class_path` in Kafka and KinesisOptions.

@mrzzy mrzzy added this to the v0.8.0 milestone Oct 13, 2020
@mrzzy mrzzy changed the title Protobuf: Replace Data Source specific format options with DataFormat message Replace Data Source specific format options with DataFormat message Oct 13, 2020
@mrzzy mrzzy changed the title Replace Data Source specific format options with DataFormat message WIP: Replace Data Source specific format options with DataFormat message Oct 14, 2020
@mrzzy mrzzy changed the title WIP: Replace Data Source specific format options with DataFormat message Replace Data Source specific format options with DataFormat message Oct 14, 2020
@mrzzy
Copy link
Collaborator Author

mrzzy commented Oct 16, 2020

/hold due to Protobuf API change

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…ve objects.

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…, DataFormat specs

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…ables

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…e current API

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…ages

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
DataFormat.proto

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
messages

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…d by git

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
… instead of expected str

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mrzzy, pyalex

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pyalex
Copy link
Collaborator

pyalex commented Oct 19, 2020

/lgtm

@pyalex
Copy link
Collaborator

pyalex commented Oct 19, 2020

/unhold

@feast-ci-bot feast-ci-bot merged commit b9e6597 into feast-dev:master Oct 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants