Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spark2kafka process blank line #1227

Merged
merged 10 commits into from
Dec 15, 2022
Merged

Conversation

wycccccc
Copy link
Collaborator

@wycccccc wycccccc commented Dec 8, 2022

在做測試的時候同樣遇到了blank line的問題。
想確認一下是默認將blankline處理掉,還是要新增一個參數讓用戶決定blankline的合法性

@wycccccc wycccccc requested a review from chia7712 December 8, 2022 08:46
@chia7712
Copy link
Contributor

chia7712 commented Dec 8, 2022

想確認一下是默認將blankline處理掉,還是要新增一個參數讓用戶決定blankline的合法性

一樣預設跳過,但也要有參數可以改成拋出錯誤

@wycccccc
Copy link
Collaborator Author

wycccccc commented Dec 9, 2022

爲了應對以後可能增加的command line,我將scala的command line加了回來。
同時完善了處理blankline的邏輯。

@wycccccc
Copy link
Collaborator Author

現在改用最單純的方式處理處理blankline,並且移除了對pk non-nullable的判斷

@chia7712
Copy link
Contributor

@wycccccc 整體看起來還ok,可否麻煩你手動驗證一下空白行的部分?如果沒問題的話就可以合併

@wycccccc
Copy link
Collaborator Author

整體看起來還ok,可否麻煩你手動驗證一下空白行的部分?如果沒問題的話就可以合併

沒有問題,它有正確忽略

@wycccccc wycccccc merged commit cfa7c5b into opensource4you:main Dec 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants