[bugfix](hive)fix error for writing to hive #34520
Conversation
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
TeamCity be ut coverage result: |
|
run buildall |
github-actions
bot
removed
the
approved
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
the
approved
|
run buildall |
github-actions
bot
removed
the
approved
|
TeamCity be ut coverage result: |
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
the
approved
…49998) ### What problem does this PR solve? In this PR #34520, we only handle HivePartitionWriter. But this should be applied to all hdfs writer. This PR fix it, unify the logic to make it work with both hive and iceberg writer. If the path is an absolute full path like `hdfs://host/path/to/file`, use `hdfs://host/` as fs name, otherwise, use default fs name.
…49998) ### What problem does this PR solve? In this PR #34520, we only handle HivePartitionWriter. But this should be applied to all hdfs writer. This PR fix it, unify the logic to make it work with both hive and iceberg writer. If the path is an absolute full path like `hdfs://host/path/to/file`, use `hdfs://host/` as fs name, otherwise, use default fs name.
…pache#49998) ### What problem does this PR solve? In this PR apache#34520, we only handle HivePartitionWriter. But this should be applied to all hdfs writer. This PR fix it, unify the logic to make it work with both hive and iceberg writer. If the path is an absolute full path like `hdfs://host/path/to/file`, use `hdfs://host/` as fs name, otherwise, use default fs name.
5 participants
Proposed changes
When the nameserver specified by defaultFS is different from the nameserver of the actual table, write failure may occur.
Moreover, when the orc writing fails, it still goes to the commit process (actually, after the writing fails, FE should be able to get the error message from BE, and should not go to the commit process)
Fix:
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...