Skip to content

Conversation

@wuwenchi
Copy link
Contributor

@wuwenchi wuwenchi commented May 8, 2024

Proposed changes

mirror #34520

When the nameserver specified by defaultFS is different from the nameserver of the actual table, write failure may occur.
Moreover, when the orc writing fails, it still goes to the commit process (actually, after the writing fails, FE should be able to get the error message from BE, and should not go to the commit process)

Fix:

  1. When orc writing fails, an exception is thrown. The upstream captures the exception and returns it to status for FE to obtain.
  2. When creating fs in the writing process, if the writing path contains a nameserver, then use this nameserver directly to create fs. (In this way, when writing to hive, there is no need to configure defaultFS in the catalog)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions
Copy link
Contributor

github-actions bot commented May 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@wuwenchi wuwenchi changed the title [bugfix](hive)fix error for writing to hive [bugfix](hive)fix error for writing to hive for 2.1 May 8, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@github-actions
Copy link
Contributor

github-actions bot commented May 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@github-actions
Copy link
Contributor

github-actions bot commented May 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

if (idx != string::npos) {
idx = path.find("/", idx + 3);
if (idx != string::npos) {
hdfs_params.fs_name = path.substr(0, idx);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

give a comment to show that parse fs_name from hdfs://host:port and hdfs://dfsnameservice

@morningman morningman merged commit 4dd5379 into apache:branch-2.1 May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants