Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDFS-17715. [ARR] Add documentation for asynchronous router. #7318

Merged
merged 11 commits into from
Jan 24, 2025
Merged

HDFS-17715. [ARR] Add documentation for asynchronous router. #7318

merged 11 commits into from
Jan 24, 2025

Conversation

KeeProMise
Copy link
Member

@KeeProMise KeeProMise commented Jan 23, 2025 •
edited
Loading

Description of PR

We need to add a description document to describe the function, configuration and other information of the asynchronous router.
seeAlso: https://issues.apache.org/jira/browse/HDFS-17715

How was this patch tested?

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hfutatzhanghb and others added 5 commits January 20, 2025 14:38
@hfutatzhanghb @KeeProMise
HADOOP-19361. RPC DeferredMetrics bugfix. (apache#7220). Contributed …
1f1d7c1 
…by hfutatzhanghb.

Reviewed-by: Jian Zhang <keepromise@apache.org>
@hfutatzhanghb @KeeProMise
HDFS-17640.[ARR] RouterClientProtocol supports asynchronous rpc. (apa…
1e0dabc 
…che#7188)

Co-authored-by: Jian Zhang <keepromise@apache.org>
@hfutatzhanghb @KeeProMise
HDFS-17650. [ARR] The router server-side rpc protocol PB supports asy…
d83dae4 
…nchrony. (apache#7139). Contributed by hfutatzhanghb.

Co-authored-by: Jian Zhang <keepromise@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
@hfutatzhanghb @KeeProMise
HDFS-17651.[ARR] Async handler executor isolation (apache#7244). Cont…
616d707 
…ributed by hfutatzhanghb.

Reviewed-by: Jian Zhang <keepromise@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
@KeeProMise
[ARR] Add documentation for asynchronous router.
4e4700c
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 12m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HDFS-17531 Compile Tests _
+1 💚 mvninstall 36m 57s HDFS-17531 passed
+1 💚 mvnsite 0m 47s HDFS-17531 passed
+1 💚 shadedclient 70m 56s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 35s the patch passed
-1 ❌ blanks 0m 0s /blanks-eol.txt The patch has 8 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚 mvnsite 0m 37s the patch passed
+1 💚 shadedclient 33m 13s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 39s The patch does not generate ASF License warnings.
119m 34s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/1/artifact/out/Dockerfile
GITHUB PR #7318
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux ab467daa951a 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision HDFS-17531 / 4e4700c
Max. process+thread count 700 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/1/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@KeeProMise
Copy link
Member Author

@Hexiaoqiao @ferhui
Hi, if you have time, please help to take a look at this PR, thanks!

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 40s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HDFS-17531 Compile Tests _
+1 💚 mvninstall 35m 40s HDFS-17531 passed
+1 💚 mvnsite 0m 47s HDFS-17531 passed
+1 💚 shadedclient 69m 47s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 36s the patch passed
+1 💚 shadedclient 35m 11s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 36s The patch does not generate ASF License warnings.
109m 6s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/2/artifact/out/Dockerfile
GITHUB PR #7318
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux ebf5a0ed4347 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision HDFS-17531 / f05c565
Max. process+thread count 562 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/2/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 51s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HDFS-17531 Compile Tests _
+1 💚 mvninstall 40m 31s HDFS-17531 passed
+1 💚 mvnsite 0m 49s HDFS-17531 passed
+1 💚 shadedclient 79m 23s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 36s the patch passed
+1 💚 shadedclient 38m 28s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
121m 33s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/3/artifact/out/Dockerfile
GITHUB PR #7318
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux 30525c0759bb 5.15.0-125-generic #135-Ubuntu SMP Fri Sep 27 13:53:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision HDFS-17531 / f05c565
Max. process+thread count 525 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/3/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 47s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HDFS-17531 Compile Tests _
+1 💚 mvninstall 40m 0s HDFS-17531 passed
+1 💚 mvnsite 0m 49s HDFS-17531 passed
+1 💚 shadedclient 79m 23s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 36s the patch passed
+1 💚 shadedclient 38m 36s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
121m 40s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/4/artifact/out/Dockerfile
GITHUB PR #7318
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux 5268605eb913 5.15.0-125-generic #135-Ubuntu SMP Fri Sep 27 13:53:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision HDFS-17531 / 5393932
Max. process+thread count 571 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/4/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HDFS-17531 Compile Tests _
+1 💚 mvninstall 35m 38s HDFS-17531 passed
+1 💚 mvnsite 0m 47s HDFS-17531 passed
+1 💚 shadedclient 69m 31s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 35s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 36s the patch passed
+1 💚 shadedclient 33m 18s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
106m 12s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/5/artifact/out/Dockerfile
GITHUB PR #7318
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux 5a1b24b31979 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision HDFS-17531 / 8743956
Max. process+thread count 555 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/5/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HDFS-17531 Compile Tests _
+1 💚 mvninstall 36m 0s HDFS-17531 passed
+1 💚 mvnsite 0m 47s HDFS-17531 passed
+1 💚 shadedclient 70m 9s branch has no errors when building and testing our client artifacts.
-0 ⚠️ patch 70m 30s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
+1 💚 mvnsite 0m 37s the patch passed
+1 💚 shadedclient 34m 18s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 40s The patch does not generate ASF License warnings.
107m 55s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/6/artifact/out/Dockerfile
GITHUB PR #7318
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux 730a28192c37 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision HDFS-17531 / 31852d5
Max. process+thread count 709 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/6/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@KeeProMise KeeProMise mentioned this pull request Jan 24, 2025
Closed
4 tasks
@Hexiaoqiao Hexiaoqiao merged commit b47e11e into apache:HDFS-17531 Jan 24, 2025
4 checks passed
@Hexiaoqiao
Copy link
Contributor

Committed to HDFS-17531. Thanks @KeeProMise .

KeeProMise added a commit to KeeProMise/hadoop that referenced this pull request Feb 11, 2025
@KeeProMise
HDFS-17715. [ARR] Add documentation for asynchronous router. (apache#…
82bccee 
…7318). Contributed by Jian Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
KeeProMise added a commit that referenced this pull request Feb 14, 2025
@KeeProMise
HDFS-17715. [ARR] Add documentation for asynchronous router. (#7318).…
ad83682 
… Contributed by Jian Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
KeeProMise added a commit to KeeProMise/hadoop that referenced this pull request Feb 20, 2025
@KeeProMise
HDFS-17715. [ARR] Add documentation for asynchronous router. (apache#…
cc9b314 
…7318). Contributed by Jian Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
KeeProMise added a commit to KeeProMise/hadoop that referenced this pull request Feb 20, 2025
@KeeProMise
HDFS-17715. [ARR] Add documentation for asynchronous router. (apache#…
74d320f 
…7318). Contributed by Jian Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Hexiaoqiao pushed a commit to Hexiaoqiao/hadoop that referenced this pull request Feb 20, 2025
@KeeProMise @Hexiaoqiao
HDFS-17715. [ARR] Add documentation for asynchronous router. (apache#…
05ae3e0 
…7318). Contributed by Jian Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Hexiaoqiao Hexiaoqiao Hexiaoqiao approved these changes

Assignees
No one assigned
Labels
HDFS RBF trunk
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@KeeProMise @hadoop-yetus @Hexiaoqiao @hfutatzhanghb