Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-27646 Should not use pread when prefetching in HFilePreadReader #5063

Merged
merged 1 commit into from
Mar 20, 2023

Conversation

sunhelly
Copy link
Contributor

Using isolate stream reader for prefetching in pread reader, to speed up the prefetching progress.
The blocks cached when prefetching will be released after the pread reader is closed.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 17s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 27s master passed
+1 💚 compile 2m 52s master passed
+1 💚 checkstyle 0m 43s master passed
+1 💚 spotless 0m 50s branch has no errors when running spotless:check.
+1 💚 spotbugs 1m 54s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 0s the patch passed
+1 💚 compile 2m 48s the patch passed
+1 💚 javac 2m 48s the patch passed
+1 💚 checkstyle 0m 40s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 14m 21s Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚 spotless 0m 41s patch has no errors when running spotless:check.
+1 💚 spotbugs 1m 48s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 11s The patch does not generate ASF License warnings.
45m 7s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #5063
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname Linux 66c5a43ab9ac 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 10037df
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 50s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 16s master passed
+1 💚 compile 0m 56s master passed
+1 💚 shadedjars 6m 30s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 32s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 14s the patch passed
+1 💚 compile 0m 53s the patch passed
+1 💚 javac 0m 53s the patch passed
+1 💚 shadedjars 6m 7s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 21s the patch passed
_ Other Tests _
-1 ❌ unit 234m 19s hbase-server in the patch failed.
263m 16s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #5063
Optional Tests javac javadoc unit shadedjars compile
uname Linux dcfbb7d6b779 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 10037df
Default Java Temurin-1.8.0_352-b08
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/testReport/
Max. process+thread count 2663 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 38s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 5m 52s master passed
+1 💚 compile 1m 14s master passed
+1 💚 shadedjars 6m 39s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 41s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 5m 24s the patch passed
+1 💚 compile 1m 1s the patch passed
+1 💚 javac 1m 1s the patch passed
+1 💚 shadedjars 5m 43s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 28s the patch passed
_ Other Tests _
-1 ❌ unit 262m 2s hbase-server in the patch failed.
293m 58s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #5063
Optional Tests javac javadoc unit shadedjars compile
uname Linux faddcf3016a5 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 10037df
Default Java Eclipse Adoptium-11.0.17+8
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/testReport/
Max. process+thread count 2500 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 6s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 3m 35s master passed
+1 💚 compile 2m 24s master passed
+1 💚 checkstyle 0m 35s master passed
+1 💚 spotless 0m 44s branch has no errors when running spotless:check.
+1 💚 spotbugs 1m 30s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 22s the patch passed
+1 💚 compile 2m 20s the patch passed
+1 💚 javac 2m 20s the patch passed
+1 💚 checkstyle 0m 34s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 12m 43s Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚 spotless 0m 40s patch has no errors when running spotless:check.
+1 💚 spotbugs 1m 34s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 12s The patch does not generate ASF License warnings.
38m 16s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #5063
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname Linux ed095061148e 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / d2b0074
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 86 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 2m 12s Docker mode activated.
-0 ⚠️ yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 42s master passed
+1 💚 compile 0m 42s master passed
+1 💚 shadedjars 4m 38s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 22s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 12s the patch passed
+1 💚 compile 0m 41s the patch passed
+1 💚 javac 0m 41s the patch passed
+1 💚 shadedjars 4m 32s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 22s the patch passed
_ Other Tests _
+1 💚 unit 199m 35s hbase-server in the patch passed.
224m 0s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #5063
Optional Tests javac javadoc unit shadedjars compile
uname Linux 5273a7ac57e5 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / d2b0074
Default Java Eclipse Adoptium-11.0.17+8
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/testReport/
Max. process+thread count 2601 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 2m 14s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 9s master passed
+1 💚 compile 0m 38s master passed
+1 💚 shadedjars 5m 9s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 26s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 58s the patch passed
+1 💚 compile 0m 42s the patch passed
+1 💚 javac 0m 42s the patch passed
+1 💚 shadedjars 5m 30s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 21s the patch passed
_ Other Tests _
+1 💚 unit 209m 14s hbase-server in the patch passed.
234m 15s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #5063
Optional Tests javac javadoc unit shadedjars compile
uname Linux 924caba0124e 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / d2b0074
Default Java Temurin-1.8.0_352-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/testReport/
Max. process+thread count 2641 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 15s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 5m 32s master passed
+1 💚 compile 3m 29s master passed
+1 💚 checkstyle 0m 48s master passed
+1 💚 spotless 1m 7s branch has no errors when running spotless:check.
+1 💚 spotbugs 2m 22s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 5m 17s the patch passed
+1 💚 compile 3m 26s the patch passed
+1 💚 javac 3m 26s the patch passed
+1 💚 checkstyle 1m 5s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 18m 33s Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚 spotless 0m 59s patch has no errors when running spotless:check.
+1 💚 spotbugs 2m 21s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 12s The patch does not generate ASF License warnings.
56m 39s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #5063
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname Linux c669005bedaf 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 37858bb
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 85 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 35s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 25s master passed
+1 💚 compile 0m 39s master passed
+1 💚 shadedjars 4m 36s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 21s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 45s the patch passed
+1 💚 compile 0m 35s the patch passed
+1 💚 javac 0m 35s the patch passed
+1 💚 shadedjars 4m 34s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 21s the patch passed
_ Other Tests _
+1 💚 unit 231m 22s hbase-server in the patch passed.
253m 26s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #5063
Optional Tests javac javadoc unit shadedjars compile
uname Linux 65320223f263 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 37858bb
Default Java Temurin-1.8.0_352-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/testReport/
Max. process+thread count 2649 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 58s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 34s master passed
+1 💚 compile 0m 48s master passed
+1 💚 shadedjars 4m 27s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 26s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 21s the patch passed
+1 💚 compile 0m 46s the patch passed
+1 💚 javac 0m 46s the patch passed
+1 💚 shadedjars 4m 25s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 26s the patch passed
_ Other Tests _
-1 ❌ unit 232m 6s hbase-server in the patch failed.
255m 35s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #5063
Optional Tests javac javadoc unit shadedjars compile
uname Linux 13dbdd67f000 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 37858bb
Default Java Eclipse Adoptium-11.0.17+8
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/testReport/
Max. process+thread count 2684 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5063/3/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@sunhelly
Copy link
Contributor Author

The failed UTs are not relevant to this PR, and all passed in local tests.
Ping @bbeaudreault @Apache9 , if there are any other concerns, please let me know. Thanks.

Copy link
Contributor

@bbeaudreault bbeaudreault left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks

@sunhelly sunhelly merged commit 201846b into apache:master Mar 20, 2023
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
…apache#5063)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
…apache#5063)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
…apache#5063)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 20, 2023
…apache#5063)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit that referenced this pull request Mar 20, 2023
…#5063) (#5122)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit that referenced this pull request Mar 20, 2023
…#5063) (#5123)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
sunhelly added a commit that referenced this pull request Mar 20, 2023
…#5063) (#5124)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
wesiq pushed a commit to web-iq/hbase that referenced this pull request Mar 27, 2023
…apache#5063)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
…apache#5063) (apache#5123)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit fde0f4b)
Change-Id: Id727198b9ef22a293f0984521bd6816266b106cb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants