Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-2221. Monitor datanodes in ozoneperf compose cluster #1563

Closed
wants to merge 1 commit into from

Conversation

elek
Copy link
Member

@elek elek commented Oct 1, 2019

ozoneperf compose cluster contains a prometheus but as of now it collects the data only from scm and om.

We don't know the exact number of datanodes (can be scaled up and down) therefor it's harder to configure the datanode host names. I would suggest to configure the first 10 datanodes (which covers most of the use cases)

How to test?

cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozoneperf
docker-compose up -d
firefox http://localhost:9090/targets

 

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 82 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
0 yamllint 1 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
-1 mvninstall 60 hadoop-hdds in trunk failed.
-1 mvninstall 41 hadoop-ozone in trunk failed.
-1 compile 18 hadoop-hdds in trunk failed.
-1 compile 12 hadoop-ozone in trunk failed.
+1 mvnsite 0 trunk passed
+1 shadedclient 1022 branch has no errors when building and testing our client artifacts.
-1 javadoc 18 hadoop-hdds in trunk failed.
-1 javadoc 16 hadoop-ozone in trunk failed.
_ Patch Compile Tests _
-1 mvninstall 31 hadoop-hdds in the patch failed.
-1 mvninstall 34 hadoop-ozone in the patch failed.
-1 compile 21 hadoop-hdds in the patch failed.
-1 compile 15 hadoop-ozone in the patch failed.
-1 javac 21 hadoop-hdds in the patch failed.
-1 javac 15 hadoop-ozone in the patch failed.
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 778 patch has no errors when building and testing our client artifacts.
-1 javadoc 19 hadoop-hdds in the patch failed.
-1 javadoc 15 hadoop-ozone in the patch failed.
_ Other Tests _
-1 unit 23 hadoop-hdds in the patch failed.
-1 unit 22 hadoop-ozone in the patch failed.
+1 asflicense 28 The patch does not generate ASF License warnings.
2259
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/Dockerfile
GITHUB PR #1563
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient yamllint
uname Linux d585472cbb1d 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 425a6c8
Default Java 1.8.0_222
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/branch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/branch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/branch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/branch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/branch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/branch-javadoc-hadoop-ozone.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-compile-hadoop-ozone.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-compile-hadoop-hdds.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-javadoc-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/testReport/
Max. process+thread count 307 (vs. ulimit of 5500)
modules C: hadoop-ozone/dist U: hadoop-ozone/dist
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/1/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 1805 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
0 yamllint 0 yamllint was not available.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
-1 mvninstall 44 hadoop-hdds in trunk failed.
-1 mvninstall 39 hadoop-ozone in trunk failed.
-1 compile 22 hadoop-hdds in trunk failed.
-1 compile 15 hadoop-ozone in trunk failed.
+1 mvnsite 0 trunk passed
+1 shadedclient 892 branch has no errors when building and testing our client artifacts.
-1 javadoc 22 hadoop-hdds in trunk failed.
-1 javadoc 19 hadoop-ozone in trunk failed.
_ Patch Compile Tests _
-1 mvninstall 34 hadoop-hdds in the patch failed.
-1 mvninstall 37 hadoop-ozone in the patch failed.
-1 compile 26 hadoop-hdds in the patch failed.
-1 compile 19 hadoop-ozone in the patch failed.
-1 javac 26 hadoop-hdds in the patch failed.
-1 javac 19 hadoop-ozone in the patch failed.
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 706 patch has no errors when building and testing our client artifacts.
-1 javadoc 23 hadoop-hdds in the patch failed.
-1 javadoc 20 hadoop-ozone in the patch failed.
_ Other Tests _
-1 unit 28 hadoop-hdds in the patch failed.
-1 unit 26 hadoop-ozone in the patch failed.
+1 asflicense 32 The patch does not generate ASF License warnings.
3838
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/Dockerfile
GITHUB PR #1563
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient yamllint
uname Linux fe9e5669e7ae 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 99cd757
Default Java 1.8.0_222
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/branch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/branch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/branch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/branch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/branch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/branch-javadoc-hadoop-ozone.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-compile-hadoop-ozone.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-compile-hadoop-hdds.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-javadoc-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/testReport/
Max. process+thread count 412 (vs. ulimit of 5500)
modules C: hadoop-ozone/dist U: hadoop-ozone/dist
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1563/2/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@elek
Copy link
Member Author

elek commented Oct 13, 2019

Thank you very much to open this pull request.

During the weekend the Ozone source code has been moved out from apache/hadoop repository to apache/hadoop-ozone repository.

This git commits are rewritten, but the branch of this pull request is also transformed (state of Saturday morning), you can use the new, migrated branch to recreate this pull request.

Your pull request is important for us: Can you please re-create your pull request in the new repository?

1. Create a new fork of https://github.com/apache/hadoop-ozone

2. Clone it and have both your fork and the apache repo as remotes:

git clone git@github.com:elek/hadoop-ozone.git
cd hadoop-ozone
git remote add apache git@github.com:apache/hadoop-ozone.git
git fetch apache

3. Fetch your migrated branch and push it to your fork.

git checkout -b HDDS-2221 apache/HDDS-2221
git push origin HDDS-2221

4. And create the new pull request on the new repository.

https://github.com/apache/hadoop-ozone/compare/master...elek:HDDS-2221?expand=1

If you need more information, please check this wiki page or contact with me (my github user name + apache.org).

Thank you, and sorry for the inconvenience.

@elek
Copy link
Member Author

elek commented Oct 13, 2019

Closing because apache/ozone#1

@elek elek closed this Oct 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants