Skip to content

Commit c3fd8c1

Browse files
SongGuyangimzhenyu
authored andcommitted
some fix about ray on yarn project
1 parent a48f0fc commit c3fd8c1

File tree

9 files changed

+69
-296
lines changed

9 files changed

+69
-296
lines changed

devops/yarn/README.md

Lines changed: 0 additions & 47 deletions
This file was deleted.

devops/yarn/README.rst

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
Prerequisites
2+
=============
3+
4+
1. Yarn app of Ray is run under the Hadoop environment version 2.8.0.
5+
You can get the Hadoop binary from
6+
`here <http://archive.apache.org/dist/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz>`__.
7+
8+
Walkthrough
9+
===========
10+
11+
Yarn environment and configuration
12+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
13+
14+
Firstly, you should have an available hadoop yarn environment and
15+
configuration.
16+
17+
Prepare ray on yarn jar file
18+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
19+
20+
.. code:: shell
21+
22+
$ make clean package
23+
24+
Prepare deploy zip file
25+
^^^^^^^^^^^^^^^^^^^^^^^
26+
27+
.. code:: shell
28+
29+
$ cd deploy
30+
$ zip -r deploy.zip .
31+
32+
Please modify the script 'deploy/run.sh' on-demand.
33+
34+
Run
35+
^^^
36+
37+
.. code:: shell
38+
39+
$ /path/to/hadoop-2.8.0/bin/yarn jar ./target/ray-on-yarn-1.0.jar org.ray.on.yarn.Client --jar ./target/ray-on-yarn-1.0.jar --rayArchive ./deploy/deploy.zip --containerVcores 2 --containerMemory 2048 --priority 10 --shellCmdPriority 10 --numRoles 1 1 --queue ray --headNodeStaticArgs "'--num-cpus 4 --num-gpus 4'" --workNodeStaticArgs "'--num-cpus 2 --num-gpus 2'"
40+
41+
Please modify the command line on-demand. Some detail about the input
42+
args is in the help doc.
43+
44+
.. code:: shell
45+
46+
/path/to/hadoop-2.8.0/bin/yarn jar ./target/ray-on-yarn-1.0.jar org.ray.on.yarn.Client --help
47+
48+
Monitoring
49+
^^^^^^^^^^
50+
51+
Please check the logs depend on your yarn platform.
52+
53+
Stop
54+
^^^^
55+
56+
.. code:: shell
57+
58+
$ /path/to/hadoop-2.8.0/bin/yarn application -kill {app_id}
59+
60+
``{app_id}`` shall be replaced by the ID of the corresponding Yarn
61+
application, e.g. ``application_1505745052163_0107``.

devops/yarn/checkstyle.xml

Lines changed: 0 additions & 241 deletions
This file was deleted.

devops/yarn/pom.xml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@
6666
</execution>
6767
</executions>
6868
<configuration>
69-
<configLocation>checkstyle.xml</configLocation>
69+
<configLocation>../../java/checkstyle.xml</configLocation>
7070
<encoding>UTF-8</encoding>
7171
<consoleOutput>true</consoleOutput>
7272
<failsOnError>false</failsOnError>
@@ -81,4 +81,4 @@
8181
</plugins>
8282
</pluginManagement>
8383
</build>
84-
</project>
84+
</project>

devops/yarn/src/main/java/org/ray/on/yarn/ApplicationMaster.java renamed to devops/yarn/src/main/java/org/ray/yarn/ApplicationMaster.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
package org.ray.on.yarn;
1+
package org.ray.yarn;
22

33
import com.google.common.annotations.VisibleForTesting;
44
import com.google.common.collect.Maps;

devops/yarn/src/main/java/org/ray/on/yarn/Client.java renamed to devops/yarn/src/main/java/org/ray/yarn/Client.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
package org.ray.on.yarn;
1+
package org.ray.yarn;
22

33
import com.google.common.collect.Maps;
44
import java.io.IOException;
@@ -176,7 +176,7 @@ public static void main(String[] args) {
176176
}
177177

178178
public Client(Configuration conf) throws Exception {
179-
this("org.ray.on.yarn.ApplicationMaster", conf);
179+
this("org.ray.yarn.ApplicationMaster", conf);
180180
}
181181

182182
Client(String appMasterMainClass, Configuration conf) {

devops/yarn/src/main/java/org/ray/on/yarn/DsConstants.java renamed to devops/yarn/src/main/java/org/ray/yarn/DsConstants.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
package org.ray.on.yarn;
1+
package org.ray.yarn;
22

33
import org.apache.hadoop.classification.InterfaceAudience;
44
import org.apache.hadoop.classification.InterfaceStability;

0 commit comments

Comments
 (0)