Skip to content

Commit

Permalink
update eco tools (#1057)
Browse files Browse the repository at this point in the history
* update eco tools

* Update nebula-algorithm.md

* Update ex-ug-import-from-neo4j.md

* Update mkdocs.yml

* Update mkdocs.yml
  • Loading branch information
foesa-yang authored Oct 18, 2021
1 parent 1e8dbf5 commit 446e563
Show file tree
Hide file tree
Showing 22 changed files with 48 additions and 47 deletions.
2 changes: 1 addition & 1 deletion docs-2.0/1.introduction/1.what-is-nebula-graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Nebula Graph支持严格的角色访问控制和LDAP(Lightweight Directory Acc

### 生态多样化

Nebula Graph开放了越来越多的原生工具,例如[Nebula Graph Studio](https://github.com/vesoft-inc/nebula-studio)[Nebula Console](https://github.com/vesoft-inc/nebula-console)[Nebula Exchange](https://github.com/vesoft-inc/nebula-spark-utils/tree/v2.0.0/nebula-exchange)等,更多工具可以查看 [生态工具概览](../20.appendix/6.eco-tool-version.md)
Nebula Graph开放了越来越多的原生工具,例如[Nebula Graph Studio](https://github.com/vesoft-inc/nebula-studio)[Nebula Console](https://github.com/vesoft-inc/nebula-console)[Nebula Exchange](https://github.com/vesoft-inc/nebula-exchange)等,更多工具可以查看 [生态工具概览](../20.appendix/6.eco-tool-version.md)

此外,Nebula Graph还具备与Spark、Flink、HBase等产品整合的能力,在这个充满挑战与机遇的时代,大大增强了自身的竞争力。

Expand Down
14 changes: 7 additions & 7 deletions docs-2.0/20.appendix/6.eco-tool-version.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Nebula Explorer(简称Explorer)是一款可以通过Web访问的图探索可

Nebula Exchange(简称Exchange)是一款Apache Spark™应用,用于在分布式环境中将集群中的数据批量迁移到Nebula Graph中,能支持多种不同格式的批式数据和流式数据的迁移。详情请参见[什么是Nebula Exchange](../nebula-exchange/about-exchange/ex-ug-what-is-exchange.md)

|Nebula Graph版本|[Exchange](https://github.com/vesoft-inc/nebula-spark-utils/tree/{{exchange.branch}}/nebula-exchange)版本(commit id)|
|Nebula Graph版本|Exchange版本(commit id)|
|:---|:---|
| {{ nebula.release }} | {{exchange.release}}(2c61ca5) |

Expand All @@ -82,39 +82,39 @@ Nebula Importer(简称Importer)是一款Nebula Graph的CSV文件导入工具

Nebula Spark Connector是一个Spark连接器,提供通过Spark标准形式读写Nebula Graph数据的能力。Nebula Spark Connector由Reader和Writer两部分组成。详情请参见[什么是Nebula Spark Connector](../nebula-spark-connector.md)

|Nebula Graph版本|[Spark Connector](https://github.com/vesoft-inc/nebula-spark-utils/tree/{{sparkconnector.branch}}/nebula-spark-connector)版本(commit id)|
|Nebula Graph版本|Spark Connector版本(commit id)|
|:---|:---|
| {{ nebula.release }} | {{sparkconnector.release}}(2c61ca5) |

## Nebula Flink Connector

Nebula Flink Connector是一款帮助Flink用户快速访问Nebula Graph的连接器,支持从Nebula Graph图数据库中读取数据,或者将其他外部数据源读取的数据写入Nebula Graph图数据库。详情请参见[什么是Nebula Flink Connector](../nebula-flink-connector.md)

|Nebula Graph版本|[Flink Connector](https://github.com/vesoft-inc/nebula-flink-connector/tree/{{flinkconnector.branch}})版本(commit id)|
|Nebula Graph版本|Flink Connector版本(commit id)|
|:---|:---|
| {{ nebula.release }} | {{flinkconnector.release}}(49b8f3d) |

## Nebula Algorithm

Nebula Algorithm(简称Algorithm)是一款基于[GraphX](https://spark.apache.org/graphx/)的Spark应用程序,通过提交Spark任务的形式使用完整的算法工具对Nebula Graph数据库中的数据执行图计算,也可以通过编程形式调用lib库下的算法针对DataFrame执行图计算。详情请参见[什么是Nebula Algorithm](../nebula-algorithm.md)

|Nebula Graph版本|[Algorithm](https://github.com/vesoft-inc/nebula-spark-utils/tree/{{algorithm.branch}}/nebula-algorithm)版本(commit id)|
|Nebula Graph版本|Algorithm版本(commit id)|
|:---|:---|
| {{ nebula.release }} | {{algorithm.release}}(2c61ca5) |

## Nebula Console

Nebula Console是Nebula Graph的原生CLI客户端。如何使用请参见[连接Nebula Graph](../2.quick-start/3.connect-to-nebula-graph.md)

|Nebula Graph版本|[Console](https://github.com/vesoft-inc/nebula-console)版本(commit id)|
|Nebula Graph版本|Console版本(commit id)|
|:---|:---|
| {{ nebula.release }} | {{console.release}}(3ce5151) |

## Nebula Docker Compose

Docker Compose可以快速部署Nebula Graph集群。如何使用请参见[Docker Compose部署Nebula Graph](../4.deployment-and-installation/2.compile-and-install-nebula-graph/3.deploy-nebula-graph-with-docker-compose.md)

|Nebula Graph版本|[Docker Compose](https://github.com/vesoft-inc/nebula-docker-compose/tree/master)版本(commit id)|
|Nebula Graph版本|Docker Compose版本(commit id)|
|:---|:---|
| {{ nebula.release }} | {{dockercompose.release}}(d42231f) |

Expand All @@ -132,7 +132,7 @@ Docker Compose可以快速部署Nebula Graph集群。如何使用请参见[Docke

[Nebula Bench](https://github.com/vesoft-inc/nebula-bench) 用于测试Nebula Graph的基线性能数据,使用LDBC v0.3.3的标准数据集。

|Nebula Graph版本|[Nebula Bench](https://github.com/vesoft-inc/nebula-bench)版本(commit id)|
|Nebula Graph版本|Nebula Bench版本(commit id)|
|:---|:---|
| {{ nebula.release }} | {{bench.release}}(661f871) |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ Nebula Graph历史版本指低于Nebula Graph v2.0.0-GA的版本,本文介绍
11. 升级其他客户端。
所有的客户端都必须升级到支持Nebula Graph v{{nebula.release}} 的版本。包括但不限于[Python](https://github.com/vesoft-inc/nebula-python)、[Java](https://github.com/vesoft-inc/nebula-java)、[go](https://github.com/vesoft-inc/nebula-go)、[C++](https://github.com/vesoft-inc/nebula-cpp)、[Flink-connector](https://github.com/vesoft-inc/nebula-flink-connector)、[Spark-util](https://github.com/vesoft-inc/nebula-spark-utils)、[Nebula Bench](https://github.com/vesoft-inc/nebula-bench)。请找到各 repo 对应的 branch。
所有的客户端都必须升级到支持Nebula Graph v{{nebula.release}} 的版本。包括但不限于[Python](https://github.com/vesoft-inc/nebula-python)、[Java](https://github.com/vesoft-inc/nebula-java)、[go](https://github.com/vesoft-inc/nebula-go)、[C++](https://github.com/vesoft-inc/nebula-cpp)、[Flink-connector](https://github.com/vesoft-inc/nebula-flink-connector)、[Algorithm](https://github.com/vesoft-inc/nebula-algorithm)、[Exchange](https://github.com/vesoft-inc/nebula-exchange)、[Spark-connector](https://github.com/vesoft-inc/nebula-spark-connector)、[Nebula Bench](https://github.com/vesoft-inc/nebula-bench)。请找到各 repo 对应的 branch。
!!! Note
Expand Down
23 changes: 12 additions & 11 deletions docs-2.0/nebula-algorithm.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Nebula Algorithm

[Nebula Algorithm](https://github.com/vesoft-inc/nebula-spark-utils/tree/master/nebula-algorithm) (简称Algorithm)是一款基于[GraphX](https://spark.apache.org/graphx/)的Spark应用程序,通过提交Spark任务的形式使用完整的算法工具对Nebula Graph数据库中的数据执行图计算,也可以通过编程形式调用lib库下的算法针对DataFrame执行图计算。
[Nebula Algorithm](https://github.com/vesoft-inc/nebula-algorithm) (简称Algorithm)是一款基于[GraphX](https://spark.apache.org/graphx/)的Spark应用程序,通过提交Spark任务的形式使用完整的算法工具对Nebula Graph数据库中的数据执行图计算,也可以通过编程形式调用lib库下的算法针对DataFrame执行图计算。

## 前提条件

Expand Down Expand Up @@ -48,22 +48,22 @@ Nebula Algorithm实现图计算的流程如下:

3. 调用GraphX提供的图算法(例如PageRank)或者自行实现的算法(例如Louvain社区发现)。

详细的实现方法可以参见相关[Scala文件](https://github.com/vesoft-inc/nebula-spark-utils/tree/master/nebula-algorithm/src/main/scala/com/vesoft/nebula/algorithm/lib)
详细的实现方法可以参见相关[Scala文件](https://github.com/vesoft-inc/nebula-algorithm/tree/master/nebula-algorithm/src/main/scala/com/vesoft/nebula/algorithm/lib)

## 获取Nebula Algorithm

### 编译打包

1. 克隆仓库`nebula-spark-utils`
1. 克隆仓库`nebula-algorithm`

```bash
$ git clone -b {{algorithm.branch}} https://github.com/vesoft-inc/nebula-spark-utils.git
$ git clone -b {{algorithm.branch}} https://github.com/vesoft-inc/nebula-algorithm.git
```

2. 进入目录`nebula-algorithm`

```bash
$ cd nebula-spark-utils/nebula-algorithm
$ cd nebula-algorithm
```

3. 编译打包。
Expand All @@ -88,15 +88,16 @@ Nebula Algorithm实现图计算的流程如下:

```bash
<dependency>
<groupId>com.vesoft</groupId>
<artifactId>nebula-algorithm</artifactId>
<version>{{algorithm.release}}</version>
<groupId>com.vesoft</groupId>
<artifactId>nebula-algorithm</artifactId>
<version>{{algorithm.release}}</version>
</dependency>
```

2. 传入参数调用算法(以PageRank为例)。更多算法请参见[测试用例](https://github.com/vesoft-inc/nebula-spark-utils/tree/master/nebula-algorithm/src/test/scala/com/vesoft/nebula/algorithm/lib)
2. 传入参数调用算法(以PageRank为例)。更多算法请参见[测试用例](https://github.com/vesoft-inc/nebula-algorithm/tree/master/nebula-algorithm/src/test/scala/com/vesoft/nebula/algorithm/lib)

!!! note

执行算法的DataFrame默认第一列是起始点,第二列是目的点,第三列是边权重(非Nebula Graph中的Rank)。

```bash
Expand All @@ -109,7 +110,7 @@ Nebula Algorithm实现图计算的流程如下:
!!! note
使用封装好的算法包有一定的局限性,例如落库到Nebula Graph时,落库的图空间中创建的Tag的属性名称必须和代码内预设的名称保持一致。如果用户有开发能力,推荐使用第一种方法。

1. 设置[配置文件](https://github.com/vesoft-inc/nebula-spark-utils/blob/{{algorithm.branch}}/nebula-algorithm/src/main/resources/application.conf)
1. 设置[配置文件](https://github.com/vesoft-inc/nebula-algorithm/blob/{{algorithm.branch}}/nebula-algorithm/src/main/resources/application.conf)

```bash
{
Expand Down Expand Up @@ -266,7 +267,7 @@ Nebula Algorithm实现图计算的流程如下:
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.algorithm.Main /root/nebula-spark-utils/nebula-algorithm/target/nebula-algorithm-{{algorithm.release}}.jar -p /root/nebula-spark-utils/nebula-algorithm/src/main/resources/application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.algorithm.Main /root/nebula-algorithm/target/nebula-algorithm-{{algorithm.release}}.jar -p /root/nebula-algorithm/src/main/resources/application.conf
```

## 视频
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 什么是Nebula Exchange

[Nebula Exchange](https://github.com/vesoft-inc/nebula-spark-utils/tree/{{exchange.branch}}/nebula-exchange)(简称Exchange)是一款Apache Spark&trade;应用,用于在分布式环境中将集群中的数据批量迁移到Nebula Graph中,能支持多种不同格式的批式数据和流式数据的迁移。
[Nebula Exchange](https://github.com/vesoft-inc/nebula-exchange)(简称Exchange)是一款Apache Spark&trade;应用,用于在分布式环境中将集群中的数据批量迁移到Nebula Graph中,能支持多种不同格式的批式数据和流式数据的迁移。

Exchange由Reader、Processor和Writer三部分组成。Reader读取不同来源的数据返回DataFrame后,Processor遍历DataFrame的每一行,根据配置文件中`fields`的映射关系,按列名获取对应的值。在遍历指定批处理的行数后,Writer会将获取的数据一次性写入到Nebula Graph中。下图描述了Exchange完成数据转换和迁移的过程。

Expand Down
8 changes: 4 additions & 4 deletions docs-2.0/nebula-exchange/ex-ug-compile.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,16 @@

## 编译Exchange

1. 在根目录克隆仓库`nebula-spark-utils`
1. 在根目录克隆仓库`nebula-exchange`

```bash
git clone -b {{exchange.branch}} https://github.com/vesoft-inc/nebula-spark-utils.git
git clone -b {{exchange.branch}} https://github.com/vesoft-inc/nebula-exchange.git
```

2. 切换到目录`nebula-exchange`

```bash
cd nebula-spark-utils/nebula-exchange
cd nebula-exchange/nebula-exchange
```

3. 打包Nebula Exchange。
Expand Down Expand Up @@ -55,7 +55,7 @@
!!! note
JAR文件版本号会因Nebula Java Client的发布版本而变化。用户可以在[Releases页面](https://github.com/vesoft-inc/nebula-java/releases)查看最新版本。

迁移数据时,用户可以参考配置文件[`target/classes/application.conf`](https://github.com/vesoft-inc/nebula-spark-utils/blob/master/nebula-exchange/src/main/resources/application.conf)
迁移数据时,用户可以参考配置文件[`target/classes/application.conf`](https://github.com/vesoft-inc/nebula-exchange/blob/master/nebula-exchange/src/main/resources/application.conf)

## 下载依赖包失败

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 配置说明

本文介绍使用Nebula Exchange时如何修改配置文件[`application.conf`](https://github.com/vesoft-inc/nebula-spark-utils/blob/master/nebula-exchange/src/main/resources/application.conf)
本文介绍使用Nebula Exchange时如何修改配置文件[`application.conf`](https://github.com/vesoft-inc/nebula-exchange/blob/master/nebula-exchange/src/main/resources/application.conf)

修改配置文件之前,建议根据数据源复制并修改文件名称,便于区分。例如数据源为CSV文件,可以复制为`csv_application.conf`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/clickhouse_application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-exchange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/clickhouse_application.conf
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/csv_application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-exchange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/csv_application.conf
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/hbase_application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-exchange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/hbase_application.conf
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/hive_application.conf -h
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-exchange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/hive_application.conf -h
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -380,7 +380,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/json_application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-echange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/json_application.conf
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/kafka_application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-exchange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/kafka_application.conf
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/maxcompute_application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-exchange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/maxcompute_application.conf
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ ${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchan
示例:

```bash
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-spark-utils/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-spark-utils/nebula-exchange/target/classes/mysql_application.conf
${SPARK_HOME}/bin/spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange /root/nebula-exchange/nebula-exchange/target/nebula-exchange-{{exchange.release}}.jar -c /root/nebula-exchange/nebula-exchange/target/classes/mysql_application.conf
```

用户可以在返回信息中搜索`batchSuccess.<tag_name/edge_name>`,确认成功的数量。例如`batchSuccess.follow: 300`
Expand Down
Loading

0 comments on commit 446e563

Please sign in to comment.