[Example] Add AMGNet example #549

HydrogenSulfate · 2023-09-22T03:39:24Z

PR types

New features

PR changes

Others

Describe

添加 AMGNet 代码和文档
在 ppsci.utils.misc 中添加 Timer 计时器上下文管理器功能，便于对代码块进行计时
train.py/eval.py/collate_fn 代码适配 pgl 的 DataLoader 和 pgl.Graph 类型的数据
添加 MeshAirfoilDataset 数据集

其他：

优化多处代码逻辑以适配 AMGNet
文档首页"快速安装"章节样式调整

参考案例: https://aistudio.baidu.com/projectdetail/6779372?contributionType=1

paddle-bot · 2023-09-22T03:39:28Z

Thanks for your contribution!

zhiminzhang0830 · 2023-09-27T08:31:19Z

examples/amgnet/utils.py

+matplotlib.use("Agg")
+
+
+def getcorsenode(latent_graph):


建议单词之间使用下划线隔开

zhiminzhang0830 · 2023-09-27T08:33:45Z

examples/amgnet/utils.py

+        plt.savefig(out_file)
+        plt.close()
+
+    if show:


检查一下该if内是否还需要抛出异常

zhiminzhang0830 · 2023-09-27T08:35:48Z

ppsci/arch/amgnet.py

+    return output
+
+
+def getcorsenode(latent_graph: "pgl.Graph") -> paddle.Tensor:


zhiminzhang0830 · 2023-09-27T08:39:43Z

ppsci/arch/amgnet.py

+        :param segment_ids: The segment indices tensor.
+        :param num_segments: The number of segments.
+        :return: A tensor of same data type as the data argument.
+        """


规范docstring

zhiminzhang0830 · 2023-09-27T08:40:29Z

ppsci/arch/amgnet.py

+    Then the result is add to output latent graph of encoder and the modified latent graph will be feed into original processor
+
+    Option: choose whether to normalize the high rank node connection
+    """


规范docstring

zhiminzhang0830 · 2023-09-27T08:46:53Z

ppsci/data/__init__.py

+            places=device.get_device(),
+            batch_sampler=sampler,
+            collate_fn=collate_fn,
+            num_workers=cfg.get("num_workers", 0),


122行、113行两处的num_workers的默认值是否可以保持一致？

已修改为1

zhiminzhang0830 · 2023-09-27T08:51:14Z

ppsci/data/dataset/__init__.py

@@ -46,6 +48,8 @@
    "LorenzDataset",
    "RosslerDataset",
    "VtuDataset",
+    "MeshAirfoilDataset",


保持与import顺序一致

zhiminzhang0830 · 2023-09-27T08:51:20Z

ppsci/data/dataset/__init__.py

@@ -46,6 +48,8 @@
    "LorenzDataset",
    "RosslerDataset",
    "VtuDataset",
+    "MeshAirfoilDataset",
+    "MeshCylinderDataset",


zhiminzhang0830 · 2023-09-27T08:54:51Z

ppsci/data/dataset/airfoil_dataset.py

+                        for _ in range(num_elems)
+                    ]
+                    marker_dict[marker_tag] = marker_elems
+            if line.startswith("NELEM"):


与上面的两个if是互斥关系，可否使用elif代替

zhiminzhang0830 · 2023-10-19T06:51:22Z

docs/zh/examples/amgnet.md

+
+- AMGNET 通过 RS 算法(Olson and Schroder, 2018)进行了图的粗化，仅使用少量节点即可进行预测，进一步提高了预测速度。
+
+下图为该方法的网络结构图。该模型的基本原理就是将网格结构转化为图结构，然后通过网格中节点的物理信息，位置信息以及节点类型对图中的节点和边进行编码。接着对得到的图神经网络使用基于代数多重网格算法(RS)的粗化层进行粗化，将所有节点分类为粗节点集和细节点集，其中粗节点集是细节点集的子集。粗图的节点集合就是粗节点集，于是完成了图的粗化，缩小了图的规模。粗化完成后通过设计的图神经网络信息传递块(GN)来总结和提取图的特征。之后图恢复层采用反向操作，使用空间插值法(Qi et al.,2017)对图进行上采样。例如要对节点 $i$ 插值，则在粗图中找到距离节点 $i$ 最近的 $k$ 个节点，然后通过公式计算得到节点 $i$ 的特征。最后，通过解码器得到每个节点的速度与压力信息。


"然后通过网格中节点的物理信息，位置信息以及节点类型对图中的节点和边进行编码" --> "然后通过网格中节点的物理信息、位置信息以及节点类型对图中的节点和边进行编码"

已修改，逗号改顿号

zhiminzhang0830 · 2023-10-19T06:57:14Z

docs/zh/examples/amgnet.md

+
+### 3.6 评估器构建
+
+在训练过程中通常会按一定轮数间隔，用验证集(测试集)评估当前模型的训练情况，因此使用 `ppsci.validate.SupervisedValidator` 构建评估器，构建过程与 [约束构建](#34) 类似，只需把数据目录改为测试集的目录，并在配置文件中设置 `EVAL.batch_size=1` 即可。


上文中约束构建是3.3 约束构建

zhiminzhang0830 · 2023-10-19T08:33:16Z

examples/amgnet/amgnet_airfoil.py

+        constraint,
+        cfg.output_dir,
+        optimizer,
+        None,


这个可以去掉吧

这个None参数属于位置参数，不能删除，删了的话代码行数会变动

zhiminzhang0830

LGTM

* update WIP code * (WIP)update AMGNet code * try import pgl to avoid importerror * try import pyamg to avoid importerror * add airfoil_dataset.py * add type checking for amgnet * try import pgl to avoid importerror * refine Timer * replace pgl.Dataset with io.Dataset * update reproded code * replace ImportError with ModuleNotFoundError * refine amgnet.py * refine amgnet_airfoil.py and amgnet_cylinder.py * refine utils.py * refine collate_fn * fix bug in eval.py * refine codes * refine codes * modify atol from 1e-8 to 1e-7 of UT test_navierstokes * refine code * add AMGNet document * fix * fix * avoid tensor converion in dataset, and move in to collate_fn * update final code * add example for AMGNet * fix doc

HydrogenSulfate added 2 commits September 20, 2023 03:12

update WIP code

de89025

(WIP)update AMGNet code

38659ca

HydrogenSulfate mentioned this pull request Sep 22, 2023

【快乐开源】迁移 AMGNet 案例至 PaddleScience #448

Closed

HydrogenSulfate force-pushed the add_AMGNet branch from 7b71329 to 38659ca Compare September 22, 2023 03:42

HydrogenSulfate added 7 commits September 22, 2023 04:37

try import pgl to avoid importerror

24d4d09

try import pyamg to avoid importerror

f1bd744

add airfoil_dataset.py

83f707c

add type checking for amgnet

e770842

try import pgl to avoid importerror

e773e20

refine Timer

fa2adda

replace pgl.Dataset with io.Dataset

c524a27

HydrogenSulfate force-pushed the add_AMGNet branch from b8aedc3 to 1254dc6 Compare September 26, 2023 03:03

update reproded code

66954a6

HydrogenSulfate force-pushed the add_AMGNet branch from 1254dc6 to 66954a6 Compare September 26, 2023 03:26

HydrogenSulfate added 6 commits September 26, 2023 03:27

replace ImportError with ModuleNotFoundError

ac13693

refine amgnet.py

f0ae844

refine amgnet_airfoil.py and amgnet_cylinder.py

283301c

refine utils.py

8fee479

refine collate_fn

f242756

fix bug in eval.py

031f210

HydrogenSulfate force-pushed the add_AMGNet branch from c4bc77b to 031f210 Compare September 26, 2023 06:33

HydrogenSulfate changed the title ~~[WIP] Add AMGNet example~~ [WIP][Example] Add AMGNet example Sep 26, 2023

HydrogenSulfate changed the title ~~[WIP][Example] Add AMGNet example~~ [Example] Add AMGNet example Sep 27, 2023

zhiminzhang0830 reviewed Sep 27, 2023

View reviewed changes

HydrogenSulfate added 3 commits October 8, 2023 03:05

refine codes

781d217

refine codes

2b8b754

modify atol from 1e-8 to 1e-7 of UT test_navierstokes

c3e560a

HydrogenSulfate force-pushed the add_AMGNet branch from c672678 to c3e560a Compare October 8, 2023 08:57

refine code

997fa3f

HydrogenSulfate force-pushed the add_AMGNet branch from 3797afe to 451491a Compare October 9, 2023 05:42

add AMGNet document

3a6e985

HydrogenSulfate force-pushed the add_AMGNet branch from 451491a to 3a6e985 Compare October 9, 2023 06:22

HydrogenSulfate added 2 commits October 11, 2023 06:40

fix

d593755

fix

11c803d

HydrogenSulfate force-pushed the add_AMGNet branch from 6a41855 to 11c803d Compare October 11, 2023 08:52

HydrogenSulfate added 3 commits October 17, 2023 02:55

avoid tensor converion in dataset, and move in to collate_fn

2381290

Merge branch 'develop' into add_AMGNet

a59bc28

update final code

7eb4aab

HydrogenSulfate force-pushed the add_AMGNet branch from 544e73c to 98f5a8a Compare October 18, 2023 05:41

add example for AMGNet

876ef4d

HydrogenSulfate force-pushed the add_AMGNet branch from 98f5a8a to 876ef4d Compare October 18, 2023 05:43

HydrogenSulfate mentioned this pull request Oct 18, 2023

🎁PaddleScience 快乐开源活动表单 #379

Open

zhiminzhang0830 reviewed Oct 19, 2023

View reviewed changes

HydrogenSulfate added 2 commits October 19, 2023 13:03

fix doc

9c04c5b

Merge branch 'develop' into add_AMGNet

a55ae30

zhiminzhang0830 approved these changes Oct 23, 2023

View reviewed changes

zhiminzhang0830 merged commit 59ca3a1 into PaddlePaddle:develop Oct 23, 2023
4 checks passed

HydrogenSulfate deleted the add_AMGNet branch October 26, 2023 13:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Example] Add AMGNet example #549

[Example] Add AMGNet example #549

HydrogenSulfate commented Sep 22, 2023 •

edited

Loading

paddle-bot bot commented Sep 22, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Sep 27, 2023

HydrogenSulfate Oct 8, 2023

zhiminzhang0830 Oct 19, 2023

HydrogenSulfate Oct 19, 2023

zhiminzhang0830 Oct 19, 2023

HydrogenSulfate Oct 19, 2023

zhiminzhang0830 Oct 19, 2023

HydrogenSulfate Oct 19, 2023

zhiminzhang0830 left a comment

		return output


		def getcorsenode(latent_graph: "pgl.Graph") -> paddle.Tensor:


		- AMGNET 通过 RS 算法(Olson and Schroder, 2018)进行了图的粗化，仅使用少量节点即可进行预测，进一步提高了预测速度。

		下图为该方法的网络结构图。该模型的基本原理就是将网格结构转化为图结构，然后通过网格中节点的物理信息，位置信息以及节点类型对图中的节点和边进行编码。接着对得到的图神经网络使用基于代数多重网格算法(RS)的粗化层进行粗化，将所有节点分类为粗节点集和细节点集，其中粗节点集是细节点集的子集。粗图的节点集合就是粗节点集，于是完成了图的粗化，缩小了图的规模。粗化完成后通过设计的图神经网络信息传递块(GN)来总结和提取图的特征。之后图恢复层采用反向操作，使用空间插值法(Qi et al.,2017)对图进行上采样。例如要对节点 $i$ 插值，则在粗图中找到距离节点 $i$ 最近的 $k$ 个节点，然后通过公式计算得到节点 $i$ 的特征。最后，通过解码器得到每个节点的速度与压力信息。


		### 3.6 评估器构建

		在训练过程中通常会按一定轮数间隔，用验证集(测试集)评估当前模型的训练情况，因此使用 `ppsci.validate.SupervisedValidator` 构建评估器，构建过程与 [约束构建](#34) 类似，只需把数据目录改为测试集的目录，并在配置文件中设置 `EVAL.batch_size=1` 即可。

[Example] Add AMGNet example #549

[Example] Add AMGNet example #549

Conversation

HydrogenSulfate commented Sep 22, 2023 • edited Loading

PR types

PR changes

Describe

paddle-bot bot commented Sep 22, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiminzhang0830 left a comment

Choose a reason for hiding this comment

HydrogenSulfate commented Sep 22, 2023 •

edited

Loading