Skip to content

Commit

Permalink
fix profiler by qa (#667)
Browse files Browse the repository at this point in the history
  • Loading branch information
kuizhiqing authored Aug 29, 2022
1 parent c93a978 commit 99cd693
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 5 deletions.
6 changes: 4 additions & 2 deletions examples/gpt/hybrid_parallel/profiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,18 +68,20 @@ python -m paddle.distributed.launch run_pretrain.py -c ./configs_1.3B_dp8.yaml -
## 结果分析

默认情况下会在训练结束后会有以下数据
在训练结束后会有以下数据

* 根据配置信息在控制台打印 summary 表格
* 在配置的 `profiler_log` 目录保存 profiler json 文件

这里保存的 json 文件可以通过如下两种方式查看:

* 在 chrome 浏览器中打开 chrome://tracing/,然后打开 json 文件查看
* 根据控制台信息安装并启动 `visualdl --logdir log_path` 然后根据提示在浏览器中查看
* 根据控制台信息安装并启动 `visualdl --logdir log_path` 然后根据提示在浏览器中**性能分析**模块查看

具体的信息含义解释以及分析方法请参考[文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/performance_improving/profiling_model.html)

> 在使用 visualdl 时,如果 log 文件数据较大,启动会比较耗时,请耐心等待。
## 附录

控制台打印的 summary 信息示例如下所示。
Expand Down
11 changes: 8 additions & 3 deletions fleetx/core/engine/eager_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -607,17 +607,22 @@ def _print_summary(self):
SummaryView.UDFView : 'udf',
}

default_views = [
SummaryView.OverView,
SummaryView.ModelView,
SummaryView.KernelView,
SummaryView.OperatorView,
]
def gen_views(cfg):
# print all summary view if detailed=True
if self.profiler_config.get('detailed', False):
return None

default_views = ['overview', 'model', 'kernel', 'op']
views = []
# override default view with user defined value if detailed=False
for view in SummaryView:
v = self.profiler_config.get('summary', {}).get(views_dict[view], None)
if v is True or (v is None and v in default_views):
if v is True or (v is None and view in default_views):
views.append(view)

return views or None
Expand All @@ -637,5 +642,5 @@ def _profiler_done(self):
profiler_log = self.profiler_config.get('profiler_log', './profiler_log')
print("For more information please install visualdl and run it with following command:")
print("-------------------------------------------------------------------------------")
print(f"visualdl --logdir --host 0.0.0.0 {profiler_log}")
print(f"visualdl --host 0.0.0.0 --logdir {profiler_log}")
print("-------------------------------------------------------------------------------")

0 comments on commit 99cd693

Please sign in to comment.