[JIT] Optimize before inlining #35562

eellison · 2020-03-27T19:43:00Z

Resubmit of #35424, only this time I run optimizations in the right order so the PR description is actually true.

This speeds up the inlining pass of FairSeq model from 180s -> 13s, and MaskRCNN model from 5s -> 1.5s.

…mize_inlining

dr-ci · 2020-03-27T19:45:54Z

💊 CircleCI build failures summary and remediations

As of commit c51bfc1 (more details on the Dr. CI page):

✅ None of the build failures appear to be your fault 💚

4/4 broken upstream at merge base f421cf3 on Apr 06 from 2:27pm to 6:52pm PDT (15 commits; 8ef82fc - 4c14005)
Please rebase on the viable/strict branch (expand for instructions)

If your commit is newer than viable/strict, you can try basing on an older, stable commit:
```
git fetch https://github.com/pytorch/pytorch viable/strict
git rebase --onto FETCH_HEAD $(git merge-base origin/master HEAD)
```
If your commit is older than viable/strict:
```
git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD
```
Check out the recency history of this "viable master" tracking branch.

🚧 4 upstream failures:

These were probably caused by upstream breakages:

caffe2_onnx_main_py3_6_clang7_ubuntu16_04_test on Apr 06 from 2:27pm to 6:52pm PDT (15 commits; 8ef82fc - 4c14005)
caffe2_onnx_ort1_py3_6_clang7_ubuntu16_04_test since Apr 06
caffe2_onnx_ort2_py3_6_clang7_ubuntu16_04_test since Apr 06
pytorch_windows_vs2019_py36_cuda10.1_test1 on Apr 06 from 2:27pm to 7:37pm PDT (18 commits; 8ef82fc - 459163b)

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 88 times.

ZolotukhinM

Looks good!

…mize_inlining

facebook-github-bot

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

…mize_inlining

facebook-github-bot

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

BowenBao · 2020-03-31T18:35:30Z

Edit:
Same error as #35401. @houseroad

Mar 31 01:07:01     def test_dim(self):
Mar 31 01:07:01         class DimModel(torch.jit.ScriptModule):
Mar 31 01:07:01             @torch.jit.script_method
Mar 31 01:07:01             def forward(self, input):
Mar 31 01:07:01                 out = input * 2
Mar 31 01:07:01                 out *= out.dim()
Mar 31 01:07:01                 return out
Mar 31 01:07:01         empty_input = torch.randn(0, requires_grad=True)
Mar 31 01:07:01         multi_dim_input = torch.randn(1, 2, 3, requires_grad=True)
Mar 31 01:07:01         self.run_test(DimModel(), empty_input)
Mar 31 01:07:01 >       self.run_test(DimModel(), multi_dim_input)
Mar 31 01:07:01 
Mar 31 01:07:01 test/onnx/test_pytorch_onnx_onnxruntime.py:2684: 
Mar 31 01:07:01 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Mar 31 01:07:01 test/onnx/test_pytorch_onnx_onnxruntime.py:126: in run_test
Mar 31 01:07:01     _run_test(model)
Mar 31 01:07:01 test/onnx/test_pytorch_onnx_onnxruntime.py:122: in _run_test
Mar 31 01:07:01     fixed_batch_size=fixed_batch_size)
Mar 31 01:07:01 test/onnx/test_pytorch_onnx_onnxruntime.py:86: in run_model_test
Mar 31 01:07:01     ort_test_with_input(ort_sess, input_copy, output, rtol, atol)
Mar 31 01:07:01 test/onnx/test_pytorch_onnx_onnxruntime.py:47: in ort_test_with_input
Mar 31 01:07:01     [np.testing.assert_allclose(out, ort_out, rtol=rtol, atol=atol) for out, ort_out in zip(outputs, ort_outs)]
Mar 31 01:07:01 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Mar 31 01:07:01 
Mar 31 01:07:01 .0 = <zip object at 0x7f3d38393d48>
Mar 31 01:07:01 
Mar 31 01:07:01 >   [np.testing.assert_allclose(out, ort_out, rtol=rtol, atol=atol) for out, ort_out in zip(outputs, ort_outs)]
Mar 31 01:07:01 E   AssertionError: 
Mar 31 01:07:01 E   Not equal to tolerance rtol=0.001, atol=1e-07
Mar 31 01:07:01 E   
Mar 31 01:07:01 E   Mismatched elements: 6 / 6 (100%)
Mar 31 01:07:01 E   Max absolute difference: 8.715158
Mar 31 01:07:01 E   Max relative difference: 0.6666667
Mar 31 01:07:01 E    x: array([[[ 3.081992, -0.586858, -4.357579],
Mar 31 01:07:01 E           [ 1.136863, -2.169045, -2.797191]]], dtype=float32)
Mar 31 01:07:01 E    y: array([[[  9.245976,  -1.760573, -13.072737],
Mar 31 01:07:01 E           [  3.410588,  -6.507134,  -8.391573]]], dtype=float32)

eellison · 2020-03-31T18:39:16Z

@BowenBao how do you recommend I proceed ? as far as I can tell the test is not failing on master.

BowenBao · 2020-03-31T18:45:26Z

@eellison yea I checked a few other PRs and they don't seem to fail on this one, so I edited my previous comment. One thing bothers me is that you mentioned in the other PR that you could not repro this error locally. Let me change the randn in the failed test_dim to ones, this should help debugging.

eellison · 2020-03-31T18:49:42Z

@BowenBao could you check out the PR ?

…mize_inlining

facebook-github-bot

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

eellison · 2020-04-03T18:52:07Z

@sanekmelnikov @J0Nreynolds I'm getting a tensorboard test failure here. i'm having trouble running the test for unrelated reasons

…mize_inlining

facebook-github-bot

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

…mize_inlining

facebook-github-bot

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

…mize_inlining

facebook-github-bot

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2020-04-07T18:14:35Z

@eellison merged this pull request in 6bc8ffe.

Summary: Resubmit of pytorch#35424, only this time I run optimizations in the right order so the PR description is actually true. This speeds up the inlining pass of FairSeq model from 180s -> 13s, and MaskRCNN model from 5s -> 1.5s. Pull Request resolved: pytorch#35562 Differential Revision: D20738922 Pulled By: eellison fbshipit-source-id: 1439cf9d1f0bc780e2d64a744694f8b3b7ba4b70

Summary: With #35562, we are running peephole optimization on inlining to reduce the number of nodes that are copied. The tracer encodes the sizes in the graph like: ``` graph(%0 : Double(7)): %1 : Function = prim::Constant[name="tensor_size"]() %2 : Tensor = prim::CallFunction(%1, %0) return (%2) ``` however people would like to reuse the graph with different shapes so running size invalidations would invalidate that. long term it might be better for the tracer to not include shape information but there are downstream users of that. Separates out FuseAddMM from peephole so that now there is a single `disable_size_optimizations` parameter, and onnx explicitly invokes fuseaddmm. Pull Request resolved: #36404 Differential Revision: D20968974 Pulled By: eellison fbshipit-source-id: 56f8f1699e3b0adeeccdfd5a67bb975fd41a2913

inline then optimize

3cacf0d

eellison requested a review from ZolotukhinM March 27, 2020 19:43

eellison requested a review from apaszke as a code owner March 27, 2020 19:43

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Mar 27, 2020

Merge branch 'master' of https://github.com/pytorch/pytorch into opti…

ff25643

…mize_inlining

ZolotukhinM approved these changes Mar 27, 2020

View reviewed changes

eellison added 2 commits March 27, 2020 14:02

Merge branch 'master' of https://github.com/pytorch/pytorch into opti…

e46ce18

…mize_inlining

add constant pooling

75ae30b

facebook-github-bot reviewed Mar 30, 2020

View reviewed changes

Merge branch 'master' of https://github.com/pytorch/pytorch into opti…

47d7e85

…mize_inlining

facebook-github-bot reviewed Mar 30, 2020

View reviewed changes

eellison added 2 commits March 31, 2020 12:40

fix test

bad13fb

Merge branch 'master' of https://github.com/pytorch/pytorch into opti…

c7aabfb

…mize_inlining

eellison force-pushed the optimize_inlining branch from af6ceaf to c7aabfb Compare April 3, 2020 16:49

facebook-github-bot reviewed Apr 3, 2020

View reviewed changes

eellison added 2 commits April 6, 2020 10:06

Merge branch 'master' of https://github.com/pytorch/pytorch into opti…

43a1300

…mize_inlining

fix expect test

2da8c0f

facebook-github-bot reviewed Apr 6, 2020

View reviewed changes

Merge branch 'master' of https://github.com/pytorch/pytorch into opti…

c780386

…mize_inlining

facebook-github-bot reviewed Apr 6, 2020

View reviewed changes

Merge branch 'master' of https://github.com/pytorch/pytorch into opti…

c51bfc1

…mize_inlining

facebook-github-bot reviewed Apr 7, 2020

View reviewed changes

facebook-github-bot closed this in 6bc8ffe Apr 7, 2020

facebook-github-bot added the merged label Apr 7, 2020

eellison mentioned this pull request Apr 10, 2020

[JIT] Dont optimize shape peepholes on inline #36404

Closed

ezyang mentioned this pull request Apr 11, 2020

Intermittent hang in BuildLivenessSets after test_svd_cuda_float64 on CUDA 10.1 configurations #36434

Closed

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JIT] Optimize before inlining #35562

[JIT] Optimize before inlining #35562

eellison commented Mar 27, 2020

dr-ci bot commented Mar 27, 2020 •

edited

Loading

ZolotukhinM left a comment

facebook-github-bot left a comment

facebook-github-bot left a comment

BowenBao commented Mar 31, 2020 •

edited

Loading

eellison commented Mar 31, 2020

BowenBao commented Mar 31, 2020

eellison commented Mar 31, 2020

facebook-github-bot left a comment

eellison commented Apr 3, 2020

facebook-github-bot left a comment

facebook-github-bot left a comment

facebook-github-bot left a comment

facebook-github-bot commented Apr 7, 2020

[JIT] Optimize before inlining #35562

[JIT] Optimize before inlining #35562

Conversation

eellison commented Mar 27, 2020

dr-ci bot commented Mar 27, 2020 • edited Loading

💊 CircleCI build failures summary and remediations

🚧 4 upstream failures:

ZolotukhinM left a comment

Choose a reason for hiding this comment

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot left a comment

Choose a reason for hiding this comment

BowenBao commented Mar 31, 2020 • edited Loading

eellison commented Mar 31, 2020

BowenBao commented Mar 31, 2020

eellison commented Mar 31, 2020

facebook-github-bot left a comment

Choose a reason for hiding this comment

eellison commented Apr 3, 2020

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot commented Apr 7, 2020

dr-ci bot commented Mar 27, 2020 •

edited

Loading

BowenBao commented Mar 31, 2020 •

edited

Loading