Reland: Fix CUDA device guard usage when first arg of kernel is scalar

[kurtamohler]

Reland PR #39870

Closes #38889

[dr-ci]

💊 CI failures summary and remediations

As of commit 928ac64 (more details on the Dr. CI page):

---

• 1/3 failures possibly* introduced in this PR
  • 1/1 non-CircleCI failure(s)
• 2/3 broken upstream at merge base a9aa636 on Jun 12 from 12:36pm to 1:40pm PDT (2 commits; a9aa636 - 8bc821f)

---

🚧 2 fixed upstream failures:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

Since your merge base is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.

• pytorch_windows_vs2019_py36_cpu_build on Jun 12 from 12:36pm to 1:40pm PDT (2 commits; a9aa636 - 8bc821f)
  • 🔁 rerun
• pytorch_windows_vs2019_py36_cuda10.1_build on Jun 12 from 12:36pm to 1:40pm PDT (2 commits; a9aa636 - 8bc821f)
  • 🔁 rerun

---

ci.pytorch.org: 1 failed

• Failed: pr/caffe2-pytorch-linux-xenial-rocm3.3-py3.6-test

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 12 times.

[kurtamohler]

Turns out that I'm getting test failures on TestTorchDeviceTypeCUDA.test_serialization_cuda only if I run all of test_torch.py. If I run just that failing test, it doesn't fail.

[ngimel]

Yeah, that happens because you are changing device in your test and don't set it back. You can either explicitly set the device back to original, or, better, use with torch.cuda.device context manager.
so there are 2 problems here

1. the failing test is poorly written, it should not rely on global state
2. your test should not change global state.
   It's ok to fix only 2).

[kurtamohler]

Oh right, that makes sense.

[facebook-github-bot]

@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@ngimel merged this pull request in db2b273.