Updating test code to follow single iterator constraint

[NivekT]

Stack from ghstack:

• Moving DataPipe buffers from __iter__ to instance (self) #388
• -> Updating test code to follow single iterator constraint #386

This should be landed after pytorch/pytorch#75995

Note that the CI is expected to fail until that PR goes into nightly

Differential Revision: D36182556

[NivekT]

@NivekT has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[ejguan]

It seems PyTorch nightly has been updated. This PR can be landed otherwise all DataPipe tests become red.

[NivekT]

Yea, but the issue with GHA is that even though nightly has landed, the internal diff hasn't, and my diff (D36182556) is dependent on the internal one landing first.

The options are:

1. Remove the dependencies from D36182556 and land this (it won't pass the internal test, because the other stack hasn't landed yet)
2. Wait for the other stack of diffs to land, then land this one (D36182556)

[ejguan]

Oh. I don't realize such problem, good to know. Let's wait until internal Diff landed. (Decoupling is more important now)

[ejguan]

If the whole stack of PRs in PT Core got reverted because the failed internal tests for TorchData, you might need to combine this PR with the PR in PyTorch core in a single Diff and land it from internal Phabricator.

[NivekT]

@NivekT has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.