Some tests are not skipped properly

[yangarbiter]

🐛 Bug

The bug is discovered in this discussion. These tests (kaldi_io_test.py, backend/soundfile/load_test.py, backend/soundfile/save_test.py) should be skipped if soundfile or kaldi_io are not installed. But instead these tests just Failed.

To Reproduce

Steps to reproduce the behavior:

1. conda create -n temp -c pytorch-nightly pytorch
2. conda activate temp
3. git clone --recursive https://github.com/pytorch/audio
4. cd audio
5. git submodule update --init --recursive
6. pip install typing pytest scipy numpy parameterized pyyaml
7. NO_CUDA=1 BUILD_SOX=1 MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
8. cd test/torchaudio_unittest
9. pytest kaldi_io_test.py
10. pytest backend/soundfile/load_test.py
11. pytest backend/soundfile/save_test.py

Expected behavior

The related tests should be skipped if soundfile or kaldi_io are not installed.

Environment

• What commands did you used to install torchaudio (conda/pip/build from source)?
  Please refer to the section To Reproduce.
• If you are building from source, which commit is it?
  7deea25
• What does torchaudio.__version__ print? (If applicable)
  '0.10.0a0+7deea25'
  Collecting environment information...
  PyTorch version: 1.10.0.dev20210618
  Is debug build: False
  CUDA used to build PyTorch: None
  ROCM used to build PyTorch: N/A

OS: macOS 11.4 (x86_64)
GCC version: Could not collect
Clang version: 12.0.5 (clang-1205.0.22.9)
CMake version: version 3.20.4
Libc version: N/A

Python version: 3.9 (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.3
[pip3] torch==1.10.0.dev20210618
[pip3] torchaudio==0.10.0a0+7deea25
[conda] blas 1.0 mkl
[conda] mkl 2021.2.0 hecd8cb5_269
[conda] numpy 1.20.3 pypi_0 pypi
[conda] pytorch 1.10.0.dev20210618 py3.9_0 pytorch-nightly
[conda] torchaudio 0.10.0a0+7deea25 pypi_0 pypi

Additional context

CC @mthrok @vincentqb @carolineechen

[mthrok]

for the kaldi_io test, the test class needs to be decorated with skipIf.

audio/test/torchaudio_unittest/kaldi_io_test.py

Line 7 in 533c2d4

class Test_KaldiIO(common_utils.TorchaudioTestCase):

Example:

audio/test/torchaudio_unittest/functional/librosa_compatibility_test_impl.py

Line 10 in 533c2d4

LIBROSA_AVAILABLE = is_module_available('librosa')

audio/test/torchaudio_unittest/functional/librosa_compatibility_test_impl.py

Lines 25 to 26 in 533c2d4

	@unittest.skipIf(not LIBROSA_AVAILABLE, "Librosa not available")
	class Functional(TestBaseMixin):

[mthrok]

For soundfile test, the failing ones are MockedTest and it fails after the test when mock is try to recover soundfile name, which is not imported in case soundfile is not installed.

A solution will be to use magic mock to define something as soundfile when it's not available.

Adding something like

else:
    soundfile = MagicMock()

In the following,

audio/test/torchaudio_unittest/backend/soundfile/save_test.py

Lines 22 to 23 in 533c2d4

	if _mod_utils.is_module_available("soundfile"):
	import soundfile

[yangarbiter]

I added SkipIF in kaldi_io_test.py and it works well.

I added

else:
    soundfile = MagicMock()

in load_test.py and save_test.py, but it does not seem to work.

It seems that in this line and this line, they will try to import soundfile, and there would be an error (like the following).

a = (<torchaudio_unittest.backend.soundfile.save_test.MockedSaveTest testMethod=test_sph_nist_int32_8000_1_True_PCM_S_32>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../miniconda3/envs/temp/lib/python3.9/site-packages/parameterized/parameterized.py:533:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
torchaudio_unittest/backend/soundfile/save_test.py:133: in test_sph
    self.assert_non_wav(fmt, dtype, sample_rate, num_channels,
../../../../miniconda3/envs/temp/lib/python3.9/unittest/mock.py:1334: in patched
    with self.decoration_helper(patched,
../../../../miniconda3/envs/temp/lib/python3.9/contextlib.py:117: in __enter__
    return next(self.gen)
../../../../miniconda3/envs/temp/lib/python3.9/unittest/mock.py:1316: in decoration_helper
    arg = exit_stack.enter_context(patching)
../../../../miniconda3/envs/temp/lib/python3.9/contextlib.py:429: in enter_context
    result = _cm_type.__enter__(cm)
../../../../miniconda3/envs/temp/lib/python3.9/unittest/mock.py:1389: in __enter__
    self.target = self.getter()
../../../../miniconda3/envs/temp/lib/python3.9/unittest/mock.py:1564: in <lambda>
    getter = lambda: _importer(target)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

target = 'soundfile'

    def _importer(target):
        components = target.split('.')
        import_path = components.pop(0)
>       thing = __import__(import_path)
E       ModuleNotFoundError: No module named 'soundfile'

../../../../miniconda3/envs/temp/lib/python3.9/unittest/mock.py:1236: ModuleNotFoundError