[Bug] TuneReportCallback function uses Pytorch trainer attribute that has been deprecated

[dobbali]

Search before asking

• I searched the issues and found no similar issues.

Ray Component

Ray Tune

What happened + What you expected to happen

Ray Tune throws an Attribute error when TuneReportCallback is used as a callback in Pytorch Lightning Trainer function.

AttributeError: 'Trainer' object has no attribute 'running_sanity_check'

This happens as Trainer.running_sanity_checkis deprecated in favor of Trainer.sanity_checking in the latest version of Pytorch

Here are related Pytorch PRs:
Lightning-AI/pytorch-lightning#9209
Lightning-AI/pytorch-lightning#4945

Full Traceback :

Traceback (most recent call last):
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/trial_runner.py", line 739, in _process_trial
    results = self.trial_executor.fetch_result(trial)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/ray_trial_executor.py", line 746, in fetch_result
    result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 82, in wrapper
    return func(*args, **kwargs)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/worker.py", line 1621, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(TuneError): ray::ImplicitFunc.train_buffered() (pid=85834, ip=192.168.1.112, repr=<types.ImplicitFunc object at 0x1d57781f0>)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/trainable.py", line 178, in train_buffered
    result = self.train()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/trainable.py", line 237, in train
    result = self.step()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/function_runner.py", line 379, in step
    self._report_thread_runner_error(block=True)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/function_runner.py", line 526, in _report_thread_runner_error
    raise TuneError(
ray.tune.error.TuneError: Trial raised an exception. Traceback:
ray::ImplicitFunc.train_buffered() (pid=85834, ip=192.168.1.112, repr=<types.ImplicitFunc object at 0x1d57781f0>)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/function_runner.py", line 260, in run
    self._entrypoint()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/function_runner.py", line 328, in entrypoint
    return self._trainable_func(self.config, self._status_reporter,
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/function_runner.py", line 594, in _trainable_func
    output = fn()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/utils/trainable.py", line 353, in _inner
    inner(config, checkpoint_dir=None)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/utils/trainable.py", line 344, in inner
    trainable(config, **fn_kwargs)
  File "/Users/manoj.dobbali/Projects/ray/dco-research/dco/model_arch_lgt.py", line 196, in train_dco
    trainer.fit(model)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 737, in fit
    self._call_and_handle_interrupt(
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 682, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 772, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1195, in _run
    self._dispatch()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1275, in _dispatch
    self.training_type_plugin.start_training(self)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 202, in start_training
    self._results = trainer.run_stage()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1285, in run_stage
    return self._run_train()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1307, in _run_train
    self._run_sanity_check(self.lightning_module)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1371, in _run_sanity_check
    self._evaluation_loop.run()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 151, in run
    output = self.on_run_end()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 140, in on_run_end
    self._on_evaluation_end()
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 202, in _on_evaluation_end
    self.trainer.call_hook("on_validation_end", *args, **kwargs)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1491, in call_hook
    callback_fx(*args, **kwargs)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/pytorch_lightning/trainer/callback_hook.py", line 221, in on_validation_end
    callback.on_validation_end(self, self.lightning_module)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/integration/pytorch_lightning.py", line 115, in on_validation_end
    self._handle(trainer, pl_module)
  File "/Users/manoj.dobbali/Library/Caches/pypoetry/virtualenvs/dco-dUEv40jP-py3.8/lib/python3.8/site-packages/ray/tune/integration/pytorch_lightning.py", line 174, in _handle
    if trainer.running_sanity_check:
AttributeError: 'Trainer' object has no attribute 'running_sanity_check'

depreciated

Versions / Dependencies

Python 3.8.8
pytorch-lightning = "^1.5.5"
ray = {extras = ["tune"], version = "^1.9.0"}

Reproduction script

As described

Anything else

Every time

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[amogkam]

@dobbali the integration only works with PTL 1.4 or lower for now. We are currently working on support for PTL 1.5!

Closing this as duplicate of #20741