Metrics that call NLTK.download cause lightning to hang in offline environments

[JohnGiorgi]

🐛 Bug

Any metric that calls nltk.download is going to cause an issue in an offline environment, as the command will just hang. Examples include:

https://github.com/Lightning-AI/metrics/blob/6ae08a807fbd36976574bbffcf94140fce5564d6/src/torchmetrics/functional/text/rouge.py#L48

This actually extends to other Lightning packages as well, like Flash:

https://github.com/Lightning-AI/lightning-flash/blob/48a25006cad455672063dc5dc9141c43fcf6607d/flash_examples/question_answering.py#L23

I realized this when trying to run a Lightning Flash script on one of my clusters (offline) compute nodes, and my code simply hung before training.

To Reproduce

Just call nlkt.download in any offline environment, e.g.

Code sample

import nltk
# This will hang even if you have already downloaded the punkt data while online...
nltk.download("punkt")

Expected behavior

Ideally, if the required nltk data is already downloaded, nltk.download would not be called so that lightning will not hang.

Environment

• TorchMetrics version: torchmetrics==0.10.3 from pip
• Python & PyTorch Version (e.g., 1.0): Python 3.10, Torch 1.13.1
• Any other relevant information such as OS (e.g., Linux): Linux

Additional context

I think the simplest fix would be to replace the nltk.download commands with a try/except, e.g.

try:
    nltk.word_tokenize("Check if punkt data is available")
except LookupError:
    nltk.download("punkt", quiet=True, force=False)

at the very least, this would allow someone to call nltk.download first in an online environment (e.g. on a login node) before running some lightning code in an offline environment (e.g. a compute node).

I think even better would be to log a message here, warning a user that a download is being attempted and that if they are offline they will need to download the data manually while online using nltk.download(...).

Would be happy to PR this if maintainers agree.

[github-actions]

Hi! thanks for your contribution!, great first issue!