BUG: fails to collect if a global variable is not __iter__able

[NickCrews]

Not sure if this is the exact root cause or not.

First, pip install ibis-framework.
Then, in a task file have from ibis import _

This results in

────────────────────────────────────────────────── Could not collect tasks/task_other.py ───────────────────────────────────────────────────

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/nc/code/scg/kobuk/.venv/lib/python3.11/site-packages/ibis/common/deferred.py:96 in        │
│ __iter__                                                                                         │
│                                                                                                  │
│    93 │   │   return Deferred(Attr(self, name))                                                  │
│    94 │                                                                                          │
│    95 │   def __iter__(self):                                                                    │
│ ❱  96 │   │   raise TypeError(f"{self.__class__.__name__!r} object is not iterable")             │
│    97 │                                                                                          │
│    98 │   def __getitem__(self, name):                                                           │
│    99 │   │   return Deferred(Item(self, name))                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: 'Deferred' object is not iterable

however, I tried to repro with putting this in the task file, and this does NOT error:

class NonIterable:
    def __iter__(self):
        raise TypeError("NonIterable is not iterable")


bad = NonIterable()

so not exactly sure the root cause.

[tobiasraabe]

It is a funny problem.

The problem is that pytask will check each symbol in a module while collecting tasks. One of the checks is whether something has a @task decorator and this function is responsible or, more importantly, this line.

Since _ probably overwrote __getattr__, it has all attributes, and so the protocol check will be true, and isclass(_) is False.

We need to find a way to guard pytask from these objects better.

[tobiasraabe]

Some notes about what I found out.

• from ibis import _ indeed overwrites __getattr__ and allows every attribute to be created like from pytask import mark.
• In Python <= 3.11, any isinstance(_, PTask) check will return True because isinstance checks for protocols are imperfect and call getattr on _ which will yield an attribute.
• In Python >= 3.12, the call was changed from hasattr to inspect.getattr_static and now the isinstance check returns False link
• The issue in CPython https://github.com/python/cpython/issues/102433.
• It is backported to typing_extensions, but I do not want to pin the package to >=4.6.0. lik: https://github.com/python/typing_extensions/pull/140.