Workaround lru_cache decorator on method garbage collection problem

[MetRonnie]

These changes close #330

@lru_cache, if used on a method, retains the class instance beyond the point it would normally be garbage collected. This workaround is from https://stackoverflow.com/a/68550238/3217306

This PR also includes some minor GH Actions test workflow updates and removes pytest-asyncio (see pytest-dev/pytest-asyncio#257 (comment))

Requirements check-list

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Does not need tests
• No change log entry required (invisible to users)
• No documentation update required.
• No core dependency changes.

[codecov-commenter]

Codecov Report

Merging #344 (0617e9c) into master (dcb91cc) will increase coverage by 0.28%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #344      +/-   ##
==========================================
+ Coverage   77.76%   78.05%   +0.28%     
==========================================
  Files          10       10              
  Lines        1048     1048              
  Branches      202      202              
==========================================
+ Hits          815      818       +3     
+ Misses        196      194       -2     
+ Partials       37       36       -1     

Impacted Files	Coverage Δ
cylc/uiserver/authorise.py	89.56% <100.00%> (+0.86%)	⬆️
cylc/uiserver/workflows_mgr.py	82.94% <100.00%> (+0.58%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dcb91cc...0617e9c. Read the comment docs.

[MetRonnie]

MacOS test failure is pretty strange, doesn't seem related to this change, but doesn't fail on master though

[MetRonnie]

Well I don't know why it's resolved itself. Possibly needed asyncio_mode=strict in pytest.ini

However, still getting these printed out after the end of the pytest output

Task was destroyed but it is pending!
task: <Task pending name='Task-57' coro=<WorkflowsManager.run() running at
    /home/runner/work/cylc-uiserver/cylc-uiserver/cylc/uiserver/workflows_mgr.py:435>
    wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f79f6f8ce50>()]>
    cb=[IOLoop.add_future.<locals>.<lambda>() at
    /opt/hostedtoolcache/Python/3.9.12/x64/lib/python3.9/site-packages/tornado/ioloop.py:688]>

[datamel]

Well I don't know why it's resolved itself. Possibly needed asyncio_mode=strict in pytest.ini

However, still getting these printed out after the end of the pytest output

Task was destroyed but it is pending!
task: <Task pending name='Task-57' coro=<WorkflowsManager.run() running at
    /home/runner/work/cylc-uiserver/cylc-uiserver/cylc/uiserver/workflows_mgr.py:435>
    wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f79f6f8ce50>()]>
    cb=[IOLoop.add_future.<locals>.<lambda>() at
    /opt/hostedtoolcache/Python/3.9.12/x64/lib/python3.9/site-packages/tornado/ioloop.py:688]>

I have raised this with @oliver-sanders before, from memory, it only occurs when the python version is >=3.9. See conversation after #324 (review)

[MetRonnie]

It's happening for Python 3.7 on this branch and master: https://github.com/cylc/cylc-uiserver/runs/6160205704?check_suite_focus=true#step:7:80

I saw this comment on #324:

The warnings are due to a regression in jupyter_server:

https://github.com/jupyter-server/jupyter_server/pull/588/files#diff-3cd116aa8a4bea11f638c9209aa3b58e13a3a0aad127d310b0198b4931ca962cR305-R306

Is it worth raising an issue there?

[datamel]

Is it worth raising an issue there?

Looks like they are aware: jupyter-server/jupyter_server#672 (comment)

[oliver-sanders]

Makes sense, this async means tests need an explicit pytest.mark.asyncio? Need to check that relevant tests are decorated correctly otherwise they will be skipped from here on.

[MetRonnie]

Hmm, adding @pytest.mark.asyncio to the async tests that were missing them brings back this error:

FAILED cylc/uiserver/tests/test_auth.py::test_cylc_handler -
    tornado.simple_httpclient.HTTPTimeoutError: Timeout during request

Seems to be only the tests that use the jp_fetch fixture

[MetRonnie]

After a brief look at pytest-tornasync it seems the plugin has a similar behaviour to pytest-asyncio with async_mode=auto: It tries to run all tests that are coroutines in a tornado.IoLoop. When both plugins try to do the same thing, unexpected things happen.

pytest-dev/pytest-asyncio#257 (comment)

[oliver-sanders]

When both plugins try to do the same thing, unexpected things happen.

Makes sense, thanks for investigating this.

[datamel]

I have manually tested this, working well. This was the initial implementation, @oliver-sanders just checking there is not a reason for the requested change, I am struggling to remember... be7d04d#r728858282

[MetRonnie]

#204 (comment) - that was before we realised about the problem of using @lru_cache to decorate methods: https://stackoverflow.com/q/33672412/3217306

[datamel]

#204 (comment) - that was before we realised about the problem of using @lru_cache to decorate methods: https://stackoverflow.com/q/33672412/3217306

Yes, I understand why it is being changed now, I am wondering if there was a specific reason why it was requested to be changed before. As in, was there a problem with that implementation that I can not think of now?

[MetRonnie]

It's a little yukky compared to the cleanliness of simply decorating the method

[datamel]

This branch has conflicts now. If you want to get this rebased I can give it a final check before it goes in @MetRonnie?

[MetRonnie]

Will merge master into this branch rather than rebase - makes the conflict resolution diff easy to see