"NoneType object is not callable" on stopping P2P

[slush0]

I have very simple inference testing script. No threading or any advanced stuff. Basically "hello world" inference on Petals. Everything is going well, but when the script is exiting, I always get this error:

Exception ignored in: <function P2P.__del__ at 0x7f4ac1feed40>
Traceback (most recent call last):
  File "/home/dev/.local/lib/python3.10/site-packages/hivemind/p2p/p2p_daemon.py", line 632, in __del__
  File "/home/dev/.local/lib/python3.10/site-packages/hivemind/p2p/p2p_daemon.py", line 659, in _terminate
  File "/home/dev/.local/lib/python3.10/site-packages/multiaddr/multiaddr.py", line 254, in value_for_protocol
TypeError: 'NoneType' object is not callable

It is rather cosmetic issue, but something is not OK there.

[borzunov]

@slush0 Thanks for reporting! That's a bug in hivemind, I'll it fix it :)

[slush0]

For report clarity, this is the failing script.

$ cat test2.py 
#!/usr/bin/env python
import torch
from transformers import BloomTokenizerFast
from petals import DistributedBloomForCausalLM

dev = 'cpu'
prompt = """Lorem ipsum dolor sit amet"""
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloomz-petals").to(dev)
tokenizer = BloomTokenizerFast.from_pretrained("bigscience/bloomz-petals")
inputs = tokenizer(prompt, return_tensors="pt").input_ids.to(dev)
outputs = model.generate(inputs,
            max_new_tokens=40,
            temperature=0.00001,
            repetition_penalty=1.2)
print(tokenizer.decode(outputs[0]))

[borzunov]

Update: I still haven't fixed this bug, since I didn't understand its cause after a short investigation and switch to tasks with higher priority. I'd appreciate if someone (including new contributors) looks into it.

[borzunov]

@slush0 I can't reproduce this bug in the latest versions of hivemind and Petals, I assume it could have been resolved in hivemind/Petals or their dependencies. I'll close this issue, but please let me know if this bug still happens in your environment (even when you update hivemind & Petals).

[borzunov]

Please also note that the repetition_penalty argument was ignored in our code due to a bug. Actually, this parameter is not implemented yet in the main branch of Petals. Please let me know if you need it, since I have a separate (almost finished) branch with this parameter (if you do, I'll finish and merge it).

[borzunov]

The issue is still relevant: #368 (comment)

[borzunov]

Fixing it in learning-at-home/hivemind#579

[ivangabriele]

Understood! Yes from what I understand there is no concept of multi-version installation in pip.

Thank you for the quick reply and PR, I'll wait for the merge and next release of hivemind then ^^.

[borzunov]

@ivangabriele You don't have to wait for the next release, since the Petals main branch depends on the latest master branch of hivemind.

I've merged the hivemind fix, so now you can just upgrade petals and the bug should be gone:

pip install --upgrade git+https://github.com/bigscience-workshop/petals

Let us know if you meet any other issues!

[ivangabriele]

@borzunov I didn't reply yesterday but I just confirm here that your hivemind deps change fixed my issue ^^.