TypeError: Object of type '<class 'pathlib.PosixPath'>' cannot be converted by tosexp.

[asmeurer]

I tried using emacs-jedi against the git version of jedi, and I am getting this error doing a goto-definition:

ERROR:epc:Unexpected error
Traceback (most recent call last):
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/epc/handler.py", line 244, in _handle
    self._send(*reply)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/epc/utils.py", line 51, in new_method
    ret = method(self, *args, **kwds)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/epc/handler.py", line 222, in _send
    string = encode_message(*args)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/epc/handler.py", line 82, in encode_message
    return encode_object([Symbol(name)] + list(args), **kwds)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/epc/handler.py", line 78, in encode_object
    return encode_string(dumps(obj, **kwds))
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 318, in dumps
    return tosexp(obj, **kwds)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 376, in tosexp
    return Bracket(obj, '(').tosexp(_tosexp)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 502, in tosexp
    c = ' '.join(tosexp(v) for v in self._val)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 502, in <genexpr>
    c = ' '.join(tosexp(v) for v in self._val)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 372, in <lambda>
    _tosexp = lambda x: tosexp(
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 376, in tosexp
    return Bracket(obj, '(').tosexp(_tosexp)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 502, in tosexp
    c = ' '.join(tosexp(v) for v in self._val)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 502, in <genexpr>
    c = ' '.join(tosexp(v) for v in self._val)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 372, in <lambda>
    _tosexp = lambda x: tosexp(
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 400, in tosexp
    return _tosexp(dict_to_plist(obj))
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 372, in <lambda>
    _tosexp = lambda x: tosexp(
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 376, in tosexp
    return Bracket(obj, '(').tosexp(_tosexp)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 502, in tosexp
    c = ' '.join(tosexp(v) for v in self._val)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 502, in <genexpr>
    c = ' '.join(tosexp(v) for v in self._val)
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 372, in <lambda>
    _tosexp = lambda x: tosexp(
  File "/Users/aaronmeurer/Documents/emacs-jedi/env/lib/python3.8/site-packages/sexpdata.py", line 404, in tosexp
    raise TypeError(uformat(
TypeError: Object of type '<class 'pathlib.PosixPath'>' cannot be converted by `tosexp`. It's value is 'PosixPath('/Users/aaronmeurer/Documents/ndindex/ndindex/tests/test_integerarray.py')'

I think Jedi is switching to use pathlib everywhere.

If you don't use the master version of Jedi, you get some other pathlib related errors, which have been fixed in the dev version of jedi.

[asmeurer]

I tried using the git version of sexpdata, and I get

Traceback (most recent call last):
  File "/Users/aaronmeurer/Documents/python-epc/epc/handler.py", line 238, in _handle
    (name, uid, args) = unpack_message(sexp)
  File "/Users/aaronmeurer/Documents/python-epc/epc/handler.py", line 87, in unpack_message
    return (data[0].value(), data[1], data[2:])
AttributeError: 'Symbol' object has no attribute 'value'

This also happens with the git version of python-epc.

[asmeurer]

Apparently sexpdata removed value a long time ago jd-boyd/sexpdata@6e9b0c2, but python-epc was never updated.

[asmeurer]

@cfroehli @immerrr #354 probably fixed the problem in my first comment (I haven't checked yet), but what about #348 (comment)? Does python-epc need to be updated?

[immerrr]

@asmeurer oh, sorry, I didn't mean to close this. Do you have a repro for the issue?

[asmeurer]

I think it was coming from using the git version of sexpdata and a goto-definition. I'll need to check the latest version of emacs-jedi later to see if I can still reproduce it.

[asmeurer]

I've been using the git versions of python-epc, emacs-jedi (before the fix from #354), and sexpdata with the following patch:

diff --git a/sexpdata.py b/sexpdata.py
index c4c6ce2..1f1da1b 100644
--- a/sexpdata.py
+++ b/sexpdata.py
@@ -81,6 +81,8 @@ from collections import namedtuple, Iterable, Mapping
 from itertools import chain
 from string import whitespace

+import pathlib
+
 BRACKETS = {'(': ')', '[': ']'}


@@ -402,6 +404,16 @@ def _(obj, str_as='string', **kwds):
     else:
         raise ValueError('str_as={0!r} is not valid'.format(str_as))

+@tosexp.register(pathlib.PosixPath)
+def _(obj, str_as='string', **kwds):
+    kwds['str_as'] = str_as
+    if str_as == 'symbol':
+        return str(obj)
+    elif str_as == 'string':
+        return tosexp(String(str(obj)))
+    else:
+        raise ValueError('str_as={0!r} is not valid'.format(str_as))
+

 @tosexp.register(type(None))
 def _(obj, none_as='()', **kwds):
@@ -421,6 +433,9 @@ def _(obj, **kwds):

 class String(unicode):

+    def value(self):
+        return unicode(self)
+
     def __eq__(self, other):
         """
         >>> from itertools import permutations

The last change is something that should be fixed in python-epc, not sexpdata, but it should give an idea of what is broken. At a glance, the pathlib thing might be relevant to upstream into sexpdata, assuming it is still a concern.

[immerrr]

Yeah, I think it should be possible to fix this if we discover where does jediepcserver send pathlib.PosixPath to Emacs, it can be converted to a string there, and no changes elsewhere would be required.

[cfroehli]

The pathlib instance I've spotted was the d.module_path attribute, sent to emacs via the definition_to_xxx method (helpers building the required dict to be answer the deferred call). I've forced a str there so a basic string is passed back to emacs instead of the pathlib instance (jediepcserver.py lines 350 & 360). We may prefer a call to unicode though (just be careful in case of python2 interpreter as str/unicode semantic changed).

[immerrr]

@cfroehli thank you!

I think it should be fine to return a native str and let sexpdata convert it to unicode if necessary. Let's wait for the confirmation that this is fixed, and looks like we can close this.

[pawciobiel]

@cfroehli I didn't get any errors with pathlib so I assume calling str() on module_path works OK.
@immerrr I've added a tiny fix for missing value() in tkf/python-epc#12 and although tested it locally it's difficult for me to say if it can break anything else... Any help with this would be highly appreciated.

[pawciobiel]

@asmeurer out of curiosity, why didn't you add your patch as PR to https://github.com/jd-boyd/sexpdata ?

[glyph]

The upstream bug has been fixed, so this should be closed: jd-boyd/sexpdata#36