Strange remote environment variable causes error

[pigay]

Hi,

with a SshMachine, strange remote environment variable causes an error:

Traceback (most recent call last):
  File "rpyc_test.py", line 5, in <module>
    mach = SshMachine("avakas.mcia.univ-bordeaux.fr", user="pigay", keyfile="/home/pigay/.ssh/id_rsa")
  File "/usr/local/lib/python2.7/dist-packages/plumbum/machines/ssh_machine.py", line 106, in __init__
    new_session = new_session)
  File "/usr/local/lib/python2.7/dist-packages/plumbum/machines/remote.py", line 132, in __init__
    self.env = RemoteEnv(self)
  File "/usr/local/lib/python2.7/dist-packages/plumbum/machines/remote.py", line 18, in __init__
    self._curr = dict(line.split("=", 1) for line in self.remote._session.run("env")[1].splitlines())
ValueError: dictionary update sequence element #44 has length 1; 2 is required

The environment that causes this error is the second part of a function definition. Here is what appears with an env command on the remote machine:

BASH_FUNC_module()=() {  eval `/usr/bin/modulecmd bash $*`
}

The return character in BASH_FUNC_module causes an error in machines/remote.py because of splitline call, I think.

Would it be ok to simply catch the ValuerError, or to use a more complicated parsing method?

Regards,

Pierre

[pigay]

I managed to make it work by modifying RemoteEnv constructor like this:

class RemoteEnv(BaseEnv):
    """The remote machine's environment; exposes a dict-like interface"""

    __slots__ = ["_orig", "remote"]
    def __init__(self, remote):
        self.remote = remote
        lines =  self.remote._session.run("env")[1].splitlines()
        for i, l in enumerate(lines[:]):
            if "=" not in l:
                lines[i-1] += "\n" + l
                del lines[i]
        self._curr = dict(line.split("=", 1) for line in lines)
        self._orig = self._curr.copy()
        BaseEnv.__init__(self, self.remote.path, ":")

it's quite awful I'm afraid...

[tomerfiliba]

but you didn't say what the bug was :)
could you elaborate?

-tomer

---

Tomer Filiba
tomerfiliba.com http://www.facebook.com/tomerfiliba
http://il.linkedin.com/in/tomerfiliba

On Mon, Nov 3, 2014 at 1:22 AM, Pierre Gay notifications@github.com wrote:

I managed to make it work by modifying RemoteEnv constructor like this:

class RemoteEnv(BaseEnv):
"""The remote machine's environment; exposes a dict-like interface"""

__slots__ = ["_orig", "remote"]
def __init__(self, remote):
    self.remote = remote
    lines =  self.remote._session.run("env")[1].splitlines()
    for i, l in enumerate(lines[:]):
        if "=" not in l:
            lines[i-1] += " " + l
            del lines[i]
    self._curr = dict(line.split("=", 1) for line in lines)
    self._orig = self._curr.copy()
    BaseEnv.__init__(self, self.remote.path, ":")

it's quite awful I'm afraid...

—
Reply to this email directly or view it on GitHub
#162 (comment).

[pigay]

I thought I had said it in the first post ;)

Anyway:

It's that the way you get environment doesn't cope multiline variables correctly: When you call self.remote._session.run("env")[1].splitlines(), any variable that contains a \n character will be splitted into 2 lines and the second one will trigger the ValueError because split("=", 1) returns only one value.

My fix proposal is horrible, but I can't find something better for the moment.

[adamckay]

Hi,

I'm also experiencing this issue - could this be patched? :)

[campbellr]

I've also been hitting this.

I have the following environment variable that breaks due to splitlines: 'COMP_WORDBREAKS= \t\n"\'><;|&(:'

This seems to fix it:

- self._curr = dict(line.split("=", 1) for line in self.remote._session.run("env")[1].splitlines())
+ self._curr = dict(line.split("=", 1) for line in self.remote._session.run("env -0")[1].split('\x00'))

Although I'm not sure how cross-platform the -0 flag is for env...

UPDATE: Hm, turns out that doesn't actually work (seems to break gathering the returncode):

>>> from plumbum.machines.paramiko_machine import ParamikoMachine
>>> from paramiko import AutoAddPolicy
>>> machine = ParamikoMachine('localhost', missing_host_policy=AutoAddPolicy())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/localhome/campbr9/.local/lib/python2.7/site-packages/plumbum/machines/paramiko_machine.py", line 165, in __init__
    BaseRemoteMachine.__init__(self, encoding, connect_timeout)
  File "/localhome/campbr9/.local/lib/python2.7/site-packages/plumbum/machines/remote.py", line 132, in __init__
    self.env = RemoteEnv(self)
  File "/localhome/campbr9/.local/lib/python2.7/site-packages/plumbum/machines/remote.py", line 18, in __init__
    self._curr = dict(line.split("=", 1) for line in self.remote._session.run('env -0')[1].split('\x00'))
  File "/localhome/campbr9/.local/lib/python2.7/site-packages/plumbum/machines/session.py", line 224, in run
    return run_proc(self.popen(cmd), retcode)
  File "/localhome/campbr9/.local/lib/python2.7/site-packages/plumbum/commands/processes.py", line 207, in run_proc
    stdout, stderr)
plumbum.commands.processes.ProcessExecutionError: Command line: "env -0 ; echo $? ; echo '--.END1038318204.6.--' ; echo '--.END1038318204.6.--' 1>&2"
Exit code: Unknown
Stdout:  | XDG_SESSION_ID=24VIRTUALENVWRAPPER_SCRIPT=/localhome/campbr9/.local/bin/virtualenvwrapper.shCOMP_WORDBREAKS=     

[campbellr]

Awesome, thanks @tomerfiliba (sorry about the broken 'fix' i suggested)!

[pigay]

Thanks @tomerfiliba ;)