RNS python package does not recognize dependencies

[Sgiath]

Describe the bug

I want to use binaries from RNS package. I am able to install it and run it but once I get to the part of the program that requires serial (pyserial package ) it fails with this error:

[12:33:03] Flashing RNode firmware to device on /dev/ttyACM0
Pyserial is not installed for /nix/store/04gg5w1s662l329a8kh9xcwyp0k64v5a-python3-3.12.4/bin/python3.12. Check the README for installation instructions.
Traceback (most recent call last):
  File "/home/sgiath/.config/rnodeconf/update/1.72/esptool.py", line 27, in <module>
    import serial
ModuleNotFoundError: No module named 'serial'
[12:33:03] Error from flasher (1) while writing.

But I can see pyserial listed as dependency, so I am not sure what is going on.

Steps To Reproduce

Steps to reproduce the behavior:

1. Add pkgs.python312Packages.rns to your packages
2. Run rnodeconf --autoinstall and select correct values for your device
3. Once it downloads the firmware and tries to flash it, it will fail with the error above

Expected behavior

pyserial is recognized as installed dependency

Notify maintainers

@fabaff

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.10.5-xanmod1, NixOS, 24.11 (Vicuna), 24.11.20240821.c374d94`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.5`
 - nixpkgs: `/nix/store/j8pbrsb3nybdap3hhg9kw0ffqd4rlbx6-source`

---

Add a 👍 reaction to issues you find important.

[Sgiath]

Digging a bit deeper it seems like the program is generating some python script in my config folder .config/rnodeconf/update/1.72/esptool.py which is executed outside of the context of the package. Any idea how to solve it?

[dotlambda]

It seems to download its own version of esptool. We should patch the code to use the esptool package from Nixpkgs.

[aos]

I think this is what's happening: https://github.com/markqvist/Reticulum/blob/a1be97bd69e54ac10bdd90f82af1b50649b35b88/RNS/Utilities/rnodeconf.py#L3689

It's literally writing a script to file and then calling it. I thought about changing the shebang to use the Nixpkgs python3 version via sed, but I don't know how to also include the pyserial dependency as part of it.

[Sgiath]

You could probably replace it with something like this? (taken from wiki)

#! /usr/bin/env nix-shell 
#! nix-shell -i python3 -p python3Packages.pyserial

Not sure if there are other dependencies that are needed or how to patch the code (I am so far only user of NixOS)

[Sgiath]

OK this are the imports from the file that gets decoded:

import argparse,base64,binascii,copy,hashlib,inspect,io,itertools,os,shlex,string,struct,sys,time,zlib

try:
  import serial
except ImportError:
  print('Pyserial is not installed for %s. Check the README for installation instructions.'%sys.executable)
  raise

try:
  if 'serialization' in serial.__doc__ and 'deserialization' in serial.__doc__:
    raise ImportError("\nesptool.py depends on pyserial, but there is a conflict with a currently installed package named 'serial'.\n\nYou may be able to work around this by 'pip uninstall serial; pip install pyserial' but this may break other installed Python software that depends on 'serial'.\n\nThere is no good fix for this right now, apart from configuring virtualenvs. See https://github.com/espressif/esptool/issues/269#issuecomment-385298196 for discussion of the underlying issue(s).")
except TypeError:
  pass

try:
  import serial.tools.list_ports as list_ports
except ImportError:
  print('The installed version (%s) of pyserial appears to be too old for esptool.py (Python interpreter %s). Check the README for installation instructions.'%(sys.VERSION,sys.executable))
  raise
except Exception:
  if sys.platform == 'darwin':
    list_ports = _A
  else:
    raise

no other imports in the rest of the file. So it seems like only the pyserial needs to be added.

[aos]

I sent a PR here: #337285 do you wanna test it out? You can build it locally by pulling down this PR and running this in the root of nixpkgs directory:

$ nix-build -A python3Packages.rns

[Sgiath]

Look what I have found: markqvist/Reticulum#60 (comment)

Could this be the whole reason behind this?