Cannot use Debug mode in INET anymore

[haug-den-lucas]

Hello,

for some reason I cannot use the Debug mode in INET anymore and I cannot figure out why.

For testing I completely reinstalled OMNeT++ 6.0.3 in a new folder to ensure there is not some old configuration issue.
I can use the debug mode for these OMNeT++ samples (e.g. the TicToc sample)

Additionally I created a new workspace in which I installed INET 4.5.2 from the INET v4.5.2 git tag and another new workspace where I've let OMNeT++ install INET automatically.

When I try to run any configuation in debug mode (e.g. timeawareshaper), it builds the project and I'm prompted with the following error:

For every subsequent run where I try to run the configuration in debug mode I get the following error:

When I delete the .metadata folder in my workspace and open the workspace again I jump back to the first error.

I start OMNeT++ in a conda environment:

Conda Environment extract

$ conda list
# packages in environment at /home/haugls/miniconda3/envs/omnet:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
blas                      1.0                         mkl  
bottleneck                1.3.7           py312ha883a20_0  
brotli                    1.0.9                h5eee18b_8  
brotli-bin                1.0.9                h5eee18b_8  
bzip2                     1.0.8                h5eee18b_6  
ca-certificates           2024.3.11            h06a4308_0  
contourpy                 1.2.0           py312hdb19cb5_0  
cycler                    0.11.0             pyhd3eb1b0_0  
cyrus-sasl                2.1.28               h52b45da_1  
dbus                      1.13.18              hb2f20db_0  
expat                     2.6.2                h6a678d5_0  
fontconfig                2.14.1               h4c34cd2_2  
fonttools                 4.51.0          py312h5eee18b_0  
freetype                  2.12.1               h4a9f257_0  
glib                      2.78.4               h6a678d5_0  
glib-tools                2.78.4               h6a678d5_0  
gst-plugins-base          1.14.1               h6a678d5_1  
gstreamer                 1.14.1               h5eee18b_1  
icu                       73.1                 h6a678d5_0  
intel-openmp              2023.1.0         hdb19cb5_46306  
jpeg                      9e                   h5eee18b_1  
kiwisolver                1.4.4           py312h6a678d5_0  
krb5                      1.20.1               h143b758_1  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.38                 h1181459_1  
lerc                      3.0                  h295c915_0  
libbrotlicommon           1.0.9                h5eee18b_8  
libbrotlidec              1.0.9                h5eee18b_8  
libbrotlienc              1.0.9                h5eee18b_8  
libclang                  14.0.6          default_hc6dbbc7_1  
libclang13                14.0.6          default_he11475f_1  
libcups                   2.4.2                h2d74bed_1  
libdeflate                1.17                 h5eee18b_1  
libedit                   3.1.20230828         h5eee18b_0  
libffi                    3.4.4                h6a678d5_1  
libgcc                    7.2.0                h69d50b8_2  
libgcc-ng                 11.2.0               h1234567_1  
libgfortran-ng            11.2.0               h00389a5_1  
libgfortran5              11.2.0               h1234567_1  
libglib                   2.78.4               hdc74915_0  
libgomp                   11.2.0               h1234567_1  
libiconv                  1.16                 h5eee18b_3  
libllvm14                 14.0.6               hdb19cb5_3  
libpng                    1.6.39               h5eee18b_0  
libpq                     12.17                hdbd6064_0  
libstdcxx-ng              11.2.0               h1234567_1  
libtiff                   4.5.1                h6a678d5_0  
libuuid                   1.41.5               h5eee18b_0  
libwebp-base              1.3.2                h5eee18b_0  
libxcb                    1.15                 h7f8727e_0  
libxkbcommon              1.0.1                h5eee18b_1  
libxml2                   2.10.4               hfdd30dd_2  
lz4-c                     1.9.4                h6a678d5_1  
matplotlib                3.8.4           py312h06a4308_0  
matplotlib-base           3.8.4           py312h526ad5a_0  
mkl                       2023.1.0         h213fc3f_46344  
mkl-service               2.4.0           py312h5eee18b_1  
mkl_fft                   1.3.8           py312h5eee18b_0  
mkl_random                1.2.4           py312hdb19cb5_0  
mysql                     5.7.24               h721c034_2  
ncurses                   6.4                  h6a678d5_0  
numexpr                   2.8.7           py312hf827012_0  
numpy                     1.26.4          py312hc5e2394_0  
numpy-base                1.26.4          py312h0da6c21_0  
openjpeg                  2.4.0                h3ad879b_0  
openssl                   3.0.14               h5eee18b_0  
packaging                 23.2            py312h06a4308_0  
pandas                    2.2.2           py312h526ad5a_0  
pcre2                     10.42                hebb0a14_1  
pillow                    10.3.0          py312h5eee18b_0  
pip                       24.0            py312h06a4308_0  
ply                       3.11            py312h06a4308_1  
posix-ipc                 1.1.1                    pypi_0    pypi
pybind11-abi              5                    hd3eb1b0_0  
pyparsing                 3.0.9           py312h06a4308_0  
pyqt                      5.15.10         py312h6a678d5_0  
pyqt5-sip                 12.13.0         py312h5eee18b_0  
python                    3.12.3               h996f2a0_1  
python-dateutil           2.9.0post0      py312h06a4308_2  
python-tzdata             2023.3             pyhd3eb1b0_0  
pytz                      2024.1          py312h06a4308_0  
qt-main                   5.15.2              h53bd1ea_10  
readline                  8.2                  h5eee18b_0  
scipy                     1.13.1          py312hc5e2394_0  
seaborn                   0.13.2          py312h06a4308_0  
setuptools                69.5.1          py312h06a4308_0  
sip                       6.7.12          py312h6a678d5_0  
six                       1.16.0             pyhd3eb1b0_1  
sqlite                    3.45.3               h5eee18b_0  
tbb                       2021.8.0             hdb19cb5_0  
tk                        8.6.14               h39e8969_0  
tornado                   6.3.3           py312h5eee18b_0  
tzdata                    2024a                h04d1e81_0  
unicodedata2              15.1.0          py312h5eee18b_0  
wheel                     0.43.0          py312h06a4308_0  
xz                        5.4.6                h5eee18b_1  
zlib                      1.2.13               h5eee18b_1  
zstd                      1.5.5                hc292b87_2

$ pip list
Package         Version
--------------- -----------
Bottleneck      1.3.7
contourpy       1.2.0
cycler          0.11.0
fonttools       4.51.0
kiwisolver      1.4.4
matplotlib      3.8.4
mkl-fft         1.3.8
mkl-random      1.2.4
mkl-service     2.4.0
numexpr         2.8.7
numpy           1.26.4
packaging       23.2
pandas          2.2.2
pillow          10.3.0
pip             24.0
ply             3.11
posix_ipc       1.1.1
pyparsing       3.0.9
PyQt5           5.15.10
PyQt5-sip       12.13.0
python-dateutil 2.9.0.post0
pytz            2024.1
scipy           1.13.1
seaborn         0.13.2
setuptools      69.5.1
sip             6.7.12
six             1.16.0
tornado         6.3.3
tzdata          2023.3
unicodedata2    15.1.0
wheel           0.43.0

Does someone have an idea how to fix this?

[rhornig]

make sure that you source setenv in the INET root directory and start the IDE only after that, from that particular shell. The debugger tries to set up some fomatters and it cannot find them as they are defined in inetroot/python dir which is added only by the setenv script.

[haug-den-lucas]

Sadly that didn't work.
I deleted the .metadata in my workspace again and did a make cleanall in the inet project to start clean again.

Then performed the following steps:

~/workspaces/test$ conda activate omnet
~/workspaces/test$ source ~/omnetpp-6.0.3_2/setenv 
Environment for 'omnetpp-6.0.3' in directory '/home/haugls/omnetpp-6.0.3_2' is ready.
~/workspaces/test$ source inet/setenv 
Environment for INET 4.5.2 in directory '/home/haugls/workspaces/test/inet' is ready.
~/workspaces/test$ omnetpp
Starting the OMNeT++ IDE...

Then I again imported the INET project using the "Open projects from file system" menu and tried to run a simulation in debug mode.
Got the first error again.

[haug-den-lucas]

Some additional information:
I also tried installing OMNeT++ again with the following changes:

• Use X11 instead of Wayland
• Don't use conda at all but instead use the system-wide packages

Getting the same two errors and no clue what else to try.

I think the problem occurs since I moved from Ubuntu 22.04 to 24.04.
It was a clean install, no upgrade

[rhornig]

It's strange as the file reporting the error (gdbinit.py) is in OMNeT++ and does not reference any inet specific package so I don't know why it throws an exeception. I will try to reproduce this. I recently upgraded to ubuntu 24.04, too.

[levy]

Try starting the python repl and import inet, maybe you will see the problem.

[haug-den-lucas]

It's strange as the file reporting the error (gdbinit.py) is in OMNeT++ and does not reference any inet specific package so I don't know why it throws an exeception. I will try to reproduce this. I recently upgraded to ubuntu 24.04, too.

I'm currently setting up a Ubuntu 24.04 VM and I will try to reproduce the issue there.

Try starting the python repl and import inet, maybe you will see the problem.

~/workspaces/test3/inet4.5$ source setenv
Environment for INET 4.5.2 in directory '/home/haugls/workspaces/test3/inet4.5' is ready.
~/workspaces/test3/inet4.5$ python3
Python 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import inet
No module named 'py4j': optional inet.common.ide package will not work
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/__init__.py", line 10, in <module>
    from inet.documentation import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/documentation/__init__.py", line 1, in <module>
    from inet.documentation.chart import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/documentation/chart.py", line 10, in <module>
    from inet.simulation.project import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/simulation/__init__.py", line 12, in <module>
    from inet.simulation.config import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/simulation/config.py", line 12, in <module>
    from inet.simulation.project import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/simulation/project.py", line 126, in <module>
    ned_exclusions=[s.strip() for s in open(get_workspace_path("inet/.nedexclusions")).readlines()],
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/haugls/workspaces/test3/inet/.nedexclusions'
>>> 

[rhornig]

Hmmm. .nedexclusions file should be optional, so this is an issue. To work aorund just touch .nedexclusions to generate an empty file.

Besides, the following packages are used by the inet module: dask distributed py4j ipython optimparallel sewar. We should properly document this and at least create a proper python/requirements.txt file.

@levy I'm not sure why the debugger's init script wants to import the inet module at all. This prevents debugging if the above modules are not present in the actual python virtual environment.

[haug-den-lucas]

Hmmm. .nedexclusions file should be optional, so this is an issue. To work aorund just touch .nedexclusions to generate an empty file.

Even when the file is present inet tries to load it from [workspace]/inet/.nedexclusions even though the automatic install installs inet in [workspace]/inet4.5

I created a new workspace and pulled the inet 4.5.2 tag manually into [workspace]/inet
Additionally i installed all required python modules by inet. Now I am able to run "import inet" in the python console.
But only if I run source setenv before.
Launching the Python console in omnet even if I launched omnet from a shell which has also executed source setenv for inet it still doesn't work

Also for debugging i still get the same error.

My Ubuntu 24.04 VM is still setting up.
I'll give an update on this later

[haug-den-lucas]

More detailed:
In my bash i execute source inet/setenv and then omnetpp

Within omnet I can rightclick the inet folder and click on "Show in local terminal" i can do the following:

:~/workspaces/test4/inet$ python3
Python 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import inet
No module named 'py4j': optional inet.common.ide package will not work
>>> 

However, if I click on the "Open console" in the bottom part and then on "PyDev console" i get the following error:

import sys; print('%s %s' % (sys.executable or sys.platform, sys.version))

/usr/bin/python3 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0]
0.03s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
import inet
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/haugls/omnetpp-6.0.3/ide/plugins/org.python.pydev.core_11.0.3.202310301107/pysrc/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'inet'

So for some reason it feels like I am in the correct environment in one case but I'm not on the other.

Also the PYTHONPATH is different in both cases.
First case:

>>> print(os.getenv("PYTHONPATH"))
/home/haugls/workspaces/test4/inet/python:::/home/haugls/omnetpp-6.0.3/python

Second case:

print(os.getenv("PYTHONPATH"))
/home/haugls/omnetpp-6.0.3/ide/plugins/org.python.pydev.core_11.0.3.202310301107/pysrc/pydev_sitecustomize:/home/haugls/omnetpp-6.0.3/ide/plugins/org.omnetpp.scave.pychart_6.0.3.240223-17fcae5ef3/python:/home/haugls/omnetpp-6.0.3/python:/usr/lib/python3.12:/usr/lib/python3.12/lib-dynload:/home/haugls/.local/lib/python3.12/site-packages:/usr/local/lib/python3.12/dist-packages:/usr/lib/python3/dist-packages

[haug-den-lucas]

So I was able to recreate this issue in a clean Ubuntu 24.04 VM. I can export the VM and provide the VM image if that helps.

Steps to reproduce:
1: Install Ubuntu 24.04 with the default settings.

2: Install all packages as stated in the OMNeT++ installation guide:

sudo apt-get install build-essential clang lld gdb bison flex perl python3 python3-pip qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools libqt5opengl5-dev libxml2-dev zlib1g-dev doxygen graphviz libwebkit2gtk-4.0-37 xdg-utils

This will fail as it libwebkit2gtk-4.0-37 is not found, so I installed everything else

3: Install python modules:

$ python3 -m pip install --user --upgrade numpy pandas matplotlib scipy seaborn posix_ipc

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

So instead install them with apt-get:

sudo apt-get install python3-numpy python3-pandas python3-matplotlib python3-scipy python3-seaborn

posix_ipc does not exist here so in installed that using

pip install --break-system-packages posix_ipc

4: Downlad and setup omnetpp:

source setenv
./configure WITH_OSG=no
make

5: Start omnetpp from console

omnetpp

6: Create an empty workspace (~/workspaces/test in my case)

7: Install inet git clone --branch v4.5.2 https://github.com/inet-framework/inet (needed to install git here)

8: Import the inet folder in OMNeT++ in your workspace

9: Try to debug the timeawareshaper showcase

[haug-den-lucas]

And here the .log file from [workspace]/.metadata/.log:

omnetpp.log

[rhornig]

Normally the debugger should not depend on optional packages, but it seems it is not the case. It is definitely a bug. As a workaround try to install the following packages in the virtual env: dask distributed py4j ipython optimparallel sewar

[haug-den-lucas]

Normally the debugger should not depend on optional packages, but it seems it is not the case. It is definitely a bug. As a workaround try to install the following packages in the virtual env: dask distributed py4j ipython optimparallel sewar

Sadly even then I cannot use the debug mode.
In my VM additional to the previous steps I did the following:

sudo apt-get install python3-dask python3-distributed python3-ipython
pip install py4j ipython optimparallel sewar --break-system-packages

I completely deleted the workspace and reperformed steps 5 to 8 above

Checked that everything is set up correctly:

$ source setenv
Environment for INET 4.5.2 in directory '/home/ubuntu/workspaces/test/inet' is ready.
$ python3
Python 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import inet
>>> exit()

Which worked just fine

When repeating step 9 I still get the same error.

[haug-den-lucas]

// Edit the first error in step 9 is the same, if I execute it again I now get the following error:

Which is weird, because I can import omnetpp.scave without any problems in the python console.

[haug-den-lucas]

Another interseting thing, when I try to recreate the issue manually

$ gdb
GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git
[...]

For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) source omnetpp-6.0.3/misc/gdb/gdbinit.py 
Warning: 'set index-cache on', an alias for the command 'set index-cache enabled', is deprecated.
Use 'set index-cache enabled on'.

Pretty printers initialized: libstdc++
Pretty printers initialized: omnetpp
(gdb)

Everything works just fine.

If I click the debug button in OMNeT++ now I again get this error:

Error in final launch sequence:

Failed to execute MI command:
source /home/haugls/omnetpp-6.0.3//misc/gdb/gdbinit.py
Error message from debugger back end:
Error occurred in Python: No module named 'omnetpp.scave'
Failed to execute MI command:
source /home/haugls/omnetpp-6.0.3//misc/gdb/gdbinit.py
Error message from debugger back end:
Error occurred in Python: No module named 'omnetpp.scave'
Error occurred in Python: No module named 'omnetpp.scave'

If I now perform the same steps as above again and excute the gdbinit manually, the output differs:

$ gdb
GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git
[...]

For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) source omnetpp-6.0.3/misc/gdb/gdbinit.py 
Warning: 'set index-cache on', an alias for the command 'set index-cache enabled', is deprecated.
Use 'set index-cache enabled on'.

Pretty printers initialized: libstdc++
Pretty printers initialized: omnetpp
/tmp/gdbinit.tmp
Python Exception <class 'ModuleNotFoundError'>: No module named 'omnetpp.scave'
Error occurred in Python: No module named 'omnetpp.scave'
(gdb) 

This is due to a file /tmp/gdbinit.py being generated (by the IDE I assume) with the following contents:

exec(open('/home/haugls/workspaces/test/inet/.gdbinit.py').read(), {}, {'__file__':'/home/haugls/workspaces/test/inet/.gdbinit.py'})

// Edit:
Apparently the /tmp/gdbinit.py is only generated when the .gdbinit.py in the inet folder is present.
Deleting this file at least let's me execute code in debug mode.
This is a state I can now work with, so I probably won't proactively try to dive deeper.
If you require more information from me regarding my environment or reproducing this, I will ofc provide that information.

I'm still really curious why I did not have this problem on my earlier Kubuntu 22.04 though.

[levy]

Thanks for writing a detailed bug report! We will get back to you seen when @rhornig is back in the office.

[tomatze]

Hello,
I also ran into the issue described here.
Deleting .gdbinit.py as suggested by @haug-den-lucas also works for me as a fix for now.

I'm running arch linux and figured out, that I can switch between a working version and a version with this issue by updating/downgrading the gdb package.
When I use gdb version 14.2, everything works as expected.
The issue only occurs after updating the gdb package to version 15.1.

I used the following commands to update/downgrade gdb:
Downgrade: sudo pacman -U https://archive.archlinux.org/packages/g/gdb/gdb-14.2-2-x86_64.pkg.tar.zst https://archive.archlinux.org/packages/g/gdb-common/gdb-common-14.2-2-x86_64.pkg.tar.zst
Update: sudo pacman -U https://archive.archlinux.org/packages/g/gdb/gdb-15.1-1-x86_64.pkg.tar.zst https://archive.archlinux.org/packages/g/gdb-common/gdb-common-15.1-1-x86_64.pkg.tar.zst

According to https://www.sourceware.org/gdb/news/ and https://ftp.gnu.org/gnu/gdb/ there are no gdb versions in between version numbers 14.2 and 15.1. So it seems that the issue first appears with version 15.1.

[haug-den-lucas]

I just checked and my the version on my system seems to be 15.0.50.20240403-0ubuntu
So I assume it breaks with an update from gdb 14 to 15.

[haug-den-lucas]

Hi, I think this issue should be repoened.
The issue doesn't seem to be fixed.

Current INET master branch and OMNeT++ 6.1

[rhornig]

Please note that the FIX is not yet taken-over to the master branch. It is fixed only in the 4.5.x branch.

[haug-den-lucas]

Accoring to what is shown on this page (top left), the commit seems to be part of the master branch already:
f157a18

[rhornig]

indeed. (I forgot to fetch from the repo before checking this).

A tip: Make sure that 'setenv' form INET is also sourced before launching the IDE. That script adds the inet spcific packages to the PYTHONPATH.

[haug-den-lucas]

Thanks, sourcing setenv seems to fix the problem when launching omnetpp from the command line.
A fix that also works when launching OMNeT++ from the Desktop or similar would be great though.

[rhornig]

The problem is that the 'setenv' script of inet is obviously INET specific. When you start the OMNeT++ IDE, there is absolutely no knowledge what kind of models will be used and the verious paths (NED, PYTHON etc. MUST be set before the IDE so the IDE process could inherit it).

Apart from sourcing the OMNeT++ and INET setenv scripts from your .profile I have no idea.