Problem with installation of CULanes dataset

[Wakarox]

Hello,
I have a problem with the installation of the CULanes dataset.
When i do the "make" command, i have an "error opencv2/core/core.hpp : No such file or directory".
I have all the libraries on my computer but i am not using conda.
Can someone help me ?

[voldemortX]

@Wakarox Check if you have the c++ OpenCV on your system?

[Wakarox]

I am using cv2.
I commented the line 21 as the tutorial said.

[voldemortX]

I am using cv2.
I commented the line 21 as the tutorial said.

that is weird. you sure it is not python opencv?

[Wakarox]

I have the error on the include of the library. I did an installation of opencv and have the files in /usr/local/includes. Maybe i have a problem with the path of the files ?

[voldemortX]

@Wakarox Can you display opencv version by pkg-config --modversion opencv ?

[Wakarox]

OpenCV is not found. I don't know why ...

[voldemortX]

OpenCV is not found. I don't know why ...

It seems you may have a faulty installation of OpenCV. Are you using ubuntu, I think it can be installed via apt-get.

[Wakarox]

I have pkg-config --modversion opencv4 which returns 4.5.5.

[voldemortX]

I have 3.2.0 on my own machine and it works.

[Wakarox]

Not me. I don't know why :/

[voldemortX]

@Wakarox Perhaps try install opencv 2.x?

[Wakarox]

culane_list_convertor.py show something when running it?

[voldemortX]

culane_list_convertor.py show something when running it?

nope. no printing means success, as in all Linux commands.

[Wakarox]

My error code :
pytorch-auto-drive/tools/culane_evaluation$ make
CXX src/lane_compare.cpp
In file included from include/lane_compare.hpp:4,
from src/lane_compare.cpp:8:
include/spline.hpp:6:10: fatal error: opencv2/core/core.hpp: Aucun fichier ou dossier de ce type
6 | #include <opencv2/core/core.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:40 : build/src/lane_compare.o] Erreur 1

[voldemortX]

My error code : pytorch-auto-drive/tools/culane_evaluation$ make CXX src/lane_compare.cpp In file included from include/lane_compare.hpp:4, from src/lane_compare.cpp:8: include/spline.hpp:6:10: fatal error: opencv2/core/core.hpp: Aucun fichier ou dossier de ce type 6 | #include <opencv2/core/core.hpp> | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile:40 : build/src/lane_compare.o] Erreur 1

I reproduced this error with a newly installed opencv4 from source. And I can make with adding CXXFLAGS += -I/usr/local/include/opencv4, for instance at Line 22. Or if you have other place for your includes, change the -I path.

[Wakarox]

The include is now good, i found something like you said yesterday.
But now, i have a problem with the shared libraries.

CXX/LD evaluate
/usr/bin/ld: build/src/evaluate.o: undefined reference to symbol '_ZN2cv6imreadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi'
/usr/bin/ld : /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 : erreur lors de l'ajout de symboles : DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [Makefile:44 : evaluate] Erreur 1

[voldemortX]

The include is now good, i found something like you said yesterday. But now, i have a problem with the shared libraries.

CXX/LD evaluate /usr/bin/ld: build/src/evaluate.o: undefined reference to symbol '_ZN2cv6imreadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi' /usr/bin/ld : /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 : erreur lors de l'ajout de symboles : DSO missing from command line collect2: error: ld returned 1 exit status make: *** [Makefile:44 : evaluate] Erreur 1

As far as I know, this might be caused by installing multiple versions of opencv. For instance, when I install opencv3 from apt and sudo install the opencv4 zip, I see a similar error.

[Wakarox]

So you suggest me to unistall opencv, and reinstall it ?

[voldemortX]

In my case I uninstalled the apt one. Or you can find a way to link that .so specifically, I think it is already the last step for this build.

[Wakarox]

I tried to unistall it, it didn't work, i always have the same issue... I am searching a way to link that .so but all i tried didn't work too

[voldemortX]

@Wakarox I think you can search for opencv installation issues in general, since this probably is not specific to these codes. If you really can't make it work, you can try use culane_evaluation_py instead. I believe LaneATT have a full python version. But it slightly misalign with the cpp version.

[Wakarox]

I have a new error when i changed the Makefile deleting the 2 lines of condition:

CXX/LD evaluate
/usr/bin/ld : build/src/lane_compare.o : dans la fonction « LaneCompare::get_lane_similarity(std::vector<cv::Point_, std::allocator<cv::Point_ > > const&, std::vector<cv::Point_, std::allocator<cv::Point_ > > const&) » :
lane_compare.cpp:(.text+0x56e) : référence indéfinie vers « cv::_InputArray::_InputArray(cv::MatExpr const&) »
/usr/bin/ld : build/src/lane_compare.o : dans la fonction « cv::MatExpr::operator cv::Mat() const » :
lane_compare.cpp:(.text._ZNK2cv7MatExprcvNS_3MatEEv[_ZNK2cv7MatExprcvNS_3MatEEv]+0x2c) : référence indéfinie vers « cv::Mat::Mat() »
collect2: error: ld returned 1 exit status
make: *** [Makefile:43 : evaluate] Erreur 1

[voldemortX]

@Wakarox At this point, I'd suggest you clean your environment of any opencv related stuff, delete the build dir in culane_evaluation. Then install opencv via the opencv documentation, and use the last step sudo make install to install it in /usr. If you installed opencv4 (their latest version), add the Line 22 like I did and it should work fine.

Alternatively, apt install libopencv-dev might also work but I don't really know.

[Wakarox]

That's good now, i found a solution (adding the library in the Makefile).
Now, i am blocked with the config of lane detection.

mokhtarbouain@iris-Precision-Tower-7810:~/Documents/Python_scripts/pytorch-auto-drive$ python3 main_landet.py --config=/home/mokhtarbouain/Téléchargements/resnet18-5c106cde.pth
Traceback (most recent call last):
File "main_landet.py", line 11, in
from utils.args import parse_arg_cfg, read_config, map_states, add_shortcuts, cmd_dict
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/init.py", line 4, in
from . import models
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/init.py", line 16, in
from .swin import SwinTransformer
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/swin.py", line 13, in
from timm.models.layers import DropPath, to_2tuple, trunc_normal_
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/init.py", line 2, in
from .models import create_model, list_models, is_model, list_modules, model_entrypoint,
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/init.py", line 1, in
from .cspnet import *
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/cspnet.py", line 20, in
from .helpers import build_model_with_cfg
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/helpers.py", line 17, in
from .layers import Conv2dSame, Linear
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/layers/init.py", line 7, in
from .cond_conv2d import CondConv2d, get_condconv_initializer
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/layers/cond_conv2d.py", line 16, in
from .helpers import to_2tuple
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/layers/helpers.py", line 6, in
from torch._six import container_abcs
ImportError: cannot import name 'container_abcs' from 'torch._six' (/home/mokhtarbouain/.local/lib/python3.8/site-packages/torch/_six.py)

[voldemortX]

@Wakarox Good to know you made opencv work!

It seems your timm version doesn't support your pytorch version. Do you have pytorch > 1.8? It looks the same issue as in this one:

#37

[Wakarox]

I have Pytorch 1.10.2.

import torch
print(torch.version)
1.10.2+cu102

[voldemortX]

I have Pytorch 1.10.2.

import torch
print(torch.version)
1.10.2+cu102

try 1.6 or 1.8 instead. And of course the corresponding torchvision version.

[Wakarox]

Done.
Now, i have another error, i am not sure but i think it's mmcv.
mokhtarbouain@iris-Precision-Tower-7810:~/Documents/Python_scripts/pytorch-auto-drive$ python3 main_landet.py --config=/home/mokhtarbouain/Téléchargements/resnet18-5c106cde.pth
Traceback (most recent call last):
File "main_landet.py", line 11, in
from utils.args import parse_arg_cfg, read_config, map_states, add_shortcuts, cmd_dict
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/init.py", line 2, in
from . import losses
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/losses/init.py", line 3, in
from .hungarian_loss import HungarianLoss
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/losses/hungarian_loss.py", line 10, in
from ..models.lane_detection import cubic_curve_with_projection
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/init.py", line 7, in
from . import segmentation
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/segmentation/init.py", line 5, in
from .erfnet import ERFNet
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/segmentation/erfnet.py", line 10, in
from ..common_models import non_bottleneck_1d
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/init.py", line 1, in
from .blocks import *
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/blocks/init.py", line 1, in
from .inverted_residual import InvertedResidual, InvertedResidualV3
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/blocks/inverted_residual.py", line 3, in
from ..plugins import SELayer
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/plugins/init.py", line 6, in
from .feature_flip_fusion import FeatureFlipFusion, DCN_v2_Ref
File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/plugins/feature_flip_fusion.py", line 3, in
from mmcv.ops import ModulatedDeformConv2d, modulated_deform_conv2d # DCNv2
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/ops/init.py", line 2, in
from .active_rotated_filter import active_rotated_filter
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/ops/active_rotated_filter.py", line 8, in
ext_module = ext_loader.load_ext(
File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNK2at10TensorBase8data_ptrIdEEPT_v

[voldemortX]

@Wakarox Yes it should be mmcv, but I never encountered this before. Maybe try find it in the mmcv repo? However, are you installing things directly to your system (.local)? That is highly not recommended. Please try use conda if possible and delete the python in .local.

Not considering the python install issue, do check if your mmcv-full is matched with CUDA and pytorch. Did you install mmcv with torch 1.10 before downgrading to below 1.8? In that case, maybe mmcv needs reinstall also

[voldemortX]

@Wakarox Sorry I don't know much about rosbag, but if the input format does not match the current visualization format, you'll have to adjust the APIs yourself to connect these programs. Perhaps you will need to write a new vis Runner or expose some API functions from PytorchAutoDrive based on your use case.

If you made some addtions that would benefit ros integration, I'd be more than happy to take pull requests.

[Wakarox]

I will try to use only one config pretrained (resnet18_culane). I need to filter the files for this config. What files i will need to keep ?

[voldemortX]

I will try to use only one config pretrained (resnet18_culane). I need to filter the files for this config. What files i will need to keep ?

I think you need the entry file, transforms and model definitions (including the config). And lane runner & dataset if training or testing are still required. In fact, best way is find the important files and keep others through import.

[Wakarox]

Hello @voldemortX ,
I have a question for visualization, can i use just a test image without keypoints txt for checking the accuracy of the program ?

Edit : That's good, i just forgot to use --pred on my image in my line command. Now, let's go to try to add it in ROS !
Now, next question : can i have the equation of the lane detected by the code ?

[voldemortX]

Hello @voldemortX ,
I have a question for visualization, can i use just a test image without keypoints txt for checking the accuracy of the program ?

Edit : That's good, i just forgot to use --pred on my image in my line command. Now, let's go to try to add it in ROS !
Now, next question : can i have the equation of the lane detected by the code ?

by equation do you mean a curve equation? In that case LSTR have some sort of polynomial and BezierLaneNet has a Bezier curve at network output, which you can insert codes to save. Other methods do not have such things and you will need to fit a curve equation by yourself.

[Wakarox]

Hello @voldemortX
Where can I find the coordinates of the points ?
I am using a model without curve equation (resnet18_baseline_tusimple), i will add it myself.

[voldemortX]

@Wakarox Perhaps get them somewhere around here? Before they were drawn.

pytorch-auto-drive/utils/vis_utils.py

Line 93 in 76e8330

def lane_detection_visualize_batched(images, masks=None, keypoints=None,

If you are using dataset inference, you can find things in ./output

[Wakarox]

@Wakarox Perhaps get them somewhere around here? Before they were drawn.

pytorch-auto-drive/utils/vis_utils.py

Line 93 in 76e8330

def lane_detection_visualize_batched(images, masks=None, keypoints=None,

If you are using dataset inference, you can find things in ./output

Hello,
I am trying to add your code in ROS, but i need to know if all the parameters in this function are necessary.
I want to use my own images with the predict function, but i want to keep the dataset module for training and testing on my own dataset.
I modified some files to reduce the quantity of code, but it's not enough yet, and i am a little lost.
If you need to see my change, i will send you my code.
Best Regards,

[voldemortX]

@Wakarox Can I maybe see your change in git? I could suggest what more could be deleted when I go through them.

[Wakarox]

I just created a Git for this application : https://github.com/Wakarox/pytorch_auto_drive_ros
I deleted a lot of semantic segmentation code and i just begin to modify lane detection.
For the moment, i didn't push all the ROS environment, i will do it later.

[voldemortX]

@Wakarox It would be best if it can include the git history of pytorch_auto_drive so it can indicate which files are deleted already. Nevertheless, I will probably go through them before the weekend.

[Wakarox]

Hello,
I forgot to push the downloaded version, but all you have on GitHub is for lane detection.
I am trying to add your code on ROS, but i have errors that i am trying to debug.

[Wakarox]

Now, i need to put a fixed pre-trained weights in the code, put a fixed config, and do an input and an output in real-time.

[voldemortX]

Now, i need to put a fixed pre-trained weights in the code, put a fixed config, and do an input and an output in real-time.

Which config file would you like to keep? If that is specified. it would be easier to delete redundant codes

[Wakarox]

For the moment, i am using baseline_resnet18_tusimple

[Wakarox]

But i don't find where you do your --config and --checkpoint in the code to have a single config

[voldemortX]

@Wakarox Sorry I can't download your ros project (bad network connection and you have a big checkpoint file in it). I did about all I could for a file-level deletion, hope it would suffice (If you get into errors, try patch up the missing things with the master branch or delete some redundant imports should do the trick). Here: https://github.com/voldemortX/pytorch-auto-drive/tree/resnet18-tusimple

If you want to go deeper, you will have to delete visualization codes, onnx/trt conversion codes, profiling codes, or get into deleting part of some files. What I already deleted are about 20K lines of code.

[Wakarox]

Hello @voldemortX ,
I am using visualization code because i want to see in real-time the updates. Is this wrong ?
I deleted LaneDetVideo because i think LaneDetDir is already taking picture by picture.
But for the moment, i didn't delete any files concerning lane detection.

[voldemortX]

Hello @voldemortX , I am using visualization code because i want to see in real-time the updates. Is this wrong ? I deleted LaneDetVideo because i think LaneDetDir is already taking picture by picture. But for the moment, i didn't delete any files concerning lane detection.

That should be correct. But you can mod them to only inference if you don't really need to draw anything.

[Wakarox]

So i can't use prediction ?

[voldemortX]

So i can't use prediction ?

Could you elaborate on this? I don't quite get the meaning.

[Wakarox]

I am using the model baseline_resnet18_tusimple with his pretrained weights.
I want to predict the lines in my records with this configuration using my ros records, so in real time.

[voldemortX]

I am using the model baseline_resnet18_tusimple with his pretrained weights.
I want to predict the lines in my records with this configuration using my ros records, so in real time.

Well I think this usage aligns more with the visualization codes.

[Wakarox]

And later, with all the points, i will calculate the curvature of the lines. So i want to know where the points are calculated. And i want to delete the args, so i need to add config, input and output in the same file. For the moment, when i launch with rosrun, i have this error :
[ERROR] [1652861251.852334]: bad callback: <bound method listener.callback of <main.listener object at 0x7fd4381075b0>>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/mokhtarbouain/Documents/work_lane_detection/src/Deep_Learning_detection_lane/scripts/deep_learning_lane_detection.py", line 29, in callback
cv2_image = LaneDetDir(LaneDetVisualizer)
File "/home/mokhtarbouain/Documents/work_lane_detection/src/common_tools_pkg/src/common_tools_pkg/utils/runners/lane_det_visualizer.py", line 85, in init
super().init(cfg)
File "/home/mokhtarbouain/Documents/work_lane_detection/src/common_tools_pkg/src/common_tools_pkg/utils/runners/base.py", line 245, in init
super().init(cfg)
File "/home/mokhtarbouain/Documents/work_lane_detection/src/common_tools_pkg/src/common_tools_pkg/utils/runners/base.py", line 45, in init
self.model = MODELS.from_dict(cfg['model'])
TypeError: 'ABCMeta' object is not subscriptable

I thought it's because my input is just the picture and not all the configs what i need to launch the detection lane code.
Maybe you could help me ?

[voldemortX]

Sorry I don't think I know this kind of error. Maybe there are some problems because of the deletions? If you don't want configs, you can use the Classes yourself by putting in init() args according to the configs.

[Rsweater]

You can see if there is opencv2 under /usr/include, if not, see if it contains opencv4/opencv2, if so, you can use sudo ln -s /usr/include/opencv4/opencv2 /usr/include/ to link it directly to the parent directory.

[shifan-Z]

The include is now good, i found something like you said yesterday. But now, i have a problem with the shared libraries.

CXX/LD evaluate /usr/bin/ld: build/src/evaluate.o: undefined reference to symbol '_ZN2cv6imreadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi' /usr/bin/ld : /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 : erreur lors de l'ajout de symboles : DSO missing from command line collect2: error: ld returned 1 exit status make: *** [Makefile:44 : evaluate] Erreur 1

I encountered the same issue, and my solution was to check the installation path of the openCV library files and modify the Makefile's LIBRARY_DIRS to /usr/local/lib.