✨ If you need a README in Korean, please check the 'korean' branch.
✨한국어로 된 readme가 필요하시다면 'korean' branch를 확인해주세요
Lane Detection Using Hough Transform and Bird's Eye View
Computer Vision Class for the Second Semester of 2023
- Select ROI (Region of Interest)
- Transform to Bird’s-eye view
- Convert to HSV color space
- Adaptive binarization of brightness channel (V)
- Noise removal (closing operation)
- Edge detection with Canny edge operator
- Lane candidate detection with Hough Transform
- Representative lane processing
- Lane merging
- Filtering horizontal lines
- Filtering when lanes are detected too close together
- Additional processing
- Creating a virtual lane if only one lane is detected
- Expanding and marking the road area
- Screen color transition and warning sound when changing lanes
// roi 설정
//선택한 점을 roiPoints 벡터에 추가하고, 선택한 점을 원으로 표시
void selectROI(int event, int x, int y, int flags, void* userdata) {
if (event == EVENT_LBUTTONDOWN) {
roiPoints.push_back(Point2f((float)x, (float)y));
std::cout << "Point selected: " << x << ", " << y << std::endl;
circle(*(Mat*)userdata, Point(x, y), 5, Scalar(0, 255, 0), FILLED);
imshow("Frame", *(Mat*)userdata);
if (roiPoints.size() == 4) {
std::cout << "ROI선택완료" << std::endl;
destroyWindow("Frame");
}
}
}
Before the video starts, the user manually specifies the ROI (Region of Interest) through four clicks on the frame screen to define the area. The selected points are marked with green dots. Once all four points are specified, the lane detection video plays automatically, taking the defined ROI into account. If the ROI values are not specified, an error occurs with the message "ROI selection incomplete." In real driving scenarios, the ROI values can be fixed according to the vehicle for lane detection. However, in this project, since lane detection is performed through a video, the user is allowed to select the ROI to increase usability.
// 원근 변환을 위한 ROI 점 저장
Point2f perspectiveROI[4];
perspectiveROI[0] = Point2f(0, 0);
perspectiveROI[1] = Point2f(originalSize.width - 1, 0);
perspectiveROI[2] = Point2f(0, originalSize.height - 1);
perspectiveROI[3] = Point2f(originalSize.width - 1, originalSize.height - 1);
//원근 변환 행렬
Mat Matrix = getPerspectiveTransform(roiPoints.data(), perspectiveROI);
//역 변환 행렬
Mat inverseMatrix = getPerspectiveTransform(perspectiveROI, roiPoints.data());
The warpPerspective function is used to calculate a transformation matrix to perform a perspective transformation. This converts the ROI values selected by the user into a Bird’s-eye view. Converting to a Bird's-eye view ensures that the slopes of both lanes appear constant, which is advantageous for lane detection. If there's a point (x, y) in a 2D plane, and it is transformed to (x', y') using a transformation matrix, then the transformation matrix M can be denoted as follows.
By using the inverse transformation matrix (inverseMatrix) later on, the lanes detected in the Bird's-eye view can be transformed back to align with the original image.
// HSV 색 공간으로 변환
Mat hsvbirdEyeView;
cvtColor(birdEyeView, hsvbirdEyeView, COLOR_BGR2HSV);
Mat vChannel;
Mat channels[3];
split(hsvbirdEyeView, channels);To analyze only the brightness values from the original RGB color space image, the color space is converted to HSV.
▲ Appearance after binarization
▲ Lane detection using threshold (sensitive to changes in illumination)
▲ Lane detection using adaptiveThreshold
Binarization is performed on the brightness channel V obtained after converting the color space to HSV to detect lanes. At this time, adaptiveThreshold, which divides the image into small areas and uses different threshold values for each area, is utilized. This method adapts to different lighting conditions across various parts of the image, allowing for binarization that performs better than standard methods, especially in conditions with changing illumination such as entering or exiting tunnels. The parameter ADAPTIVE_THRESH_MEAN_C is used to leverage the mean value of neighboring areas as the threshold value.
//잡음제거를 위한 커널 사이즈 정의
Mat kernel = getStructuringElement(MORPH_RECT, Size(5, 5));
Mat kernel2 = getStructuringElement(MORPH_RECT, Size(5, 5));
// 닫힘
morphologyEx(binarybirdEyeView, binarybirdEyeView, MORPH_CLOSE, kernel2);
▲ Without noise removal
▲ After noise removal using the closing operation
For noise removal, the 'closing' operation is utilized, which significantly reduces noise compared to before. To perform the closing operation, the parameter MORPH_CLOSE is used in the function morphologyEx.
//케니 에지 연산자로 에지 검출
Canny(binarybirdEyeView, edges, 50, 150);
▲ The third image with detected edges
Edges are detected in the binarized image, which has undergone noise removal, using the Canny method. A low threshold of 50 and a high threshold of 150 are set.
// 허프변환으로 차선 검출
vector<Vec4i> lines;
HoughLinesP(edges, lines, 1, CV_PI / 180, 40, 90, 300);To detect lanes, line segments are first detected in the image with detected edges. Here, the HoughLinesP function is used to probabilistically inspect pixels. Rho is set to 1, and theta is set to CV_PI/180 to detect lines in all directions. The threshold is set to 40, the minimum line length to detect, minLineLength, is 90, and the maximum distance between points on a line, maxLineGap, is set to 300.
▲ Appearance of lanes being confused due to road maintenance marks
▲ Only the outermost lane is marked
There were instances where black road maintenance marks were mistaken for lanes. To solve this, lanes with similar slopes were merged, and only the outermost lane was recognized to reduce the frequency of false data detection. For lane merging, a function named merge_lines was created. It takes a vector named lines as input and merges lanes with similar slopes. For the outermost lanes, leftMost is initialized to INT_MAX since it represents the farthest right end point, and rightMost is initialized to 0 as it represents the farthest left end point. Then, the leftmost lane and the rightmost lane for each lane were detected.
▲ Horizontal lines on the road being recognized as lanes
Occasionally, horizontal lines appear on the road. These were filtered out based on their slope values. Lines with a slope less than 1 were considered horizontal and were not displayed.
▲ Two lanes appearing close together and near the center of the image
▲ After filtering processing
When changing lanes during driving or when lane detection is unstable, there are instances where lanes appear too close to the image's center or two lanes are detected too closely together. To filter these cases, a threshold for the distance between two lanes was set, and lanes closer than this threshold were not displayed.
// 한쪽 차선만 검출될 때 반대편에 가상의 차선을 그려주는 함수
Vec4i helperLine(Mat& frame, const Vec4i& line, const Mat& inverseMatrix, Scalar color, int frameCenterX) {
// 라인에서 포인트 추출
Point pt1 = Point(line[0], line[1]);
Point pt2 = Point(line[2], line[3]);
// 선 중심점
Point midPoint = (pt1 + pt2) * 0.5;
// 프레임 센터
int mirrorMidX = frameCenterX + (frameCenterX - midPoint.x);
// 거리 계산
int dx = (pt1.x - midPoint.x);
int dy = (pt1.y - midPoint.y);
// 반대편에 가상의 차선 계산
Point mirrorPt1 = Point(mirrorMidX - dx, pt1.y);
Point mirrorPt2 = Point(mirrorMidX + dx, pt2.y);
// 포인트를 역변환행렬을 이용하여 다시 원위치로 계산
vector<Point2f> linePoints, transLinePoints;
linePoints.push_back(Point2f(mirrorPt1.x, mirrorPt1.y));
linePoints.push_back(Point2f(mirrorPt2.x, mirrorPt2.y));
perspectiveTransform(linePoints, transLinePoints, inverseMatrix);
// Point2f를 Point로 변환
Point fakePt1 = Point(cvRound(transLinePoints[0].x), cvRound(transLinePoints[0].y));
Point fakePt2 = Point(cvRound(transLinePoints[1].x), cvRound(transLinePoints[1].y));
// 가상의 차선 그리기
if (fakePt1.x >= 0 && fakePt1.x < frame.cols && fakePt2.x >= 0 && fakePt2.x < frame.cols) {
cv::line(frame, fakePt1, fakePt2, color, 3, cv::LINE_AA);
}
Vec4i fakeLine;
fakeLine[0] = mirrorPt1.x;
fakeLine[1] = mirrorPt1.y;
fakeLine[2] = mirrorPt2.x;
fakeLine[3] = mirrorPt2.y;
return fakeLine;
}
▲ Pink lane: Arbitrarily created lane (not actually detected)
When one lane is faint, a lane on the opposite side is drawn based on the detected lane. If the left lane is detected, the right lane is created, and vice versa. This process allows for more stable driving when one lane is faint or missing.
▲ Photo before road area expansion
▲ Photo after road area expansion
The detected lanes are used to identify the road area. There was an issue where the road appeared narrower in cases where very short lanes were detected. To address this, the length of the lane is extended using the equation of the line and two points of the lane. Based on this, the road area within the ROI is marked.
bool warningsound = false;
// 25회 이상 검출 불가일 경우 화면을 붉은색으로 바꿔서 경고를 줌
if (warning && missingLaneCounter > 25) {
if (redOverlay.empty()) {
redOverlay = Mat::zeros(frame.size(), frame.type());
redOverlay.setTo(Scalar(0, 0, 255));
warningsound = true;
}
const double alpha = 0.5;
addWeighted(finalLaneFrame, 1 - alpha, redOverlay, alpha, 0, finalLaneFrame);
}
if (warningsound) {
system("C:/opencv_vision/warning_sound.mp3");
warningsound = false;
}
If both lanes are not detected in a specific frame, a warning is accumulated once. If this warning accumulates more than 25 times consecutively, it is recognized as a lane change or lane departure, changing the screen to red and emitting a warning sound to alert the driver. The system is designed to request the driver to pay attention to the road ahead through warnings in cases of lane changes or undetected lanes.
