1. Describe your pipeline. As part of the description, explain how you modified the draw_lines() function.
My pipeline consisted of 9 steps.
- Create a copy of the input image for processing into a line overlay image.
- Convert the image into Grayscale.
- Normalize the image values once in Grayscale.
- Apply a Gaussian Blurr filter with a kernel size = 5. This smoothes out the edges, making the small edges less detectable, and therefore making the edge detection process more robust.
- Apply a Canny Edge Detection filter with a low threshold of 50 and a high threshold of 110.
- Crop the resulting image down to a specific Region of Interest. The region chosen maximizes the removal of points above the horizon, and to the left and right of the vehicle.
- the Hough Transform is then applied to the Region of Interest. Here the shorter lines (< 40 px) are ignored. The rest of the detected lines are set to be merged over a gap distance of 25 px.
- The resulting Hough Line List is then used to generate 2 lines, each representing the bondaries of the Lane on the left and right side of the car. The function
left_right_lineis explained in more detail below.
- The resulting image obtained from the pipeline is then combined with the original image.
The first step in line processing is to determine which lines belong on the left group and which belong to the group on the right. This is accomplished by calculating the slope of each line. According to the image perspective, if the slope is negative, the line belongs to the left group, otherwise, if the slope is positive, the line belongs to the right group. It is asumed that the car trevels on a parallel path to the line. this asumptions allows us to eliminate any line which do not have a sufficient slope to be a close match to lane line geometry (any slope with an absolute value below 0.5).
The second step is to extrapolate the grouped lines into a single representative line for each group. To generate such lines it is assumed that they extend towards the rear of the car and beyond the horizon, this assumption allows us to use the lower and upper limit of the Region of interest as the starting and ending point of the lines, thus minimizing the complexity of the problem to just finding the correct "X" values for each point of each line segment.
In order to find the correct x values for each point, a function f(y) = x is needed. such function should be a linear fit to the point data given by each group. This is performed by using numpy's polyfit and poly1d operations to generate a linear function that matched the two input spaces (x and y). With these linear polynomial functions, its is possible then to calculate the corresponding "X" values for the previously defined "Y" values. The resulting (x, y) pairs are the beginning and end of the line segments.
The algorithm assumes lane lines to be fairly consistent in terms of orientation, the line extrapolation operation makes a broad assumption that lane lines have extreme slopes. This works only when processing images of a car traveling parallel to a mostly straight road. In the case of a curved road the slopes of each detected line may be dampened as a part of the curve. also when a car is not traveling parallel to the lane (entering a road, or changing lanes) the orientation/slope of the lines might not be extreme enough to be detected.
it might be posible to improve the line detection by running parallel pipelines to gain the benefits of different options. Each approach could use different color spaces, or different parameters for each step of the pipeline.
There are many frames of a given video where the tuned parameters detect lines with suden changes on its parameters. since the images processed come from a video, can be fairly asumed that the line detected should have similar paramters (slope, starting point) than the previously detected lines. perhaps a EKF could be applied to track the state change of the line and decay the suden changes of the detected lines.