Introduction:
Do you want to draw on your screen but don’t have any drawing tools? Virtual Painting has you covered! This real-time object detection application allows you to draw on your screen with colored markers using Python and OpenCV. Object tracking is an essential task in computer vision that involves locating a specific object in a video sequence. You can find the full code file in the GitHub repository I created for this project. In this article, we’ll explore how to implement real-time object tracking using OpenCV and Python.
The Process:
The process of creating the marker tracking system can be broken down into several steps: detecting colors, finding contours, and identifying shape centers.
- Detecting Colors:
The first step in the process is to detect the colors of the markers. We use the OpenCV library to perform this task. In our code, we define two colors to detect: blue and yellow. For each color, we specify a range of HSV (hue, saturation, value) values that correspond to that color. We then convert each frame of the video to the HSV color space and use the cv2.inRange() function to create a binary mask that indicates which pixels in the frame fall within the specified color range.
def detect_color(frame, color):
"""
Detects a specified color in a frame and returns the masked frame.
"""
frame_hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
lower = np.array(color[0:3])
upper = np.array(color[3:6])
mask = cv2.inRange(frame_hsv, lower, upper)
masked_frame = cv2.bitwise_and(frame, frame, mask=mask)
return masked_frame- Finding Contours:
Once we have the masked frames for each color, we need to find the contours of the markers in the frames. Contours are the boundaries of objects in an image, and can be used to identify the shapes of objects. In our code, we use the cv2.findContours() function to find the contours in the masked frames. We then filter the contours based on their size, only keeping those with an area greater than a certain threshold.
def find_contours(masked_frame):
"""
Finds contours in a masked frame and returns a list of contours.
"""
frame_gray = cv2.cvtColor(masked_frame, cv2.COLOR_BGR2GRAY)
frame_blur = cv2.GaussianBlur(frame_gray, (7, 7), 1)
frame_edges = cv2.Canny(frame_blur, 50, 50)
contours, _ = cv2.findContours(frame_edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
return contours- Identifying Shape Centers:
Finally, we need to identify the centers of the shapes represented by the contours. To do this, we use the cv2.boundingRect() function to find the bounding rectangle of each contour, which gives us the x and y coordinates of the top-left corner of the rectangle, as well as its width and height. We then calculate the center of the rectangle by adding half the width to the x coordinate of the top-left corner, and half the height to the y coordinate of the top-left corner.
def find_shape_center(contour):
"""
Finds the shape of a contour and returns the number of corners and the bounding rectangle.
"""
area = cv2.contourArea(contour)
if area > 50: # and area < 350
x, y, width, height = cv2.boundingRect(contour)
center_x, center_y = int(x + (width / 2)), int(y + (height / 2))
return center_x, center_y
return None, None- Drawing Markers:
Once we have identified the centers of the markers, we can draw them on the video frames. We use the cv2.circle() function to draw circles at the marker centers, and we color each marker according to its corresponding color.
def draw_points(frame, point):
"""
Draws a point on a frame.
"""
colors_to_draw = [
# BLUE
(255, 0, 0),
# YELLOW
(0, 255, 255)]
cv2.circle(frame, [point[0], point[1]], 10, colors_to_draw[point[2]], cv2.FILLED)- Main Loop:
In the main loop, the code reads a frame from the video capture, applies color detection to the frame using detect_color, finds the contours in the masked frame using find_contours, and finds the center coordinates of the contours using find_shape_center. The center coordinates and color of the contours are added to the points list. Finally, the draw_points function is called to draw circles on the frame at the center coordinates of the detected contours. The resulting frame is then displayed in a window. The loop continues until the user presses q to quit or c to clean the markers.
points = []
colors_to_detect = [
# BLUE
[ 80, 167, 0, 132, 255, 255],
# YELLOW
[24,86, 194, 65, 234, 255]
]
video_capture = cv2.VideoCapture(0)
width, height = 640, 480
brightness = 0
video_capture.set(3, width)
video_capture.set(4, height)
while True:
success, frame = video_capture.read()
for color in colors_to_detect:
masked_frame = detect_color(frame, color)
contours = find_contours(masked_frame)
for contour in contours:
x, y = find_shape_center(contour)
if x is not None or y is not None :
points.append([x, y, colors_to_detect.index(color)])
if len(points) != 0:
for point in points:
draw_points(frame,point)
frame_h = cv2.flip(frame, 1)
cv2.imshow("Result", frame_h)
if cv2.waitKey(1) == ord('q'):
# quit
break
elif cv2.waitKey(1) == ord('c'):
# clean marker
points = []
By combining these steps, we are able to create a real-time marker tracking system that can detect and track markers of different colors. This system could be used in a variety of applications, such as robotics, gaming, and augmented reality.
Conclusion
In this article, we’ve seen how to implement real-time object tracking using OpenCV and Python. We used the `detect_color` function to highlight specified colors in a frame, the `find_contours` function to find contours in the masked frame, and the `find_shape_center` function to find the center coordinates of the shapes in the contours. We also used the `draw_points` function to draw a point on the frame for each center coordinate found. With these functions, we were able to track objects in real-time using a video feed.
In conclusion, Virtual Painting is a fun and easy way to draw on your screen without any physical drawing tools. With a few lines of code and some customization, you can create a unique drawing experience that is perfect for all ages. So why not give it a try and unleash your inner artist?