AP_camera: Camera tracking example scripts (WIP) #27903
Conversation
c9c3a19 to
7de2491
Compare
|
Could you use an RTSP stream direct without needing to use gstreamer? |
|
@timtuxworth , yeah there are workarounds, just need to catch the rtsp stream and give it to cv2 capture |
| float('nan'), # focal_length | ||
| float('nan'), # sensor_size_h | ||
| float('nan'), # sensor_size_v | ||
| 640, # resolution_h |
There was a problem hiding this comment.
ideally we should pull these resolutions from the rtsp stream that we're using. Not a blocker though
| def __init__(self, video_source=None): | ||
| # Set default video source if not provided | ||
| if video_source is None: | ||
| video_source = 'udpsrc port=5600 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! videoconvert ! appsink' |
There was a problem hiding this comment.
I thought that with the siyi cameras, only H265 worked. I don't think H264 streaming works with them
| 640, # resolution_h | ||
| 480, # resolution_v | ||
| 0, # lens_id | ||
| 4095, # flags |
There was a problem hiding this comment.
SHould use enumeration entries rather than just a number. Maybe create a flags local variable
cfc9193 to
d70f55d
Compare
|
|
||
|
|
||
| def main(): | ||
| gimbal_control = GimbalControl('127.0.0.1:14560') |
There was a problem hiding this comment.
some argparse arguments and documentation would be nice
d70f55d to
2d75399
Compare
| self.out = self.initialize_video_writer() | ||
| self.frame = None | ||
|
|
||
| self.tracker = cv2.legacy.TrackerCSRT_create() # Change to legacy namespace |
There was a problem hiding this comment.
do you think it makes sense to add a KCF option here aswell via constructor?
this would allow people more easy to switch for a faster variant on ressource restricted platforms
There was a problem hiding this comment.
@julled Thanks, i was thinking similar. 😃
| gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools | ||
| ``` | ||
|
|
||
| 3. default opencv from pip should be uninstalled and again built with gstreamer support (If already did this go to next step) |
There was a problem hiding this comment.
I think we should avoid requiring a custom build of opencv to include gstreamer support, and instead use standard packages. We can add a class to convert from the gstreamer frames to ones compatible with opencv, such as the example here: https://www.ardusub.com/developers/opencv.html
| diff_x = (centre_x_copy - (640 / 2)) / 2 | ||
| diff_y = -(centre_y_copy - (480 / 2)) / 2 | ||
|
|
||
| self.send_gimbal_manager_pitch_yaw_angles(float("NaN"), float("NaN"), math.radians(diff_y), math.radians(diff_x)) |
There was a problem hiding this comment.
I think we're going to need PI(D) controllers to set gains on the conversion from (diff_x, diff_y) to (pitch_rate_rads, yaw_rate_rads). From initial testing with a SIYI A8, P gains of around 0.1 are a reasonable start. Unadjusted, the gimbal overshoots and tracking is lost immediately.
| norm_x = msg.param1 | ||
| norm_y = msg.param2 | ||
| norm_w = msg.param3 | ||
| norm_h = msg.param4 |
There was a problem hiding this comment.
https://mavlink.io/en/messages/common.html#MAV_CMD_CAMERA_TRACK_RECTANGLE
- param3 is bottom right corner x not width
- param4 is bottom right corner y not height
|
@khanasif786 - I've rebased this PR on master here: https://github.com/srmainwaring/ardupilot/tree/mav_cmd_camera_example_scripts.
|
|
A couple of small things:
|
This PR realtes to #27773
There are currently two scripts more scripts may be added.