Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record Multiple Cameras with python #8183

Closed
brunovollmer opened this issue Jan 18, 2021 · 10 comments
Closed

Record Multiple Cameras with python #8183

brunovollmer opened this issue Jan 18, 2021 · 10 comments

Comments

@brunovollmer
Copy link


Required Info
Camera Model D435 + T265
Firmware Version 05.12.09.00
Operating System & Version Ubuntu 20.02
Kernel Version (Linux Only) 5.4.0-56-generic
Platform PC
SDK Version legacy / 2.40.0
Language Python
Segment Robot

Issue Description

Hey,

my goal is to record synced footage from my multicam setup, which consists of a D435 and a T265 camera. I used this issue as a starting point for my record script (attachted below). When I run the script I receive the following error:

Traceback (most recent call last):
  File "scripts/record.py", line 133, in <module>
    main()
  File "scripts/record.py", line 127, in main
    record(args.output, args.time, args.width, args.height)
  File "scripts/record.py", line 81, in record
    depth_sensor.set_option(rs.option.inter_cam_sync_mode, 1)
RuntimeError: object doesn't support option #42

Anybody an idea how to fix this?

Additionally I'm wondering if it is possible to record in the same .bag file?

Record Script:

def find_cameras(ctx):
    devices = ctx.devices

    data = {}

    for dev in devices:
        name = dev.get_info(rs.camera_info.name)
        serial_number = dev.get_info(rs.camera_info.serial_number)

        data[name] = serial_number
        print(
            f"found camera with name: {name} and serial number: {serial_number}"
        )

    return data


def record(output_path, record_time, width, height):
    context = rs.context()
    camera_data = find_cameras(context)

    base_path = '/'.join(output_path.split("/")[:-1])
    file_name = output_path.split("/")[-1]
    
    cfg_depth = rs.config()
    cfg_depth.enable_record_to_file(f"{base_path}/depth_{file_name}")
    cfg_depth.enable_stream(rs.stream.depth,
                        width=width,
                        height=height,
                        format=rs.format.z16,
                        framerate=30)
    cfg_depth.enable_stream(rs.stream.color,
                        width=width,
                        height=height,
                        format=rs.format.bgr8,
                        framerate=30)
    cfg_depth.enable_device(camera_data["Intel RealSense D435"])

    pipe_depth = rs.pipeline()

    cfg_pose = rs.config()
    cfg_pose.enable_record_to_file(f"{base_path}/pose_{file_name}")
    cfg_pose.enable_stream(rs.stream.pose)
    cfg_pose.enable_device(camera_data["Intel RealSense T265"])

    pipe_pose = rs.pipeline()
            
    depth_profile = pipe_depth.start(cfg_depth)
    pose_profile = pipe_pose.start(cfg_pose)

    depth_sensor = depth_profile.get_device().first_depth_sensor()
    depth_sensor.set_option(rs.option.inter_cam_sync_mode, 1)

    pose_sensor = pose_profile.get_device().first_pose_sensor()
    pose_sensor.set_option(rs.option.inter_cam_sync_mode, 3)

    time.sleep(record_time)

    pipe_depth.stop()
    pipe_pose.stop()

    del pipe_depth
    del pipe_pose

    del cfg_depth
    del cfg_pose
@sam598
Copy link

sam598 commented Jan 20, 2021

inter_cam_sync_mode is used to synchronize RealSense depth cameras that have been wired together. See this article for more information:
https://dev.intelrealsense.com/docs/external-synchronization-of-intel-realsense-depth-cameras#section-3-camera-connections

Unfortunately the T265 does not support any kind of hardware synchronization. If you try to set inter_cam_sync_mode on the T265 you will get an error.

If you need a precise synchronization between the D435 and T265 you will need to find some way to align them in software, which is not an easy task to do reliably. If you have a D435i there may be a way to align the IMU data with the T265 pose data.

@brunovollmer
Copy link
Author

My solution currently is to record both cameras in two separate depth files and then find the closest pose frame timestamp for each color + depth frame timestamp. This works fine but seems kind of hacky to me. Would still be interested in a more "developed" solution if there exists one.

@RealSenseSupport
Copy link
Collaborator

@brunovollmer Can rosbag record function meet your requirement? It can record all currently available stream 'topics' using its -all condition. You can launch each camera independently in a separate ROS 'terminal' window, and the collective topics from all of the camera sources should get captured by the 'all' function of rosbag record. For more details, you can refer to the link in https://wiki.ros.org/rosbag/Commandline#rosbag_record

@brunovollmer
Copy link
Author

This would be a workaround but I would prefer a single script to record.

@RealSenseSupport
Copy link
Collaborator

@brunovollmer Sorry at present we don't have such plan to add new feature to record multiple cameras into one bag file. Please keep using the workaround you're using now. Thanks for your understanding!

@RealSenseSupport
Copy link
Collaborator

@brunovollmer Any other questions about this? If no, will close it accordingly. Thanks!

@alkasm
Copy link
Contributor

alkasm commented Apr 20, 2021

As a related aside to this issue, the specific RuntimeError: object doesn't support option #42 error seems to be due to setting inter_cam_sync_mode after the pipeline has started. Note that @brunovollmer's error is actually thrown on setting the option on the D435, before the script attempts to set it on the T265.

I believe that this was previously allowed, though didn't actually set anything until the device was stopped and started again (per other issues on this repo). Now it seems to actually throw this error if you attempt it at runtime. If you set it before you start the pipeline, e.g. if you set it on the device from a context, then it seems to be ok. For example:

ctx = rs.context()
device = ctx.devices[0]
sensor = device.first_depth_sensor()
sensor.set_option(rs.option.inter_cam_sync_mode, 1.0)
pipeline = rs.pipeline(ctx)
profile = pipeline.start()

@RealSenseSupport
Copy link
Collaborator

@alkasm Thanks for your sharing!
@brunovollmer Do you get chance to try what @alkasm suggested? Looking forward to your update. Thanks!

@RealSenseSupport
Copy link
Collaborator

@brunovollmer Any other questions about this ticket? Could you please update? Thanks!

@brunovollmer
Copy link
Author

No thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants