Calculate objects volume with realsense D435i python

[fredericoliz]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	{ R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	{Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version	{ legacy / 2.. }
Language	{C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment	{Robot/Smartphone/VR/AR/others }

I placed a Realsense D435i at the top of a board to track and calculate the volume of objects while passing through a threadmill. I already know the distance between the camera and the threadmill but i'm having issues on how to calculate between bigger/smaller objects.
This is an example of a frame that i get while using the camera:

What i want is to take this "square" and calculate its volume using the depth image its area. Any suggestions?

[MartyG-RealSense]

There is the Python 'Box Dimensioner' SDK example.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples/box_dimensioner_multicam

[RealSenseCustomerSupport]

---

Hi @fredericoliz,

Do you need any further help regarding this topic?

[nirandiw]

Hi @MartyG-RealSense @RealSenseCustomerSupport,

I want to use a point cloud and estimate the volume of an object with a single frame captured from a single Intel RealSense camera. Is there any example to get the volume estimation from a point cloud?

So far I know how to get the point cloud. My code is given below

import pyrealsense2 as rs
import numpy as np
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
pc = rs.pointcloud()
pipeline.start(config)

i = 0
while i<5:

    frames = pipeline.wait_for_frames()
    depth_frame = frames.get_depth_frame()
    print(depth_frame)
    color_frame = frames.get_color_frame()
    print(color_frame)
    i = i+1
    pc.map_to(color_frame)
    points = pc.calculate(depth_frame)
    vtx = np.asanyarray(points.get_vertices())
    tex = np.asanyarray(points.get_texture_coordinates())
    print(type(points), points)
    print(type(vtx), vtx.shape, vtx)
    print(type(tex), tex.shape, tex)
pipeline.stop()

[MartyG-RealSense]

There is the box-dimensioner-multicam Python example for measuring the height, width and length of an object. The instructions say that it opens one output window for every device attached. So conceivably you may get some result from a single camera.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples/box_dimensioner_multicam

The only other Python measuring program I personally know of is by a RealSense user who converted the C++ Measure example to Python and then developed it further.

https://github.com/soarwing52/RealsensePython

[nirandiw]

Thanks. I'll take a look. Is there a cpp program that allows volume estimation from a point cloud?

[MartyG-RealSense]

When considering point cloud processing tasks such as volume estimation, it may be best to consider using an external 3D library with your RealSense camera, such as PCL (which the RealSense SDK supports through a 'wrapper' program) or Open3D.

My research of the subject indicates that the popular opinion is that the point cloud should be converted to a 3D mesh before doing the volume estimation. Example:

https://stackoverflow.com/questions/55629892/2019-point-cloud-volume-estimation-with-c/55681149#55681149

During the course of my investigation, I was reminded of the ability of the Python-based pyntcloud to do this mesh conversion. pyntcloud works with the RealSense 400 Series cameras.

https://stackoverflow.com/questions/44997054/point-cloud-to-volume

[nirandiw]

Is there a good reference for using PCL python binding with intelrealsense. Having trouble install PCL in Mac and using it.

[MartyG-RealSense]

The link below has a guide to installing PCL on OSX.

http://pointclouds.org/documentation/tutorials/compiling_pcl_macosx.php

Alternatively there is a PCL Python binding that has been tested with MacOS by its creator.

https://github.com/strawlab/python-pcl

[rcroset]

Hi @nirandiw

Did you manage to find a working example ? I am facing a similar problem and I can't find anything helpful out there. The links mentioned by @MartyG-RealSense were helpful but I'm still unable to estimate any volumes using my realsense D435.

The box-dimensioner-multicam Python example isn't really useful because it requires a chessboard.

I tried computing volumes using pyntcloud, the PCL Python binding and even Open3D. I never was able to make volume estimations. The main issue is to crop the pointcloud to focus on the object to measure. Any hint or solution ?

Thanks a lot :)

[akshayacharya97]

@nirandiw @RealSenseCustomerSupport @MartyG-RealSense I also want to calculate dimensions of my object. I am using darknet, so i m getting the bounding box. However, I do not know how to calcualte actual size of object from this now. Any help would be appreciated