Map Based Prediction Incorrect Predictions

[08beeqtufail-sudo]

Hello Everyone,
I am testing the proposed map based prediction with some of the bag files and a map that I have at my disposal. I am facing the problem that the map_based_prediction outputs some strange trajectories (e.g, a trajectory going backwards for an object moving straight in its lane etc...) .After digging in the code a little bit, i can see that the condition to check whether to do a linear prediction or the normal prediction is decided based only on the object velocity x component (object.kinematics.twist_with_covariance.twist.linear.x) . But an object's velocity in map frame is not only x component. So i would like to know the rationale behind this. Thank you for your time and great work.
Regards,
M.Tufail

[yukkysaito]

@08beeqtufail-sudo
Thanks for the report.

We are currently working on a major revision and may be able to cure it there.

https://github.com/autowarefoundation/autoware.universe/pull/491/files

[08beeqtufail-sudo]

@yukkysaito Thanks for your reply. I am following the repository you mentioned for future updates.

[mitsudome-r]

@08beeqtufail-sudo It seems like the PR mentioned by @yukkysaito is merged. Could you check if the issue still remains? If it still does, please provide a reproducing steps (or a rosbag) so that others can investigate the bug.

[08beeqtufail-sudo]

@mitsudome-r I am currently working on another project but i will test it within a couple of days and let you know.

[08beeqtufail-sudo]

@mitsudome-r @yukkysaito I was able to test the latest map_based_prediction with some bag files. I still have the same problem. I can see in the code that the part that decides whether to do linear prediction or normal prediction still checks the x component of twist of an object. But the tracked objects in my custom bag files are in map_frame. So the objects have a x velocity part and a y velocity part. So the speed of an object should be calculated based on both of these components. The decision to do linear prediction or not, should be based on this speed and not just x-component?
It is worth mentioning that the tracked objects i have in my bag file are coming from an independent perception module, not from the autoware.universe perception module. I just adapt the format in order to use with map_based_prediction module of autoware.universe.
So it is entirely possible that the tracked objects given by autoware.universe perception module only have x components, in that case, using only the x-component of velocity makes sense.

[xmfcx]

@08beeqtufail-sudo could you send some visuals, images/short videos of the issue? It'd help us to understand the actual problem and help in the solving process.

[08beeqtufail-sudo]

Hello @xmfcx ,
Below you will find some of the images from the prediction failure described above. The red-colored rectangle is the object in question, for which the predictions are wrong. Note that this is a right-hand traffic scenario, similar to european countries. I have investigated the problem, some of the problems come from the fact that the estimated velocity is not stable in my case.
But another problem is that the map_based prediction performs a linear prediction based on whether an object's x velocity is less than a certain threshold, which in my case is not a good enough criteria. Because the objects have celocities in map frames and performing linear prediction based on x component value is not sufficient.

[mitsudome-r]

@miursh @yukkysaito do you have any comment on this?

[mitsudome-r]

@miursh @yukkysaito friendly ping

[mehmetdogru]

Myself also facing this issue while #566

While I test UC-PB-008-0001_20_20-Shalun scenario observed some wrong map based predictions for NPCs when they are turning around the roundabout. Here you can see the video.

And this causes some serious issues since vehicle is not able to cross the intersection because of the wrong predicted paths.

[mitsudome-r]

@kaancolak Would be looking into this issue as well.

[mehmetdogru]

@kaancolak

Another input:

map_based_prediction node crashes most of the time while testing UC-PB-008-0001... scenarios with 7593c2b

[openscenario_interpreter_node-3] [map_based_prediction-34] terminate called after throwing an instance of 'std::length_error'
[openscenario_interpreter_node-3] [map_based_prediction-34]   what():  Exceeded upper bound
[openscenario_interpreter_node-3] [ERROR] [map_based_prediction-34]: process has died [pid 530007, exit code -6, cmd '/home/md/projects/autoware/install/map_based_prediction/lib/map_based_prediction/map_based_prediction --ros-args -r __node:=map_based_prediction -r __ns:=/perception/object_recognition/prediction --params-file /tmp/launch_params_w6aapmdb --params-file /home/md/projects/autoware/install/map_based_prediction/share/map_based_prediction/config/map_based_prediction.param.yaml -r objects:=/perception/object_recognition/objects -r /vector_map:=/map/vector_map -r objects:=/perception/object_recognition/objects'].

[angry-crab]

@08beeqtufail-sudo
Hi, thank you for the information. I was wondering if it was possible for you to provide your test data(ros2bag, pcd, lanelet map, and parameter setting).
We will discuss with perception team to identify the correct output in these cases(I guess the predicated path should stay inside lanelet components).

[xmfcx]

@08beeqtufail-sudo ping.

[xmfcx]

@08beeqtufail-sudo could you provide test data so that we can reproduce the issue?

[08beeqtufail-sudo]

@xmfcx , @angry-crab sorry for the delay. I was off for a couple of weeks. I will soon provide you with the test data.

[mitsudome-r]

@08beeqtufail-sudo Will you be able to share the data?

[08beeqtufail-sudo]

@mitsudome-r yes i will share the data, but it might take a couple of days. I will try to do it today, but i don't guarantee it.

[08beeqtufail-sudo]

@mitsudome-r @xmfcx , After having consulted with my project partners, at this stage, we feel that providing the rosbag and lanelet2 map might be an overkill. As i have earlier explained in my post, the problem is correctly identified. The fact that our perception and tracking system is different than that of autoware.universe, means we have x-velocity and y-velocity components of the tracked or detected objects. While in autoware.universe simulation, that's not the case. When i test the autoware.universe simulation, there's no problem. But when i test the autoware.universe planning with our own perception system, that's when the problem arises.
The data we have at our disposal in the bag files or the lanelet2 map, is confidential. Hence i am available if you have any further questions or require any other information. Thank you for your cooperation.
Regards,
M. Tufail

[simon-t4]

Hi ,if I understand correctly, the problem lies in the confusing the twist coordinate frame.
The original Autoware publishes predicted objects using an object centric frame, while this issue describes a perception system where the predicted path is in map frame.

The ROS twist message is I think defined with linear and angular components rather than x,y of a map frame.
The velocity is meant to be independent of pose of object so an object centric frame is preferred.

A simple fix would be for the perception system described to convert x,y components into linear and angular components needed by Autoware (and consistent with ROS message definition) and perhaps publish the two messages as different topics (if x,y frame needed by other components).

[mitsudome-r]

From ASWG:

• @xmfcx and @mitsudome-r agree to use object centric frame for velocity expression
• We still need to update/improve prediction module so that it fills in y component as necessary depending on detected velocity.

[xmfcx]

We will consider this feature while refactoring for the core architecture.

[BonoloAWF]

Noted. Removed from Bus ODD Milestone and Project.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.

[vividf]

I will close this issue since it is stale.