Reduce excessive error and warning logs on Autoware launch

[xmfcx]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I've agreed with the maintainers that I can plan this task.

Description

When Autoware is executed, it generates a significant number of warning and error logs, despite the system running as expected. The logs are not only excessive but also misrepresent the stability of the running application, as they appear even in the absence of actual functional issues. This creates a noisy debugging environment, making it challenging to identify and address genuine issues.

The issue was replicated using the latest main branch with the planning simulator demo as per the Autoware documentation: Planning Simulation Tutorial.

A video demonstrating the issue is available here:

• Autoware Error and Warning Logs Video

The logs from the run that showcase the problem can be found here:

• Autoware Logs Gist

Purpose

The primary goal is to clean up the logs by either resolving the underlying causes of the warnings and errors or by adjusting the threat levels if the logs are not indicative of real issues.
This will aid in de-cluttering the console output, making it easier to debug actual problems, and generally improve the developer experience.

Possible approaches

1. Log Analysis: Go through the logs to determine if the warnings/errors are justified. If they represent real issues, then they should be addressed with bug fixes.
2. Adjust Log Levels: If the logs are overly cautious and do not represent actual errors or critical warnings, then the log levels should be adjusted accordingly.
3. Code Refactoring: Inspect the sections of the code-base that are generating these logs and refactor if necessary to prevent false positives.
4. Configuration Tweaks: Check if the verbosity of the logs can be configured, allowing users to set the desired level of log detail.
5. Documentation Update: If certain logs are expected and do not indicate problems, this should be clearly documented to avoid confusion. (last resort, not preferred)
6. Incremental Fixing: Tackle the errors and warnings in batches, creating multiple issues or pull requests to address them systematically.

Definition of done

• The error and warning logs that are not indicative of actual issues are eliminated or their threat levels are adjusted.
• The logging output should be clear and should accurately reflect the operational status of Autoware.
• Documentation should be updated, if necessary, to reflect any changes made to log management.
• Additional issues should be created for any unresolved logs that require further investigation or long-term changes.
• The Autoware community agrees that the logging is now informative and not excessively verbose, allowing for effective debugging.

[xmfcx]

To clarify, this task encapsulates warning and error logs coming from any part of the Autoware for any reason.

[ahmeddesokyebrahim]

I would like to share updates regarding this issue.

The target of this issue is to reduce excessive log message in Autoware for the following tutorials with default parameters

• Planning Simulation
• Rosbag Replay Simulation
• AWSIM

In this sheet, we are categorize and following up the log messages during the fixing process of this issue.

The initial plan is to have the task into 5 phases :

Phase	Definition
Phase 1	Planning simulation : starting planning simulator and before setting vehicle start location. This phase produces a lot of recurrent message while nodes are initializing
Phase 2	Planning simulation : setting vehicle start location, goal location, and basic driving scenario. This phase where actually the nodes are starting to work and communicate.
Phase 3	Planning simulation : handing special driving scenarios in planning simulator like parking, …
Phase 4	Rosbag replay simulation
Phase 5	AWSIM (extended goal)

The following is the list of relevant PRs for proposing solution fixing the different log messages.
For completing this issue, these PRs must be successully reviewed and merged [list can be updated during development]

• autoware.universe - part 1
• autoware.universe - part 2
• autoware_launch
• autoware_common
• tier4/pointcloud_to_laserscan
• tier4/nebula
• autoware-documentation

[ahmeddesokyebrahim]

This is the list of bugs opened for asking the community to collaborate in fixing some of the logs messages and their associated PRs:

Phase 1, 2, and 3

• BlindSpotModule setStopLineInx fail #6204
• Error during fixed point calculations in path smoother and obstacle avoidance planner #6205
• motion_velocity_smoother Error Message while Parking #6379
• system_error_monitor Error Messages while Parking #6378
• Trajectory topic is timed out while parking #6380
• Emergency stop is operated while parking #6381
• system_error_monitor is reporting vehicle_state_report msg timeout during planning simulation initialization #6383

Phase 4

• Improving nebula std::cout logs  tier4/nebula#122
• system_error_monitor is reporting diag_array msg timeout during rosbag replay simulation initialization #6515

Bug	PR
#6204	#6405
#6205	#6298
#6378	#6696
#6380	#6696
#6381	#6696
#6379	#6713
tier4/nebula#122	tier4/nebula#118
#6383	--
#6515	--

[ahmeddesokyebrahim]

The PRs for the first 3 phases are ready for review.
Recommended Merge Sequence

1. fix(log-messages): reduce excessive log messages autoware_common#224
2. fix(log-messages): reduce excessive log messages #5971
3. fix(log-messages): reduce excessive log messages autoware_launch#760

The following one does not matter in sequence

• fix(log-messages): lower log level for a log message from info to debug tier4/pointcloud_to_laserscan#6 [does not matter]

The following phases will be addressed in separate PRs.

[ahmeddesokyebrahim]

List of PR associated to this issue

Merged :

• fix(log-messages): reduce excessive log messages #5971
• fix(log-messages): reduce excessive log messages autoware_launch#760
• fix(log-messages): reduce excessive log messages autoware_common#224
• fix(obstacle_avoidance_planner, path_smoother): change logger level of debug print #6298
• fix(map_loader): fix warnings with single point cloud map metadata #6384
• refactor(blind_spot): find first_conflicting_lane just as intersection module #6405
• fix(log-messages): lower log level for a log message from info to debug tier4/pointcloud_to_laserscan#6
• fix(log-messages): reduce excessive log messages #6406
• docs(rosbag-replay-simulation): add warnings about timestamp discrepancies and pre-playback messages autoware-documentation#532
• fix(freespace_planner): fix parking trajectory errors and warnings when parking completed #6696
• fix(log-messages): reduce excessive log messages in velodyne decoder tier4/nebula#118
• fix(freespace_planner): fix parking trajectory errors and warnings when parking completed #6696
• fix(freespace_planner): fix motion_velocity_smoother error while parking #6713

Ready for Review

• to be updated if any

WIP

• to be updated if any

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.