Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update syncd stop script to collect saisdkdump during orch abort #12936

Closed
wants to merge 12 commits into from

Conversation

vivekrnv
Copy link
Contributor

@vivekrnv vivekrnv commented Dec 5, 2022

Signed-off-by: Vivek Reddy Karri vkarri@nvidia.com

Why I did it

  • Update the syncd stop flow to collect saisdkdump and save it to /var/log/orch_abrt_status/ location during a SAI programming failure by orchagent.
    1. Limit the number of such archived to 3
    2. Create a tmp file as a sync mechanism with auto-techsupport process
  • Update the swss stop flow to remove PortInitDone. This is done so as to provide other daemons a reliable way of checking if switch_init is successful after services are deemed started. (Eg: Before running saisdkdump, switch should be initialized. techsupport will use this info to determine that)

How I did it

How to verify it


root@r-bulldog-03:/home/admin# sonic-db-cli STATE_DB SET ORCH_ABRT_STATUS 1

root@r-bulldog-03:/home/admin# systemctl restart swss

Oct 28 21:10:52.892667 r-bulldog-03 INFO syncd.sh[26913]: sai_sdk_dump_1666991448.tar.gz collected before taking stopping syncd

root@r-bulldog-03:/home/admin# tar -tvf /var/log/orch_abrt_status/sai_sdk_dump_1666991448.tar.gz
drwxr-xr-x root/root         0 2022-10-28 21:10 ./
-rw-r--r-- root/root   4883498 2022-10-28 21:10 ./sdkdump_ext_cr_001-28_10_2022-21_10_50-980950.udmp
-rw-r--r-- root/root   3964382 2022-10-28 21:10 ./sai_sdk_dump_10_28_2022_09_10_PM
-rw-r--r-- root/root       288 2022-10-28 21:10 ./sdkdump_ext_meta_001-28_10_2022-21_10_50-407583
-rw-r--r-- root/root       288 2022-10-28 21:10 ./sdkdump_ext_meta_001-28_10_2022-21_10_50-980950
-rw-r--r-- root/root   2656360 2022-10-28 21:10 ./sai_sdk_dump_10_28_2022_09_10_PM.json
-rw-r--r-- root/root   4883498 2022-10-28 21:10 ./sdkdump_ext_cr_001-28_10_2022-21_10_50-407583.udmp
-rw-r--r-- root/root   4883498 2022-10-28 21:10 ./sdkdump_ext_cr_001-28_10_2022-21_10_51-573167.udmp
-rw-r--r-- root/root       288 2022-10-28 21:10 ./sdkdump_ext_meta_001-28_10_2022-21_10_51-573167

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
@lguohan lguohan requested a review from prsunny December 17, 2022 22:34
@vivekrnv vivekrnv closed this Jan 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants