Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance route-check for multi-asic platforms #3216

Conversation

deepak-singhal0408
Copy link
Contributor

What I did

Enhanced route_check.py script to cover multi-asic platforms.
Accordingly enhanced the test files as well.

MSFT ADO: 25416673

How I did it

Enhanced the route_check.py script to take additional optional parameter(--n/namespace). Without this parameter, the check will be run on all asics in multi-asic platforms.
Different connections to DBs are modified accordingly to connect to relevant ns dbs.
Result will be encapsulated under different namespace. For single asic, results will be displayed under Default-Namespace("").

testData and the testfiles are enhanced accordinly.

How to verify it

  1. Verified that all pytest UT cases are passing.
  2. Verified the route_check_test.sh script on single asic and multi-asic platforms.
  3. Verified Monit routecheck outputs by simulating a failure scenario on both single asic and multi-asic platforms.

output from Monit Check:
Single Asic:
xxx/usr/local/bin# monit status routecheck
Monit 5.20.0 uptime: 1d 20h 32m

Program 'routeCheck'
status Status failed
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 255
last output Failure results: {{
"": {
"missed_ROUTE_TABLE_routes": [
"20c0:d9b8:99:80::/64"
]
}
}}
Failed. Look at reported mismatches above
add: {
"": []
}
del: {
"": []
}
data collected Tue, 12 Dec 2023 20:30:11

'''
Multi Asic:
'''
/bin# monit status routecheck
Monit 5.20.0 uptime: 1d 23h 51m

Program 'routeCheck'
status Status failed
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 255
last output Failure results: {{
"asic0": {
"missed_ROUTE_TABLE_routes": [
"1.0.0.0/16"
]
},
"asic1": {
"missed_ROUTE_TABLE_routes": [
"1.0.0.0/16"
]
},
"asic2": {
"missed_ROUTE_TABLE_routes": [
"1.0.0.0/16"
]
}
}}
Failed. Look at reported mismatches above
add: {
"asic0": [],
"asic1": [],
"asic2": []
}
del: {
"asic0": [],
"asic1": [],
"asic2": []
}
data collected Tue, 12 Dec 2023 23:54:23
'''

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

@deepak-singhal0408
Copy link
Contributor Author

resubmit reverted PR #3077
The regression due to #3077 has been fixed via #3130

@deepak-singhal0408
Copy link
Contributor Author

deepak-singhal0408 commented Mar 14, 2024

@prsunny @judyjoseph @abdosi this PR was earlier reverted due to an issue got exposed on multi-asic T1, which has been fixed now via #3130 . Changes from this PR are already merged into 202205 and 202305(as the FRR check code is only present in master).. Kindly help review the changes for master.

@deepak-singhal0408
Copy link
Contributor Author

@prsunny @qiluo-msft could you please help review/Merge this PR? Thanks!

@qiluo-msft qiluo-msft merged commit 3c489ba into sonic-net:master Mar 22, 2024
5 checks passed
arfeigin pushed a commit to arfeigin/sonic-utilities that referenced this pull request Apr 2, 2024
### What I did
Enhanced route_check.py script to cover multi-asic platforms.
Accordingly enhanced the test files as well.

MSFT ADO: 25416673

#### How I did it
Enhanced the route_check.py script to take additional optional parameter(--n/namespace). Without this parameter, the check will be run on all asics in multi-asic platforms.
Different connections to DBs are modified accordingly to connect to relevant ns dbs.
Result will be encapsulated under different namespace. For single asic, results will be displayed under Default-Namespace("").

testData and the testfiles are enhanced accordinly.

#### How to verify it
1. Verified that all pytest UT cases are passing.
2. Verified the route_check_test.sh script on single asic and multi-asic platforms.
3. Verified Monit routecheck outputs by simulating a failure scenario on both single asic and multi-asic platforms.


output from Monit Check:
**Single Asic:**
xxx/usr/local/bin# monit status routecheck
Monit 5.20.0 uptime: 1d 20h 32m

Program 'routeCheck'
  status                       Status failed
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  last exit value              255
  last output                  Failure results: {{
                                   "": {
                                       "missed_ROUTE_TABLE_routes": [
                                           "20c0:d9b8:99:80::/64"
                                       ]
                                   }
                               }}
                               Failed. Look at reported mismatches above
                               add: {
                                   "": []
                               }
                               del: {
                                   "": []
                               }
  data collected               Tue, 12 Dec 2023 20:30:11

'''
**Multi Asic:**
'''
/bin# monit status routecheck
Monit 5.20.0 uptime: 1d 23h 51m

Program 'routeCheck'
  status                       Status failed
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  last exit value              255
  last output                  Failure results: {{
                                   "asic0": {
                                       "missed_ROUTE_TABLE_routes": [
                                           "1.0.0.0/16"
                                       ]
                                   },
                                   "asic1": {
                                       "missed_ROUTE_TABLE_routes": [
                                           "1.0.0.0/16"
                                       ]
                                   },
                                   "asic2": {
                                       "missed_ROUTE_TABLE_routes": [
                                           "1.0.0.0/16"
                                       ]
                                   }
                               }}
                               Failed. Look at reported mismatches above
                               add: {
                                   "asic0": [],
                                   "asic1": [],
                                   "asic2": []
                               }
                               del: {
                                   "asic0": [],
                                   "asic1": [],
                                   "asic2": []
                               }
  data collected               Tue, 12 Dec 2023 23:54:23
'''
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants