-
Notifications
You must be signed in to change notification settings - Fork 59
2. Writing test and config file
Priyal Jain edited this page Apr 6, 2016
·
17 revisions
Config file supports following parameters:
1. hosts: here you have to specify login credential of device to be connected.
If you want to connect to multiple devices then write all login credentials in one yaml file and import it here.
2. tests: specify test files that you want to run
3. sqlite(optional): Use this key if you want to compare/store snapshots in database.
4. mail(optional): Use this key if you want to get notification of test results via mail.
- Sample config file for single device
hosts:
- device: 10.20.1.24
username : foo
passwd: bar
tests:
- test_no_diff.yml
- test_bgp_neighbor.yml
# (optional) use only when you want to store and compare snapshots from database
sqlite:
- store_in_sqlite: True
check_from_sqlite: True
database_name: jbb.db
compare: 1,0
# (optional) use when you want to send mail about test results
mail: send_mail.yml
- Sample config file for connecting to multiple devices
# for multiple devices with databse
hosts:
- include: devices.yml
group: EX
tests:
- test_is_equal.yml
- test_is_in.yml
# (optional) use only when you want to store and compare snapshots from database
sqlite:
- store_in_sqlite: yes
check_from_sqlite: yes
database_name: jbb.db
# (optional) use when you want to send mail about test results
mail: send_mail.yml
- devices.yml
MX:
- 10.20.1.20:
username: root
passwd: root123
- 10.21.13.14:
username: root
passwd: root123
- 10.20.6.26:
username: jsnapy
passwd: jsnapy123
EX:
- 10.2.15.210
username: root
passwd: root123
- 10.9.16.22:
username: abc
passwd: pqr
QFX:
- 10.29.1.24:
username: abc
passwd: pqr
- 10.29.6.1:
username: abc
passwd: pqr123
Purpose of writing test file is to specify command/rpc whose snapshot is
to be taken and what all nodes user wants to test and how to test them.
1. tests_include(optional): use this tag if you want to include only some test cases.
If you do not include this tag, then jsnapy will run all test cases by default.
2. command/rpc : can give either command or rpc to perform testing
- format: can specify output format
[text,xml]
for comparing text output, only --diff option is supported
3. args(optional): (used only with rpc)
- filter_xml: (can provide filtered output)
- other arguments
example:
1. - rpc: get-config
- args:
filter_xml: configuration/system/login
2. - rpc: get-interface-information
format: text
- args:
interface-name: em0
media: True
detail: True
4. item/iterate: (can have multiple iterate/item under one command/rpc)
item: if you want to use only first node in xpath
iterate: if want to keep iterating for all nodes in xpath
5. xpath: path from where u want to test output
6. tests: (this section specify test-cases, can have multiple test cases inside
one iterate/item)
test-operator <condition>
info <mssg>
err <mssg>
Eg:
tests:
- is-equal: //minimum-time, 60
info: "Test Succeeded!!, minimum-time now is equal to <{{post['//minimum-time']}}>"
err: "Test Failed!!!, minimum-time is not equal to 60, it is <{{post['//minimum-time']}}>"
1] Sample test file1 (test_is_equal.yml)
test_interfaces_terse:
- command: show interfaces terse lo*
- item:
id: ./name
xpath: //physical-interface[normalize-space(name) = "lo0"]
tests:
- is-equal: admin-status, down
info: "Test Succeeded !! admin-status is equal to post: <{{post['admin-status']}}> pre: <{{pre['admin-status']}}> with oper-status pre:<{{pre['oper-status']}}> post:<{{post['oper-status']}}>"
err: "Test Failed !! admin-status is not equal to down, it is post: <{{post['admin-status']}}> pre:<{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
2] Sample test file2 (multiple iterators and tests for single command)
tests_include:
- test_multiple_tests
- test_multiple_iter
test_multiple_tests:
- command: show interfaces terse lo*
- item:
id: ./name
xpath: //physical-interface[normalize-space(name) = "lo0"]
tests:
- is-equal: admin-status, down
info: "Test Succeeded !! admin-status is equal to <{{post['admin-status']}}> <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
err: "Test Failed !! admin-status is not equal to down, it is <{{post['admin-status']}}> <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
- is-in: oper-status, downoo, up
info: "Test Succeeded!! Physical operational status is-in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}>"
err: "Test Failed!!! Physical operational status is not in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}> "
test_multiple_iter:
- command: show interfaces terse lo*
- item:
id: ./name
xpath: //physical-interface[normalize-space(name) = "lo0"]
tests:
- is-equal: admin-status, down
info: "Test Succeeded !! admin-status is equal to <{{post['admin-status']}}> <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
err: "Test Failed !! admin-status is not equal to down, it is <{{post['admin-status']}}> <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
- iterate:
xpath: //physical-interface[normalize-space(name) = "lo0"]
tests:
- is-in: oper-status, down, up
info: "Test Succeeded!! Physical operational status is-in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}>"
err: "Test Failed!!! Physical operational status is not in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}> "
3] Sample test file3 (test_bgp_neighbor.yml)
tests_include:
- test_command_bgp
- test_rpc_bgp
test_command_bgp:
- command: show bgp neighbor
- iterate:
xpath: '/bgp-information/bgp-peer'
tests:
- is-equal: peer-address, 10.209.19.203 # element in which test is performed
err: "Test Failed!! peer-address got changed, it is now <{{post['peer-address']}}>"
info: "Test succeeded!! peer-address is equal to <{{post['peer-address']}}>"
- in-range: peer-as, 100,900 # element in which test is performed
err: "Test Failed!! peer-as is not in range of 100-200, it is: <{{post['peer-as']}}>"
info: "Test succeeded!! peer-as is in range of 100-200, it is now <{{post['peer-as']}}>"
- iterate:
xpath: '//bgp-information/bgp-peer/bgp-option-information'
tests:
- is-gt: holdtime, 10 # element in which test is performed
err: "Test Failed!! holdtime is not greater than 10, it is: <{{post['holdtime']}}>"
info: "Test succeeded!! holdtime is greater than 10, it is: <{{post['holdtime']}}>"
- is-lt: preference, 200 # element in which test is performed
err: "Test Failed!! preference is not less than 10, <{{post['preference']}}>"
info: "Test succeeded!! preference is less than 10, <{{post['preference']}}>"
test_rpc_bgp:
- rpc: get-bgp-neighbor-information
- iterate:
xpath: '//bgp-information/bgp-peer'
tests:
- not-equal: last-state,Idle # element in which test is performed
err: "Test Failed!! last state is <{{post['last-state']}}>"
info: "Test succeeded!! last state is not equal to idle, it is: <{{post['last-state']}}>"
- all-same: flap-count
err: "Test Failed!!! flap count are not all same!!, it is <{{post['flap-count']}}> "
info: "Test Succeeded!! flap count are all same, it is now <{{post['flap-count']}}>!!!"
- is-equal: flap-count, 0
err: "Test Failed!!! flap count is not equal to 0, it is: <{{post['flap-count']}}> "
info: "Test Succeeded!! flap count is equal to <{{post['flap-count']}}> !!"
For comparing current snapshot with pre-defined criteria (requires one snapshot):
1] Execute Tests Over Elements with Numeric or String Values
1. all-same: Check if all content values for the specified element are the same. It can also be used to compare all
content values against another specified element.
- all-same: flap-count
checks if all values of node <flap-count> in given Xpath is same or not.
2. is-equal: Check if the value (integer or string) of the specified element matches a given value.
- is-equal: admin-status, down
check if value of node <admin-status> in given Xpath is down or not.
3. not-equal: Check if the value (integer or string) of the specified element does not match a given value.
- not-equal: oper-status, down
check that value of node <oper-status> should not be equal to down.
4. exists: verifies the existence of an XML element in the snapshot.
- exists: no-active-alarm
checks if node </no-active-alarm> is present or not
5. not-exists: verifies xml element should not be present in snapshot
-not-exists: no-active-alarm
checks </no-active-alarm > node should not be present in snapshot.
6. contains: determines if an XML element string value contains the provided test-string value.
- contains: //package-information/name[1], jbase
checks if jbase is present in given node or not.
2] Execute Tests Over Elements with Numeric Values
1. is-gt: Check if the value of a specified element is greater than a given numeric value.
- is-gt: cpu-total, 2
checks value of <cpu-total> should be greater than 2
2. is-lt: Check if the value of a specified element is lesser than a given numeric value.
- is-lt temperature, 55
checks value of <temperature> is 55 or not.
3. in-range: Check if the value of a specified element is in the given numeric range.
- in-range memory-buffer-utilization, 20, 70
check if value of <memory-buffer-utilization> is in range of 20, 70
4. not-range: Check if the value of a specified element is outside of a given numeric range.
- not-range: memory-heap-utilization, 5 , 40
checks if value of <memory-heap-utilization> is not in range of 5 to 40
3] Execute Tests Over Elements with String Values:
1.contains: determines if an XML element string value contains the provided test-string value.
- contains: //package-information/name[1], jbase
checks if jbase is present in given node or not.
2. is-in: Check if the specified element string value is included in a given list of strings.
- is-in: oper-status, down, up
check if value of <oper-status> in is list (down, up)
3. not-in: Check if the specified element string value is NOT included in a given list of strings.
- not-in :oper-status, down, up
check if value of <oper-status> in not in list (down, up)
Compare Elements or Element Values in Two Snapshots:
1. no-diff: Compare data elements present in both snapshots, and verify if their value is the same.
- no-diff: mastership-state
checks that value of <mastership-state> in two snapshots is not different.
2. list-not-less: Check if the item is present in the first snapshot but not present in the second snapshot.
- list-not-less: interface-name
checks that if <interface-name> is present in first snapshot, then it should also present in
second snapshot.
If nothing is specified, then it will compare all nodes
3. list-not-more: Check if the item is present in the second snapshot but not present in the first snapshot.
-list-not-more address-family/address-family-name
checks that if node is present in second snapshot then it should be present in first snapshot also.
If nothing is specified, then it will compare all nodes.
4. delta: Compare the change in value of an element to a delta.
delta can be specified as:
1. an absolute percentage
2. positive, negative percentage
3. an absolute fixed value
4. a positive negative fixed value
- delta: memory-heap-utilization, 15%
Value of <memory-heap-utilization> should not differ by more than 15% between pre and post snapshots.