Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic Port Cfg - VS test #12

Closed
wants to merge 4 commits into from
Closed
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions tests/test_port_add_remove.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
# the port to be removed and add
PORT = "Ethernet0"

@pytest.mark.usefixtures('dvs_port_manager')
class TestPortAddRemove(object):

def test_remove_port_with_buffer_cfg(self, dvs, testlog):
Expand Down Expand Up @@ -54,3 +55,110 @@ def test_remove_port_with_buffer_cfg(self, dvs, testlog):

# verify that the port was removed properly since all buffer configuration was removed also
assert len(num) == num_of_ports - 1

config_db.create_entry("PORT", PORT, port_info)

def test_add_remove_a_port(self, dvs, testlog):
config_db = dvs.get_config_db()
asic_db = dvs.get_asic_db()

# get port info
port_info = config_db.get_entry("PORT", PORT)

# remove buffer pg cfg for the port
pgs = config_db.get_keys('BUFFER_PG')
for key in pgs:
if PORT in key:
config_db.delete_entry('BUFFER_PG', key)

# remove buffer queue cfg for the port
pgs = config_db.get_keys('BUFFER_QUEUE')
for key in pgs:
if PORT in key:
config_db.delete_entry('BUFFER_QUEUE', key)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we reuse part of the DVSPort remove_port ?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to remove the buffer cfg dependencies without removing the port, the removal of the port will be inside the loop

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you create a function in DVSPort - remove_port_dependencies that will be used here and in remove_port function?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is that in case new dependencies added we need to modify test cases that do that instaed of modifying one function

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are both using the same remove function:
self.dvs_port.remove_port(key) - for removing port dependencies and the port itself

between remove and add port I decided to add different sleep timeouts so a function that will remove and add ports needs to use the I variable inside and it will make the function a bit complex

So I prefer to leave it as is


# get the number of ports before removal
num_of_ports = len(asic_db.get_keys("ASIC_STATE:SAI_OBJECT_TYPE_PORT"))

for i in range(10):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is 10?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 times for adding and removing ports.
I wanted the test to remove and add ports in a loop, since i dont wat the test execution runtime will take too much, 10 times is enough.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to create a named constant value

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will add a constant value

# try to remove this port
config_db.delete_entry('PORT', PORT)
num = asic_db.wait_for_n_keys("ASIC_STATE:SAI_OBJECT_TYPE_PORT",
num_of_ports-1,
polling_config = PollingConfig(polling_interval = 1, timeout = 5.00, strict = True))

# verify that the port wasn't removed since we still have buffer cfg
assert len(num) == num_of_ports - 1

time.sleep((i+1)%3)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the logic behind (i+1)%3 ? Same comment for L104

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted the sleep time between removing and adding ports will be different in each iteration so I used this calculation

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid sleep ? Same comment for L104

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need sleep between removing and adding ports - it takes few seconds until all ports are removed/created
since we are using (i+1)%3 we have different timeout for each iteration.
different sleep timeout between add and remove ports can detect bugs, for example we found a bug on LLDP that was reproduced only on specific timeout values

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this test case designed to cover some real bug? VS does not have LLDP. How are you detecting those bugs? The only assert I see after sleep is the expectation on the number of SAI_OBJECT_TYPE_PORT objects. It will capture a situation when creation of the port fails in case there is specific sleep time interval after port deletion. Is it even a possible scenario? Can't we just assume that if we successfully deleted the port and the re-creation of the port succeded with 0 sec sleep in between it will work with any sleep interval? What is happening in the system during that sleep that can affect the test case result? The asic.db.wait_for_n_keys will sleep till the entry will appear - will it affect somehow your sleep interval logic?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test case is not covering a real bug, the lldp isue that I found was on real switch and it is just an example for race conditions that related to timeouts between delete and create.
I wanted to add several different timeouts between delete and create in order to cover such cases of race conditions that may occur between orchagent and syncd.
I guess that VS switch is not like the real switch and such cases of race conditions will probably be more relevant to real switch.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can tell from your comment is that these sleep's are redundant here as they are relevant for real switch and not for VS.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't say that, maybe I wasn't clear, I think that race condition can also be on VS switch (maybe on real switch it's more common), that's why the different sleep time in each iteration is relevant also on VS.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But how are you catching this race condition, where the test fails when you hit it?


config_db.create_entry("PORT", PORT, port_info)

num = asic_db.wait_for_n_keys("ASIC_STATE:SAI_OBJECT_TYPE_PORT",
num_of_ports)

# verify that the port wasn't removed since we still have buffer cfg
assert len(num) == num_of_ports

time.sleep(i%3)

def test_add_remove_all_the_ports(self, dvs, testlog):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test case duplicated the code in test_add_remove_a_port - can we have a single test case that can be parametrized with the number of ports and then parametrize it with 1 port and N ports where N is the total number of ports in the system?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you right, it's a kind of duplicate code, we can change it but I prefer to leave it as is, in different test cases, one of them is testing only one port and the second one is testing all the ports, I think it will be more readable.

config_db = dvs.get_config_db()
asic_db = dvs.get_asic_db()

# get the number of ports before removal
num_of_ports = len(asic_db.get_keys("ASIC_STATE:SAI_OBJECT_TYPE_PORT"))

# remove buffer pg cfg for the port
ports = config_db.get_keys('PORT')
ports_info = {}

for i in range(10):
# remove port
for key in ports:
# read port info and save it
ports_info[key] = config_db.get_entry("PORT", key)

# remove a port
self.dvs_port.remove_port(key)

# verify remove port
num = asic_db.wait_for_n_keys("ASIC_STATE:SAI_OBJECT_TYPE_PORT",
num_of_ports-len(ports))
assert len(num) == num_of_ports-len(ports)

# add port
time.sleep((i%3)+1)
for key in ports:
config_db.create_entry("PORT", key, ports_info[key])

# verify add port
num = asic_db.wait_for_n_keys("ASIC_STATE:SAI_OBJECT_TYPE_PORT",
num_of_ports)
assert len(num) == num_of_ports

time.sleep((i%2)+2)

# run ping
dvs.setup_db()
dvs.create_vlan("6")
dvs.create_vlan_member("6", "Ethernet64")
dvs.create_vlan_member("6", "Ethernet68")
dvs.set_interface_status("Vlan6", "up")
dvs.add_ip_address("Vlan6", "6.6.6.1/24")
dvs.set_interface_status("Ethernet68", "up")
dvs.set_interface_status("Ethernet64", "up")

dvs.servers[16].runcmd("ifconfig eth0 6.6.6.6/24 up")
dvs.servers[16].runcmd("ip route add default via 6.6.6.1")
dvs.servers[17].runcmd("ifconfig eth0 6.6.6.7/24 up")
dvs.servers[17].runcmd("ip route add default via 6.6.6.1")

time.sleep(2)

rc = dvs.servers[16].runcmd("ping -c 1 6.6.6.7")
assert rc == 0

rc = dvs.servers[17].runcmd("ping -c 1 6.6.6.6")
assert rc == 0