Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dvs] Add new common issues and TOC to DVS README #1405

Merged
merged 1 commit into from
Aug 21, 2020
Merged

[dvs] Add new common issues and TOC to DVS README #1405

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 54 additions & 20 deletions tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,31 @@ The DVS tests work by publishing configuration updates to redis (typically Confi

SWSS, Redis, and all the other required components run inside a virtual switch Docker container, meaning that these test cases can be run on any Linux machine - no special hardware required!

## Contents
- [Setting up your test environment](#setting-up-your-test-environment)
- [Running the tests](#running-the-tests)
- [Setting up a persistent testbed](#setting-up-a-persistent-testbed)
- [Other useful test parameters](#other-useful-test-parameters)
- [Known Issues/FAQs](#known-issues)

## Setting up your test environment
1. [Install Docker CE](https://docs.docker.com/install/linux/docker-ce/ubuntu/). Be sure to follow the [post-install instructions](https://docs.docker.com/install/linux/linux-postinstall/) so that you don't need sudo privileges to run docker commands.
2. Install the external dependencies needed to run the tests:
1. In order to set up your test environment you will need:

- A machine running Ubuntu 18.04 or 20.04
- A `generic` Linux kernel to install `team`
- `python3`

You can check these dependencies with the following:

```
cat /etc/os-release | grep ".*18.04\|20.04.*"
uname -r | grep generic
python3 --version
```

2. [Install Docker CE](https://docs.docker.com/install/linux/docker-ce/ubuntu/). Be sure to follow the [post-install instructions](https://docs.docker.com/install/linux/linux-postinstall/) so that you don't need sudo privileges to run docker commands.

3. Install the external dependencies needed to run the tests.

```
sudo modprobe team
Expand All @@ -26,19 +48,20 @@ SWSS, Redis, and all the other required components run inside a virtual switch D
```
sudo apt install libhiredis0.14
```
3. Install `python3-swsscommon_1.0.0_amd64.deb`. You will need to install all the dependencies as well in the following order:

4. Install `swsscommon`.

```
sudo dpkg -i libswsscommon_1.0.0_amd64.deb
sudo dpkg -i python3-swsscommon_1.0.0_amd64.deb
sudo dpkg -i libswsscommon_1.0.0_amd64.deb python3-swsscommon_1.0.0_amd64.deb
```

You can get these two packages by:
- [Building it from scratch](https://github.com/Azure/sonic-swss-common)
- Downloading the latest build from Jenkins:
- [Ubuntu 18.04](https://sonic-jenkins.westus2.cloudapp.azure.com/job/common/job/sonic-swss-common-build-ubuntu/lastSuccessfulBuild/artifact/target/)
- [Ubuntu 20.04](https://sonic-jenkins.westus2.cloudapp.azure.com/job/common/job/sonic-swss-common-build-ubuntu-20_04/lastSuccessfulBuild/artifact/target/)
4. Load the `docker-sonic-vs.gz` file into docker. You can get the image by:

5. Load the `docker-sonic-vs.gz` file into docker. You can get the image by:
- [Building it from scratch](https://github.com/Azure/sonic-buildimage)
- [Downloading the latest build from Jenkins](https://sonic-jenkins.westus2.cloudapp.azure.com/job/vs/job/buildimage-vs-all/lastSuccessfulBuild/artifact/target/)

Expand All @@ -50,35 +73,37 @@ cd sonic-swss/tests
sudo pytest
```

## Setting up a persistent test environment
## Setting up a persistent testbed
For those developing new features for SWSS or the DVS framework, you might find it helpful to setup a persistent DVS container that you can inspect and make modifications to (e.g. using `dpkg -i` to install a new version of SWSS to test a new feature).

1. [Download `create_vnet.sh`](https://github.com/Azure/sonic-buildimage/blob/master/platform/vs/create_vnet.sh).
2. Setup a virtual server and network:

2. Setup a virtual server and network. **Note**: It is _highly_ recommended you include the `-n 32` option or you may run into problems running the tests later.

```
docker run --privileged -id --name sw debian bash
sudo ./create_vnet.sh -n 32 sw
```
3. Start the DVS container:

3. Start the DVS container.

```
docker run --privileged -v /var/run/redis-vs/sw:/var/run/redis --network container:sw -d --name vs docker-sonic-vs
```

4. You can specify your persistent DVS container when running the tests as follows:
4. You can specify your persistent DVS container when running the tests as follows.

```
sudo pytest --dvsname=vs
```
By default if number of ports in persistent DVS < 32 (needed by testbed) then test will be aborted. To overcome that --forcedvs option can be used.

By default if the DVS has <32 ports (needed by testbed), then test will be aborted. To overcome that the --forcedvs option can be used. This automatically adds the proper number of ports for the tests to run, and cleans up the extra ports after the test is complete.

```
sudo pytest --dvsname=vs --forcedvs
```


5. Additionally, if you need to simulate a specific hardware platform (e.g. Broadcom or Mellanox), you can add this environment variable when starting the DVS container:
5. Additionally, if you need to simulate a specific hardware platform (e.g. Broadcom or Mellanox), you can add this environment variable when starting the DVS container. Note that this is not a precise 1-to-1 model, and dataplane behavior is not simulated by the DVS.

```
-e "fake_platform=mellanox"
Expand Down Expand Up @@ -140,7 +165,7 @@ For those developing new features for SWSS or the DVS framework, you might find
```

## Known Issues
- You may encounter the test run being aborted before any cases are run:
- The test run might abort before any cases are run:
```
daall@baker:~/sonic-swss/tests$ sudo pytest test_acl.py
============================= test session starts ==============================
Expand Down Expand Up @@ -172,17 +197,26 @@ For those developing new features for SWSS or the DVS framework, you might find
dpkg -r libswsscommon python3-swsscommon
dpkg --purge libswsscommon python3-swsscommon
rm -rf /usr/lib/python3/dist-packages/swsscommon/
dpkg -i libswsscommon.deb python3-swsscommon.deb
dpkg -i libswsscommon_1.0.0_amd64.deb python3-swsscommon_1.0.0_amd64.deb
```

- The tests might report a syntax error:

```
ImportError while loading conftest '/sonic/src/sonic-swss/tests/conftest.py'.
File "/sonic/src/sonic-swss/tests/conftest.py", line 56
def __init__(self, db_id: str, connector: str):
```

- You may encounter the following error message:
This indicates that `pytest` is using `python2` rather than `python3`. You need to install and use `pytest` for `python3`.

- You may encounter the following error message:
```
ERROR: Error response from daemon: client is newer than server (client API version: x.xx, server API version: x.xx)
```

You can mitigate this by upgrading to a newer version of Docker CE or editing the `DEFAULT_DOCKER_API_VERSION` in `/usr/local/lib/python3/dist-packages/docker/constants.py`, or by upgrading to a newer version of Docker CE. See [relevant GitHub discussion](https://github.com/drone/drone/issues/2048).

- Currently when pytest are run using --force-flaky and if the last test case fails pytest tear-down the module before retrying the failed test case and invoke module
setup again to run fail test case. This is know issue of pytest w.r.t flaky as tracked here (https://github.com/box/flaky/issues/128) and
(https://github.com/pytest-dev/pytest-rerunfailures/issues/51). Because of this issue all the logs are lost till last test case run as modules is teardown and setup again.
To avoid this as workaround a dummy always-pass test case is added in all modules/test files.
- Currently when pytest is run using `--force-flaky` and the last test case fails, then pytest tears down the module before retrying the failed test case and invokes the module setup again to run the failed test case. This is a known issue with [flaky](https://github.com/box/flaky/issues/128) and [pytest](https://github.com/pytest-dev/pytest-rerunfailures/issues/51).

Because of this issue, all the logs are lost except for the last test case as modules are torn down and set up again. The workaround for this is to include a dummy test case that always passes at the end of all test files/modules.