VPP Test Framework

Overview

The goal of the VPP Test Framework is to ease writing, running and debugging unit tests for the VPP. For this, python was chosen as a high level language allowing rapid development with scapy providing the necessary tool for creating and dissecting packets.

Anatomy of a test case

Python’s unittest is used as the base framework upon which the VPP test framework is built. A test suite in the VPP Test Framework consists of multiple classes derived from VppTestCase, which is itself derived from TestCase. The test class defines one or more test functions, which act as test cases.

Function flow when running a test case is:

1. setUpClass: This function is called once for each test class, allowing a one-time test setup to be executed. If this functions throws an exception, none of the test functions are executed.
2. setUp: The setUp function runs before each of the test functions. If this function throws an exception other than AssertionError or SkipTest, then this is considered an error, not a test failure.
3. test_<name>: This is the guts of the test case. It should execute the test scenario and use the various assert functions from the unittest framework to check necessary.
4. tearDown: The tearDown function is called after each test function with the purpose of doing partial cleanup.
5. tearDownClass: Method called once after running all of the test functions to perform the final cleanup.

Test temporary directory and VPP life cycle

Test separation is achieved by separating the test files and vpp instances. Each test creates a temporary directory and it’s name is used to create a shared memory prefix which is used to run a VPP instance. This way, there is no conflict between any other VPP instances running on the box and the test VPP. Any temporary files created by the test case are stored in this temporary test directory.

Virtual environment

Virtualenv is a python module which provides a means to create an environment containing the dependencies required by the VPP Test Framework, allowing a separation from any existing system-wide packages. VPP Test Framework‘s Makefile automatically creates a virtualenv inside build-root and installs the required packages in that environment. The environment is entered whenever executing a test via one of the make test targets.

Naming conventions

Most unit tests do some kind of packet manipulation - sending and receiving packets between VPP and virtual hosts connected to the VPP. Referring to the sides, addresses, etc. is always done as if looking from the VPP side, thus:

• local_ prefix is used for the VPP side. So e.g. local_ip4 address is the IPv4 address assigned to the VPP interface.
• remote_ prefix is used for the virtual host side. So e.g. remote_mac address is the MAC address assigned to the virtual host connected to the VPP.

Packet flow in the VPP Test Framework

Test framework -> VPP

VPP Test Framework doesn’t send any packets to VPP directly. Traffic is instead injected using packet-generator interfaces, represented by the VppPGInterface class. Packets are written into a temporary .pcap file, which is then read by the VPP and the packets are injected into the VPP world.

VPP -> test framework

Similarly, VPP doesn’t send any packets to VPP Test Framework directly. Instead, packet capture feature is used to capture and write traffic to a temporary .pcap file, which is then read and analyzed by the VPP Test Framework.

Test framework objects

The following objects provide VPP abstraction and provide a means to do common tasks easily in the test cases.

• VppInterface: abstract class representing generic VPP interface and contains some common functionality, which is then used by derived classes
• VppPGInterface: class representing VPP packet-generator interface. The interface is created/destroyed when the object is created/destroyed.
• VppSubInterface: VPP sub-interface abstract class, containing common functionality for e.g. VppDot1QSubint and VppDot1ADSubint classes

How VPP API/CLI is called

Vpp provides python bindings in a python module called vpp-papi, which the test framework installs in the virtual environment. A shim layer represented by the VppPapiProvider class is built on top of the vpp-papi, serving these purposes:

1. Automatic return value checks: After each API is called, the return value is checked against the expected return value (by default 0, but can be overridden) and an exception is raised if the check fails.
2. Automatic call of hooks:
   1. before_cli and before_api hooks are used for debug logging and stepping through the test
   2. after_cli and after_api hooks are used for monitoring the vpp process for crashes
3. Simplification of API calls: Many of the VPP APIs take a lot of parameters and by providing sane defaults for these, the API is much easier to use in the common case and the code is more readable. E.g. ip_add_del_route API takes ~25 parameters, of which in the common case, only 3 are needed.

Example: how to add a new test

In this example, we will describe how to add a new test case which tests VPP...

1. Add a new file called...
2. Add a setUpClass function containing...
3. Add the test code in test...
4. Run the test...

VPP Test Framework module documentation

• test
  • bfd module
  • framework module
  • hook module
  • log module
  • run_tests module
  • template_bd module
  • test_bfd module
  • test_classifier module
  • test_fib module
  • test_gre module
  • test_ip4 module
  • test_ip4_irb module
  • test_ip6 module
  • test_l2_fib module
  • test_l2bd module
  • test_l2bd_multi_instance module
  • test_l2xc module
  • test_l2xc_multi_instance module
  • test_lb module
  • test_mpls module
  • test_snat module
  • test_span module
  • test_vxlan module
  • util module
  • vpp_gre_interface module
  • vpp_interface module
  • vpp_ip_route module
  • vpp_lo_interface module
  • vpp_object module
  • vpp_papi_provider module
  • vpp_pg_interface module
  • vpp_sub_interface module

Indices and tables

• Index
• Module Index
• Search Page