Packet Throughput

Throughput graphs are generated based on the results data obtained from the CSIT-2001 test jobs. In order to verify benchmark results repeatibility selected, CSIT performance tests are executed multiple times (target: 10 times) on each physical testbed type. Box-and-Whisker plots are used to display variations in measured throughput values.

Lists of tests selected for multiple execution and graphing are captured per testbed type in test_select_list_{testbed_type}.md files.

Graphs are split into sections as follows:

  1. Header 1: VPP packet path and lookup types

    • L2 Ethernet Switching: L2 bridge-doman, L2 cross-connect and L2 patch

    • IPv4 Routing: IPv4 routing with /32 prefixes

    • IPv6 Routing: IPv6 routing with /128 prefixes

    • SRv6 Routing: SRv6 with IPv6 routing

    • IPv4 Tunnels: IPv4 overlay tunnels

    • KVM VMs vhost-user: KVM VMs connected over virtio and vhost-user interfaces

    • LXC/DRC Container Memif: Linux containers and Docker containers connected over Memif interfaces

    • IPsec IPv4 Routing: IPsec encryption/decryption with IPv4 routing

    • Virtual Topology System: VXLAN configurations with L2 bridge-domains

  2. Header 2: testbeds and NIC models

    • section name format:

      • {testbed_type}-{nic_model}

    • testbed_type:

      • 2n-skx: 2-node Xeon Skylake

      • 3n-skx: 3-node Xeon Skylake

      • 2n-clx: 2-node Xeon Cascade Lake

      • 3n-hsw: 3-node Xeon Haswell

      • 3n-tsh: 3-node Arm TaiShan

      • 2n-dnv: 2-node Atom Denverton

      • 3n-dnv: 3-node Atom Denverton

    • nic_model:

      • xxv710: xxv710 2p25GE Intel (Fortville)

      • x710: x710 4p10GE Intel (Fortville)

      • xl710: xl710 2p40GE Intel (Fortville)

      • x520: x520 2p10GE Intel (Niantic)

      • x553: x553 2p10GE Intel (Niantic)

  3. Header 3: test group names

    • section name format:

      • {frame_size}-{worker_thread_core_cfg}-{vpp_functionality}-{vpp_lookup_type}-{baseline_scale}-{nic_driver}

    • frame_size:

      • 64b: 64 byte frames, smallest frame size for untagged IPv4 packets

      • 78b: 78 byte frames, smallest frame size for untagged IPv6 packets

      • 114b: VXLAN encapsulated L2 frames

      • imix: a sequence of (7x64B, 4x570, 1x1518) byte frames

    • worker_thread_core_cfg:

      • 1t1c: 1 worker thread on 1 core, hyper-threading not used

      • 2t1c: 2 worker threads on 1 core, hyper-threading used

    • vpp_functionality (optional):

      • features: including input-acl, output-acl, macip-iacl, nat44

      • srv6: srv6 encap/decap, proxy

      • link-bonding: L2 link aggregation with 1 or 2 bonded links

      • ipsec: IPsec encryption/decryption with different ciphers

      • vts: Virtual Topology System specific tests

    • vpp_lookup_type:

      • l2switching, ip4routing, ip6routing, ip4tunnel, vhost, memif

    • baseline_scale:

      • base: baseline tests with less than 10 forwarding entries

      • scale: scale tests with up to 2 million forwarding entries

      • base-scale: both baseline and scale tests grouped together

    • nic_driver:

      • avf: VPP native avf driver for Intel Fortville NICs

      • i40e: dpdk poll mode driver for Intel Fortville NICs

      • ixgbe: dpdk poll mode driver for Intel Niantic NICs

For each test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th percentile, Max) across collected data set. Outliers are plotted as individual points.

Additional information about graph data:

  1. Graph Title: describes tested packet path, testbed topology, processor model, NIC model, packet size, number of cores and threads used by data plane workers and indication of VPP DUT configuration.

  2. X-axis Labels: indices of individual test suites as listed in Graph Legend.

  3. Y-axis Labels: measured Packets Per Second [pps] throughput values.

  4. Graph Legend: lists X-axis indices with associated CSIT test suites executed to generate graphed test results.

  5. Hover Information: lists minimum, first quartile, median, third quartile, and maximum. If either type of outlier is present the whisker on the appropriate side is taken to 1.5×IQR from the quartile (the “inner fence”) rather than the max or min, and individual outlying data points are displayed as unfilled circles (for suspected outliers) or filled circles (for outliers). (The “outer fence” is 3×IQR from the quartile.)

Note

Test results have been generated by FD.io test executor vpp performance job 2n-skx, FD.io test executor vpp performance job 3n-skx, FD.io test executor vpp performance job 2n-clx, FD.io test executor vpp performance job 3n-hsw, FD.io test executor vpp performance job 3n-tsh, FD.io test executor vpp performance job 2n-dnv and FD.io test executor vpp performance job 3n-dnv with RF result files csit-vpp-perf-2001-*.zip archived here. Required per test case data set size is 10, but for VPP tests the actual size varies per test case and is <=10.