Report Structure

FD.io CSIT-2101.1 report contains system performance and functional testing data of VPP-21.01.1 release. PDF version of this report is available for download.

CSIT-2101.1 report is structured as follows:

  1. INTRODUCTION: General introduction to FD.io CSIT-2101.1.

    • Introduction: This section.

    • Test Scenarios Overview: A brief overview of test scenarios covered in this report.

    • Physical Testbeds: Description of physical testbeds.

    • Test Methodology: Performance benchmarking and functional test methodologies.

  2. VPP PERFORMANCE: VPP performance tests executed in physical FD.io testbeds.

    • Overview: Tested logical topologies, test coverage and naming specifics.

    • Release Notes: Changes in CSIT-2101.1, added tests, environment or methodology changes, known issues.

    • Packet Throughput: NDR, PDR throughput graphs based on results from repeated same test job executions to verify repeatibility of measurements.

    • Speedup Multi-Core: NDR, PDR throughput multi-core speedup graphs based on results from test job executions.

    • Packet Latency: Latency graphs based on results from test job executions.

    • Soak Tests: Long duration soak tests are executed using PLRsearch algorithm.

    • NFV Service Density: Network Function Virtualization (NFV) service density tests focus on measuring total per server throughput at varied NFV service “packing” densities with vswitch providing host dataplane.

    • Comparisons: Performance comparisons between VPP releases and between different testbed types.

    • Throughput Trending: References to continuous VPP performance trending.

    • Test Environment: Performance test environment configuration.

    • Documentation: Pointers to CSIT source code documentation for VPP performance tests.

  3. DPDK PERFORMANCE: DPDK performance tests executed in physical FD.io testbeds.

    • Overview: Tested logical topologies, test coverage.

    • Release Notes: Changes in CSIT-2101.1, known issues.

    • Packet Throughput: NDR, PDR throughput graphs based on results from repeated same test job executions to verify repeatibility of measurements.

    • Packet Latency: Latency graphs based on results from test job executions.

    • Comparisons: Performance comparisons between DPDK releases and between different testbed types.

    • Throughput Trending: References to regular DPDK performance trending.

    • Test Environment: Performance test environment configuration.

    • Documentation: Pointers to CSIT source code documentation for DPDK performance tests.

  4. VPP DEVICE: VPP functional tests executed in physical FD.io testbeds using containers.

    • Overview: Tested virtual topologies, test coverage and naming specifics;

    • Release Notes: Changes in CSIT-2101.1, added tests, environment or methodology changes, known issues.

    • Integration Tests: Functional test environment configuration.

    • Documentation: Pointers to CSIT source code documentation for VPP functional tests.

  5. DETAILED RESULTS: Detailed result tables auto-generated from CSIT test job executions using RF (Robot Framework) output files as sources.

    • VPP Performance NDR/PDR: VPP NDR/PDR throughput and latency.

    • VPP Performance MRR: VPP MRR throughput.

    • DPDK Performance: DPDK Testpmd and L3fwd NDR/PDR throughput and latency.

  6. TEST CONFIGURATION: VPP DUT configuration data based on VPP API Test (VAT) Commands History auto-generated from CSIT test job executions using RF output files as sources.

    • VPP Performance NDR/PDR: Configuration data.

    • VPP Performance MRR: Configuration data.

  7. TEST OPERATIONAL DATA: VPP DUT operational data auto-generated from CSIT test job executions using RFoutput files as sources.

    • VPP Performance NDR/PDR: VPP show run outputs under test load.

  8. CSIT FRAMEWORK DOCUMENTATION: Description of the overall FD.io CSIT framework.

    • Design: Framework modular design hierarchy.

    • Test naming: Test naming convention.

    • Presentation and Analytics Layer: Description of PAL CSIT analytics module.

    • CSIT RF Tags Descriptions: CSIT RF Tags used for test suite and test case grouping and selection.