diff options
| author |   pmikus <pmikus@cisco.com> | 2017-02-01 18:54:58 +0100 |
|---|---|---|
| committer | Peter Mikus <pmikus@cisco.com> | 2017-02-03 20:40:09 +0000 |
| commit | 35e5a327a66c05a3356d214dcfdece59ca725998 (patch) | |
| tree | a111bb1d5b9d40f358655187d5bc4a3b899b0d06 | |
| parent | 6baa0bc9921bc13b1adff113c20a4db766c3feba (diff) | |
CSIT 1701 report files and script AD1
CSIT 1701 report files and script addendum 1
Edits to correct and align all Overview sub-sections.
Updates in Performance CSIT Release Notes - added more NDR and PDR performance changes.
Change-Id: I52b6ee89e9c536fb4ab9d30dc27cca8dbdd88a20
Signed-off-by: pmikus <pmikus@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
Signed-off-by: pmikus <pmikus@cisco.com>
26 files changed, 2504 insertions, 2209 deletions
diff --git a/docs/report/honeycomb_functional_results_virl/honeycomb_functional_results_virl.rst b/docs/report/honeycomb_functional_results_virl/honeycomb_functional_results_virl.rst index bb19405c00..b298dca43c 100644 --- a/docs/report/honeycomb_functional_results_virl/honeycomb_functional_results_virl.rst +++ b/docs/report/honeycomb_functional_results_virl/honeycomb_functional_results_virl.rst @@ -1,373 +1,378 @@ -
-.. |br| raw:: html
-
- <br />
-
-
-mgmt-cfg-acl-apihc-apivat-func
-``````````````````````````````
-
-**Honeycomb access control lists test suite.**
-
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+================================================================+===============================================================================+========+
-| TC01: Honeycomb can create ACL classify table | Check if Honeycomb API can create an ACL table. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb can remove ACL table | Check if Honeycomb API can delete an ACL table. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb manages more than one ACL table | Check if Honeycomb API can create another ACL table. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb can add ACL session to table | Check if Honeycomb API can add an ACL session to a table. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb can remove ACL session | Check if Honeycomb API can remove an ACL session. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb manages more than one ACL session on one table | Check if Honeycomb API can add another ACL session to a table. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb enables ACL on interface | Check if Honeycomb API can enable ACL on an interface. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC08: Honeycomb disables ACL on interface | Check if Honeycomb API can disable ACL on an interface. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-| TC09: Honeycomb can remove one out of multiple ACL tables | Check if Honeycomb API can delete an ACL table if more than one table exists. | PASS |
-+----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-int-apihcnc-func
-`````````````````````````
-
-**Netconf test suite. Contains test cases that need to bypass REST API.**
-
-+--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+==================================================+===============================================================================================+========+
-| TC01: Honeycomb can create and delete interfaces | Repeatedly create and delete an interface through Netconf and check the reply for any errors. | PASS |
-+--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+
-| TC02: Transaction revert test case 1 | Configure two conflicting VxLAN tunnels, then verify that neither tunnel exists. | PASS |
-+--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+
-| TC03: Transaction revert test case 2 | Configure two conflicting TAP interfaces, then verify that neither interface exists. | PASS |
-+--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-int-subint-apihc-apivat-func
-`````````````````````````````````````
-
-**Honeycomb sub-interface management test suite.**
-
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+=================================================================================+===========================================================================================================================+========+
-| TC01: Honycomb creates sub-interface | Check if Honeycomb creates a sub-interface. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb sets interface and sub-interface up | Honeycomb changes the state of interface and of its sub-interface to up. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb sets sub-interface down while its super-interface is up | Honeycomb sets the sub-interface down while its super-interface is up. It must be possible. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb sets interface and sub-interface down | Honeycomb changes the state of interface down and then changes the state of its sub-interface down, in this order. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb fails to set sub-interface up while its super-interface is down | Honeycomb tries to set the sub-interface up while its super-interface is down. It must not be possible. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb fails to delete sub-interface | Check if Honeycomb can delete an existing sub-interface. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb adds sub-interface to new bridge domain | Check if Honeycomb adds a sub-interface to bridge domain. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC08: Honeycomb enables tag-rewrite pop 1 | Check if Honeycomb enables tag-rewrite and sets its parameters correctly. Case: pop 1. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC09: Honeycomb enables tag-rewrite push | Check if Honeycomb enables tag-rewrite and sets its parameters correctly. Case: push. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC10: Honeycomb enables tag-rewrite translate 1-2 | Check if Honeycomb enables tag-rewrite and sets its parameters correctly. Case: translate 1-2. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC11: Honeycomb disables tag-rewrite | Check if Honeycomb disables the tag-rewrite. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC12: Honeycomb enables tag-rewrite pop 1 again | Check if Honeycomb can enable tag-rewrite again, once it was disabled by Honeycomb. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC13: Honeycomb modifies the tag-rewrite | Honeycomb sets the tag-rewrite: 1. pop 1, then 2. push, then 3. translate 1 - 2 Then Honeycomb disables the tag-rewrite. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC14: Honeycomb fails to set wrong vlan-type in tag-rewrite | Check that Honeycomb does not accept wrong values of vlan-type in tag-rewrite. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC15: Honeycomb configures sub-interface ipv4 address | Check if Honeycomb can configure an ipv4 address on the sub-interface. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC16: Honeycomb removes sub-interface ipv4 address | Check if Honeycomb can remove configured ipv4 addresses from the sub-interface. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-| TC17: Honeycomb modifies existing sub-interface ipv4 address | Check if Honeycomb can modify an ipv4 address already configured on the sub-interface. | PASS |
-+---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-intip4-intip6-apihc-apivat-func
-````````````````````````````````````````
-
-**Honeycomb interface management test suite.**
-
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+==============================================================+=============================================================================================+========+
-| TC01: Honeycomb configures and reads interface state | Check if Honeycomb API can modify the admin state of VPP interfaces. | PASS |
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb modifies interface IPv4 address with netmask | Check if Honeycomb API can configure interfaces for ipv4 with address and netmask provided. | PASS |
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb removes IPv4 address from interface | Check if Honeycomb API can remove configured ipv4 addresses from interface. | PASS |
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb modifies interface IPv4 address with prefix | Check if Honeycomb API can configure interfaces for ipv4 with address and prefix provided. | PASS |
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb modifies IPv4 neighbor table | Check if Honeycomb API can add and remove ARP entries. | PASS |
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb modifies interface configuration - IPv6 | Check if Honeycomb API can configure interfaces for ipv6. | PASS |
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb modifies interface configuration - MTU | Check if Honeycomb API can configure interface MTU value. | PASS |
-+--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-inttap-apihc-apivat-func
-`````````````````````````````````
-
-**Honeycomb TAP management test suite.**
-
-+---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+===============================================================+=======================================================================================+========+
-| TC01: Honeycomb configures TAP interface | Check if Honeycomb API can configure a TAP interface. | PASS |
-+---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb modifies existing TAP interface configuration | Check if Honeycomb API can re-configure and existing TAP interface with new settings. | PASS |
-+---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb removes TAP interface | Check if Honeycomb API can remove TAP interface. | PASS |
-+---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-intvhost-apihc-apivat-func
-```````````````````````````````````
-
-**Honeycomb vhost-user interface management test suite.**
-
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+=================================================================================+==============================================================================================================+========+
-| TC01: Honeycomb creates vhost-user interface - server | Check if Honeycomb creates a vhost-user interface, role: server. | FAIL |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb modifies vhost-user interface - server | Check if Honeycomb can modify properties of existing vhost-user interface, role: server. | FAIL |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb deletes vhost-user interface - server | Check if Honeycomb can delete an existing vhost-user interface, role: server. | FAIL |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb creates vhost-user interface - client | Check if Honeycomb creates a vhost-user interface, role: client. | PASS |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb modifies vhost-user interface - client | Check if Honeycomb can modify properties of existing vhost-user interface, role: client. | FAIL |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb deletes vhost-user interface - client | Check if Honeycomb can delete an existing vhost-user interface, role: client. | PASS |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb does not set vhost-user configuration on another interface type | Check if Honeycomb refuses to set vhost-user configuration for interface which is not v3po:vhost-user type. | PASS |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-| TC08: Honeycomb does not set invalid vhost-user configuration | Check if Honeycomb refuses to set invalid parameters to vhost-user interface. | PASS |
-+---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-l2bd-apihc-apivat-func
-```````````````````````````````
-
-**Honeycomb bridge domain management test suite.**
-
-+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+========================================================================+===================================================================================================================+========+
-| TC01: Honeycomb sets up l2 bridge domain | Check if Honeycomb can create bridge domains on VPP node. | PASS |
-+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb manages multiple bridge domains on node | Check if Honeycomb can manage multiple bridge domains on a single node. | PASS |
-+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb removes bridge domains | Check if Honeycomb can remove bridge domains from a VPP node. | PASS |
-+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb assigns interfaces to bridge domain | Check if Honeycomb can assign VPP interfaces to an existing bridge domain. | PASS |
-+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb cannot remove bridge domain with an interface assigned | Check if Honeycomb can remove a bridge domain that has an interface assigned to it. Expect to fail with code 500. | PASS |
-+------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-l2fib-apihc-apivat-func
-````````````````````````````````
-
-**Honeycomb L2 FIB management test suite.**
-
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+=========================================================+=================================================================================================================================+========+
-| TC01: Honeycomb adds L2 FIB entry (forward) | Honeycomb creates a bridge domain and assignes an interface to it. Then adds an L2 FIB entry (forward) to the bridge domain. | PASS |
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb adds L2 FIB entry (static, forward) | Honeycomb adds an L2 FIB entry (static, forward) to the bridge domain. | PASS |
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb adds L2 FIB entry (static, filter) | Honeycomb adds an L2 FIB entry (static, filter) to the bridge domain. | PASS |
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb adds and removes L2 FIB entry (forward) | Honeycomb adds an L2 FIB entry (forward) to the bridge domain and then Honeycomb removes it from the bridge domain. | PASS |
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb adds more than one L2 FIB entry | Honeycomb adds three L2 FIB entries to the bridge domain. | PASS |
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb fails to set wrong L2 FIB entry | Honeycomb tries to add an L2 FIB entry with wrong parameters to the bridge domain. It must fail. | PASS |
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb fails to modify existing L2 FIB entry | Honeycomb tries to modify an existing L2 FIB entry. It must fail. | PASS |
-+---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-lisp-apihc-apivat-func
-```````````````````````````````
-
-**Honeycomb Lisp test suite.**
-
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+==================================================================+====================================================================================================+========+
-| TC01: Honeycomb enables Lisp feature | Check if Honeycomb can enable the Lisp feature. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb adds locator set and locator | Check if Honeycomb can configure a locator set. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb configures Lisp - remote mapping - Bridge Domain | Check if Honeycomb can configure a remote Lisp mapping with a bridge domain. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb can remove Lisp mapping | Check if Honeycomb can remove a configured Lisp mapping. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb configures Lisp - remote mapping - VRF | Check if Honeycomb can configure a remote Lisp mapping with VRF. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb configures Lisp - local mapping - Bridge Domain | Check if Honeycomb can configure a local Lisp mapping with a bridge domain. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb configures Lisp - local mapping - VRF | Check if Honeycomb can configure a local Lisp mapping with VRF. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC08: Honeycomb configures Lisp mapping with adjacency | Check if Honeycomb can configure local and remote Lisp mappings with VRF, and configure adjacency. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC09: Honeycomb configures Lisp map resolver | Check if Honeycomb can configure a Lisp map resolver. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC10: Honeycomb enabled Lisp PITR feature | Check if Honeycomb can configure the Lisp PITR feature. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-| TC11: Honeycomb can remove configuration of Lisp features | Check if Honeycomb can disable all Lisp features. | PASS |
-+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-nsh-apihc-apivat-func
-``````````````````````````````
-
-**Honeycomb NSH test suite.**
-
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+=====================================================================+===========================================================================+========+
-| TC01: Honeycomb can configure NSH entry | Check if Honeycomb can configure an NSH entry. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| TC02: Honeycomb can remove NSH entry | Check if Honeycomb can remove an existing NSH entry. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| TC03: Honeycomb can configure new NSH entry | Check if Honeycomb can configure an NSH antry after one has been deleted. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| TC04: Honeycomb can configure multiple NSH entries at the same time | Check if Honeycomb can configure an NSH entry when one already exists. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| TC05: Honeycomb can configure NSH map | Check if Honeycomb can configure an NSH map. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| TC06: Honeycomb can remove NSH map | Check if Honeycomb can remove an existing NSH map. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| TC07: Honeycomb can modify existing NSH map | Check if Honeycomb can configure an NSH map after one has been deleted. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-| TC08: Honeycomb can configure multiple NSH maps at the same time | Check if Honeycomb can configure and NSH map when one already exists. | PASS |
-+---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+
-
-mgmt-cfg-pbb-apihc-apivat-func
-``````````````````````````````
-
-**Honeycomb provider backbone bridge test suite.**
-
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+==================================================================================+================================================================================================================================================+========+
-| TC01: Honeycomb sets PBB sub-interface | Honeycomb creates a new PBB sub-interface. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb modifies existing PBB sub-interface | Honeycomb modifies an existing PBB sub-interface. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb deletes existing PBB sub-interface | Honeycomb deletes an existing PBB sub-interface. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb fails to set wrong destination-address for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter destination-address, type yang:mac-address. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb fails to set wrong source-address for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter source-address, type yang:mac-address. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb fails to set wrong b-vlan-tag-vlan-id for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter b-vlan-tag-vlan-id, type uint16, 12 bit range, range 1..4095. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb fails to set wrong i-tag-isid for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter i-tag-isid, type uint32, 24 bit range, range 1..16777215. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-| TC08: Honeycomb fails to create new PBB sub-interface without vlan tag | Honeycomb fails to create a new PBB sub-interface without parameter b-vlan-tag-vlan-id. | PASS |
-+----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-snat44-apihc-apivat-func
-`````````````````````````````````
-
-**Honeycomb NAT test suite.**
-
-+-----------------------------------------------------+-----------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+=====================================================+=================================================================+========+
-| TC01: Honeycomb configures NAT entry | Honeycomb configures a static NAT entry. | PASS |
-+-----------------------------------------------------+-----------------------------------------------------------------+--------+
-| TC02: Honeycomb removes NAT entry | Honeycomb removes a configured static NAT entry. | PASS |
-+-----------------------------------------------------+-----------------------------------------------------------------+--------+
-| TC03: Honeycomb configures multiple NAT entries | Honeycomb configures two static NAT entries. | PASS |
-+-----------------------------------------------------+-----------------------------------------------------------------+--------+
-| TC04: Honeycomb enables NAT on interface - inbound | Honeycomb configures NAT on an interface in inbound direction. | PASS |
-+-----------------------------------------------------+-----------------------------------------------------------------+--------+
-| TC05: Honeycomb removes NAT interface configuration | Honeycomb removes NAT configuration from an interface. | PASS |
-+-----------------------------------------------------+-----------------------------------------------------------------+--------+
-| TC06: Honeycomb enables NAT on interface - outbound | Honeycomb configures NAT on an interface in outbound direction. | PASS |
-+-----------------------------------------------------+-----------------------------------------------------------------+--------+
-
-mgmt-cfg-vxlan-apihc-apivat-func
-````````````````````````````````
-
-**Honeycomb VxLAN management test suite.**
-
-+----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+============================================================================+======================================================================================================+========+
-| TC01: Honeycomb configures VxLAN tunnel | Check if Honeycomb API can configure VxLAN settings. | PASS |
-+----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb disables VxLAN tunnel | Check if Honeycomb API can reset VxLAN configuration. | PASS |
-+----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb can configure VXLAN tunnel after one has been disabled | Check if Honeycomb API can configure VxLAN settings again after previous settings have been removed. | PASS |
-+----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb does not set VxLAN configuration on another interface type | Check if Honeycomb API prevents setting VxLAN on incorrect interface. | PASS |
-+----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb does not set invalid VxLAN configuration | Check if Honeycomb API prevents setting incorrect VxLAN settings. | PASS |
-+----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb configures VxLAN tunnel with ipv6 | Check if Honeycomb API can configure VxLAN with ipv6 settings. | PASS |
-+----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+
-
-mgmt-cfg-vxlangpe-apihc-apivat-func
-```````````````````````````````````
-
-**Honeycomb VxLAN-GPE management test suite.**
-
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+=============================================================================+=================================================================================================+========+
-| TC01: Honeycomb creates VxLAN GPE tunnel | Check if Honeycomb API can configure a VxLAN GPE tunnel. | PASS |
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb removes VxLAN GPE tunnel | Check if Honeycomb API can remove VxLAN GPE tunnel. | PASS |
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb sets wrong interface type while creating VxLAN GPE tunnel | Check if Honeycomb refuses to create a VxLAN GPE tunnel with a wrong interface type set. | PASS |
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb sets wrong protocol while creating VxLAN GPE tunnel | Check if Honeycomb refuses to create a VxLAN GPE tunnel with a wrong next-protocol set. | PASS |
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-| TC05: Honeycomb sets VxLAN GPE tunnel on existing interface with wrong type | Check if Honeycomb refuses to create a VxLAN GPE tunnel on existing interface with wrong type. | PASS |
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-| TC06: Honeycomb creates VxLAN GPE tunnel with ipv6 | Check if Honeycomb API can configure a VxLAN GPE tunnel with IPv6 addresses. | PASS |
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-| TC07: Honeycomb creates a second VxLAN GPE tunnel with ipv6 | Check if Honeycomb API can configure another VxLAN GPE tunnel with IPv6 addresses. | PASS |
-+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+
-
-mgmt-notif-apihcnc-func
-```````````````````````
-
-**Honeycomb notifications test suite.**
-
-+--------------------------------------------------------------+--------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+==============================================================+==================================================================================================+========+
-| TC01: Honeycomb sends notification on interface state change | Check if Honeycomb sends a state-changed notification when the state of an interface is changed. | PASS |
-+--------------------------------------------------------------+--------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb sends notification on interface deletion | Check if Honeycomb sends an interface-deleted notification when an interface is deleted. | PASS |
-+--------------------------------------------------------------+--------------------------------------------------------------------------------------------------+--------+
-
-mgmt-statepersist-apihc-func
-````````````````````````````
-
-**Honeycomb configuration persistence test suite.**
-
-+----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+
-| Name | Documentation | Status |
-+==================================================================================+=====================================================================================================+========+
-| TC01: Honeycomb persists configuration through restart of both Honeycomb and VPP | Checks if Honeycomb maintains configuration after both Honeycomb and VPP are restarted. | PASS |
-+----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+
-| TC02: Honeycomb persists configuration through restart of Honeycomb | Checks if Honeycomb maintains configuration after it is restarted. | PASS |
-+----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+
-| TC03: Honeycomb persists configuration through restart of VPP | Checks if Honeycomb updates VPP settings after VPP is restarted. | FAIL |
-+----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+
-| TC04: Honeycomb reverts to defaults if persistence files are invalid | Checks if Honeycomb reverts to default configuration when persistence files are damaged or invalid. | PASS |
-+----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+
-
-
+ +.. |br| raw:: html + + <br /> + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `hc2vpp-csit-integration-1701-ubuntu1404 + <https://jenkins.fd.io/view/hc2vpp/job/hc2vpp-csit-integration-1701-ubuntu1404/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +mgmt-cfg-acl-apihc-apivat-func +`````````````````````````````` + +**Honeycomb access control lists test suite.** + ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++================================================================+===============================================================================+========+ +| TC01: Honeycomb can create ACL classify table | Check if Honeycomb API can create an ACL table. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb can remove ACL table | Check if Honeycomb API can delete an ACL table. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb manages more than one ACL table | Check if Honeycomb API can create another ACL table. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb can add ACL session to table | Check if Honeycomb API can add an ACL session to a table. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb can remove ACL session | Check if Honeycomb API can remove an ACL session. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb manages more than one ACL session on one table | Check if Honeycomb API can add another ACL session to a table. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb enables ACL on interface | Check if Honeycomb API can enable ACL on an interface. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC08: Honeycomb disables ACL on interface | Check if Honeycomb API can disable ACL on an interface. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ +| TC09: Honeycomb can remove one out of multiple ACL tables | Check if Honeycomb API can delete an ACL table if more than one table exists. | PASS | ++----------------------------------------------------------------+-------------------------------------------------------------------------------+--------+ + +mgmt-cfg-int-apihcnc-func +````````````````````````` + +**Netconf test suite. Contains test cases that need to bypass REST API.** + ++--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++==================================================+===============================================================================================+========+ +| TC01: Honeycomb can create and delete interfaces | Repeatedly create and delete an interface through Netconf and check the reply for any errors. | PASS | ++--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+ +| TC02: Transaction revert test case 1 | Configure two conflicting VxLAN tunnels, then verify that neither tunnel exists. | PASS | ++--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+ +| TC03: Transaction revert test case 2 | Configure two conflicting TAP interfaces, then verify that neither interface exists. | PASS | ++--------------------------------------------------+-----------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-int-subint-apihc-apivat-func +````````````````````````````````````` + +**Honeycomb sub-interface management test suite.** + ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++=================================================================================+===========================================================================================================================+========+ +| TC01: Honycomb creates sub-interface | Check if Honeycomb creates a sub-interface. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb sets interface and sub-interface up | Honeycomb changes the state of interface and of its sub-interface to up. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb sets sub-interface down while its super-interface is up | Honeycomb sets the sub-interface down while its super-interface is up. It must be possible. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb sets interface and sub-interface down | Honeycomb changes the state of interface down and then changes the state of its sub-interface down, in this order. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb fails to set sub-interface up while its super-interface is down | Honeycomb tries to set the sub-interface up while its super-interface is down. It must not be possible. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb fails to delete sub-interface | Check if Honeycomb can delete an existing sub-interface. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb adds sub-interface to new bridge domain | Check if Honeycomb adds a sub-interface to bridge domain. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC08: Honeycomb enables tag-rewrite pop 1 | Check if Honeycomb enables tag-rewrite and sets its parameters correctly. Case: pop 1. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC09: Honeycomb enables tag-rewrite push | Check if Honeycomb enables tag-rewrite and sets its parameters correctly. Case: push. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC10: Honeycomb enables tag-rewrite translate 1-2 | Check if Honeycomb enables tag-rewrite and sets its parameters correctly. Case: translate 1-2. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC11: Honeycomb disables tag-rewrite | Check if Honeycomb disables the tag-rewrite. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC12: Honeycomb enables tag-rewrite pop 1 again | Check if Honeycomb can enable tag-rewrite again, once it was disabled by Honeycomb. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC13: Honeycomb modifies the tag-rewrite | Honeycomb sets the tag-rewrite: 1. pop 1, then 2. push, then 3. translate 1 - 2 Then Honeycomb disables the tag-rewrite. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC14: Honeycomb fails to set wrong vlan-type in tag-rewrite | Check that Honeycomb does not accept wrong values of vlan-type in tag-rewrite. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC15: Honeycomb configures sub-interface ipv4 address | Check if Honeycomb can configure an ipv4 address on the sub-interface. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC16: Honeycomb removes sub-interface ipv4 address | Check if Honeycomb can remove configured ipv4 addresses from the sub-interface. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ +| TC17: Honeycomb modifies existing sub-interface ipv4 address | Check if Honeycomb can modify an ipv4 address already configured on the sub-interface. | PASS | ++---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-intip4-intip6-apihc-apivat-func +```````````````````````````````````````` + +**Honeycomb interface management test suite.** + ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++==============================================================+=============================================================================================+========+ +| TC01: Honeycomb configures and reads interface state | Check if Honeycomb API can modify the admin state of VPP interfaces. | PASS | ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb modifies interface IPv4 address with netmask | Check if Honeycomb API can configure interfaces for ipv4 with address and netmask provided. | PASS | ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb removes IPv4 address from interface | Check if Honeycomb API can remove configured ipv4 addresses from interface. | PASS | ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb modifies interface IPv4 address with prefix | Check if Honeycomb API can configure interfaces for ipv4 with address and prefix provided. | PASS | ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb modifies IPv4 neighbor table | Check if Honeycomb API can add and remove ARP entries. | PASS | ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb modifies interface configuration - IPv6 | Check if Honeycomb API can configure interfaces for ipv6. | PASS | ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb modifies interface configuration - MTU | Check if Honeycomb API can configure interface MTU value. | PASS | ++--------------------------------------------------------------+---------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-inttap-apihc-apivat-func +````````````````````````````````` + +**Honeycomb TAP management test suite.** + ++---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++===============================================================+=======================================================================================+========+ +| TC01: Honeycomb configures TAP interface | Check if Honeycomb API can configure a TAP interface. | PASS | ++---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb modifies existing TAP interface configuration | Check if Honeycomb API can re-configure and existing TAP interface with new settings. | PASS | ++---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb removes TAP interface | Check if Honeycomb API can remove TAP interface. | PASS | ++---------------------------------------------------------------+---------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-intvhost-apihc-apivat-func +``````````````````````````````````` + +**Honeycomb vhost-user interface management test suite.** + ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++=================================================================================+==============================================================================================================+========+ +| TC01: Honeycomb creates vhost-user interface - server | Check if Honeycomb creates a vhost-user interface, role: server. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb modifies vhost-user interface - server | Check if Honeycomb can modify properties of existing vhost-user interface, role: server. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb deletes vhost-user interface - server | Check if Honeycomb can delete an existing vhost-user interface, role: server. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb creates vhost-user interface - client | Check if Honeycomb creates a vhost-user interface, role: client. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb modifies vhost-user interface - client | Check if Honeycomb can modify properties of existing vhost-user interface, role: client. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb deletes vhost-user interface - client | Check if Honeycomb can delete an existing vhost-user interface, role: client. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb does not set vhost-user configuration on another interface type | Check if Honeycomb refuses to set vhost-user configuration for interface which is not v3po:vhost-user type. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ +| TC08: Honeycomb does not set invalid vhost-user configuration | Check if Honeycomb refuses to set invalid parameters to vhost-user interface. | FAIL | ++---------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-l2bd-apihc-apivat-func +``````````````````````````````` + +**Honeycomb bridge domain management test suite.** + ++------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++========================================================================+===================================================================================================================+========+ +| TC01: Honeycomb sets up l2 bridge domain | Check if Honeycomb can create bridge domains on VPP node. | PASS | ++------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb manages multiple bridge domains on node | Check if Honeycomb can manage multiple bridge domains on a single node. | PASS | ++------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb removes bridge domains | Check if Honeycomb can remove bridge domains from a VPP node. | PASS | ++------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb assigns interfaces to bridge domain | Check if Honeycomb can assign VPP interfaces to an existing bridge domain. | PASS | ++------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb cannot remove bridge domain with an interface assigned | Check if Honeycomb can remove a bridge domain that has an interface assigned to it. Expect to fail with code 500. | PASS | ++------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-l2fib-apihc-apivat-func +```````````````````````````````` + +**Honeycomb L2 FIB management test suite.** + ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++=========================================================+=================================================================================================================================+========+ +| TC01: Honeycomb adds L2 FIB entry (forward) | Honeycomb creates a bridge domain and assignes an interface to it. Then adds an L2 FIB entry (forward) to the bridge domain. | PASS | ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb adds L2 FIB entry (static, forward) | Honeycomb adds an L2 FIB entry (static, forward) to the bridge domain. | PASS | ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb adds L2 FIB entry (static, filter) | Honeycomb adds an L2 FIB entry (static, filter) to the bridge domain. | PASS | ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb adds and removes L2 FIB entry (forward) | Honeycomb adds an L2 FIB entry (forward) to the bridge domain and then Honeycomb removes it from the bridge domain. | PASS | ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb adds more than one L2 FIB entry | Honeycomb adds three L2 FIB entries to the bridge domain. | PASS | ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb fails to set wrong L2 FIB entry | Honeycomb tries to add an L2 FIB entry with wrong parameters to the bridge domain. It must fail. | PASS | ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb fails to modify existing L2 FIB entry | Honeycomb tries to modify an existing L2 FIB entry. It must fail. | PASS | ++---------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-lisp-apihc-apivat-func +``````````````````````````````` + +**Honeycomb Lisp test suite.** + ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++==================================================================+====================================================================================================+========+ +| TC01: Honeycomb enables Lisp feature | Check if Honeycomb can enable the Lisp feature. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb adds locator set and locator | Check if Honeycomb can configure a locator set. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb configures Lisp - remote mapping - Bridge Domain | Check if Honeycomb can configure a remote Lisp mapping with a bridge domain. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb can remove Lisp mapping | Check if Honeycomb can remove a configured Lisp mapping. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb configures Lisp - remote mapping - VRF | Check if Honeycomb can configure a remote Lisp mapping with VRF. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb configures Lisp - local mapping - Bridge Domain | Check if Honeycomb can configure a local Lisp mapping with a bridge domain. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb configures Lisp - local mapping - VRF | Check if Honeycomb can configure a local Lisp mapping with VRF. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC08: Honeycomb configures Lisp mapping with adjacency | Check if Honeycomb can configure local and remote Lisp mappings with VRF, and configure adjacency. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC09: Honeycomb configures Lisp map resolver | Check if Honeycomb can configure a Lisp map resolver. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC10: Honeycomb enabled Lisp PITR feature | Check if Honeycomb can configure the Lisp PITR feature. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ +| TC11: Honeycomb can remove configuration of Lisp features | Check if Honeycomb can disable all Lisp features. | PASS | ++------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-nsh-apihc-apivat-func +`````````````````````````````` + +**Honeycomb NSH test suite.** + ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++=====================================================================+===========================================================================+========+ +| TC01: Honeycomb can configure NSH entry | Check if Honeycomb can configure an NSH entry. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| TC02: Honeycomb can remove NSH entry | Check if Honeycomb can remove an existing NSH entry. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| TC03: Honeycomb can configure new NSH entry | Check if Honeycomb can configure an NSH antry after one has been deleted. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| TC04: Honeycomb can configure multiple NSH entries at the same time | Check if Honeycomb can configure an NSH entry when one already exists. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| TC05: Honeycomb can configure NSH map | Check if Honeycomb can configure an NSH map. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| TC06: Honeycomb can remove NSH map | Check if Honeycomb can remove an existing NSH map. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| TC07: Honeycomb can modify existing NSH map | Check if Honeycomb can configure an NSH map after one has been deleted. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ +| TC08: Honeycomb can configure multiple NSH maps at the same time | Check if Honeycomb can configure and NSH map when one already exists. | PASS | ++---------------------------------------------------------------------+---------------------------------------------------------------------------+--------+ + +mgmt-cfg-pbb-apihc-apivat-func +`````````````````````````````` + +**Honeycomb provider backbone bridge test suite.** + ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++==================================================================================+================================================================================================================================================+========+ +| TC01: Honeycomb sets PBB sub-interface | Honeycomb creates a new PBB sub-interface. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb modifies existing PBB sub-interface | Honeycomb modifies an existing PBB sub-interface. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb deletes existing PBB sub-interface | Honeycomb deletes an existing PBB sub-interface. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb fails to set wrong destination-address for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter destination-address, type yang:mac-address. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb fails to set wrong source-address for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter source-address, type yang:mac-address. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb fails to set wrong b-vlan-tag-vlan-id for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter b-vlan-tag-vlan-id, type uint16, 12 bit range, range 1..4095. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb fails to set wrong i-tag-isid for new PBB sub-interface | Honeycomb fails to create a new PBB sub-interface with wrong value of parameter i-tag-isid, type uint32, 24 bit range, range 1..16777215. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ +| TC08: Honeycomb fails to create new PBB sub-interface without vlan tag | Honeycomb fails to create a new PBB sub-interface without parameter b-vlan-tag-vlan-id. | PASS | ++----------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-snat44-apihc-apivat-func +````````````````````````````````` + +**Honeycomb NAT test suite.** + ++-----------------------------------------------------+-----------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++=====================================================+=================================================================+========+ +| TC01: Honeycomb configures NAT entry | Honeycomb configures a static NAT entry. | PASS | ++-----------------------------------------------------+-----------------------------------------------------------------+--------+ +| TC02: Honeycomb removes NAT entry | Honeycomb removes a configured static NAT entry. | PASS | ++-----------------------------------------------------+-----------------------------------------------------------------+--------+ +| TC03: Honeycomb configures multiple NAT entries | Honeycomb configures two static NAT entries. | PASS | ++-----------------------------------------------------+-----------------------------------------------------------------+--------+ +| TC04: Honeycomb enables NAT on interface - inbound | Honeycomb configures NAT on an interface in inbound direction. | PASS | ++-----------------------------------------------------+-----------------------------------------------------------------+--------+ +| TC05: Honeycomb removes NAT interface configuration | Honeycomb removes NAT configuration from an interface. | PASS | ++-----------------------------------------------------+-----------------------------------------------------------------+--------+ +| TC06: Honeycomb enables NAT on interface - outbound | Honeycomb configures NAT on an interface in outbound direction. | PASS | ++-----------------------------------------------------+-----------------------------------------------------------------+--------+ + +mgmt-cfg-vxlan-apihc-apivat-func +```````````````````````````````` + +**Honeycomb VxLAN management test suite.** + ++----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++============================================================================+======================================================================================================+========+ +| TC01: Honeycomb configures VxLAN tunnel | Check if Honeycomb API can configure VxLAN settings. | PASS | ++----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb disables VxLAN tunnel | Check if Honeycomb API can reset VxLAN configuration. | PASS | ++----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb can configure VXLAN tunnel after one has been disabled | Check if Honeycomb API can configure VxLAN settings again after previous settings have been removed. | PASS | ++----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb does not set VxLAN configuration on another interface type | Check if Honeycomb API prevents setting VxLAN on incorrect interface. | PASS | ++----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb does not set invalid VxLAN configuration | Check if Honeycomb API prevents setting incorrect VxLAN settings. | PASS | ++----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb configures VxLAN tunnel with ipv6 | Check if Honeycomb API can configure VxLAN with ipv6 settings. | PASS | ++----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+--------+ + +mgmt-cfg-vxlangpe-apihc-apivat-func +``````````````````````````````````` + +**Honeycomb VxLAN-GPE management test suite.** + ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++=============================================================================+=================================================================================================+========+ +| TC01: Honeycomb creates VxLAN GPE tunnel | Check if Honeycomb API can configure a VxLAN GPE tunnel. | PASS | ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb removes VxLAN GPE tunnel | Check if Honeycomb API can remove VxLAN GPE tunnel. | PASS | ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb sets wrong interface type while creating VxLAN GPE tunnel | Check if Honeycomb refuses to create a VxLAN GPE tunnel with a wrong interface type set. | PASS | ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb sets wrong protocol while creating VxLAN GPE tunnel | Check if Honeycomb refuses to create a VxLAN GPE tunnel with a wrong next-protocol set. | PASS | ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ +| TC05: Honeycomb sets VxLAN GPE tunnel on existing interface with wrong type | Check if Honeycomb refuses to create a VxLAN GPE tunnel on existing interface with wrong type. | PASS | ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ +| TC06: Honeycomb creates VxLAN GPE tunnel with ipv6 | Check if Honeycomb API can configure a VxLAN GPE tunnel with IPv6 addresses. | PASS | ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ +| TC07: Honeycomb creates a second VxLAN GPE tunnel with ipv6 | Check if Honeycomb API can configure another VxLAN GPE tunnel with IPv6 addresses. | PASS | ++-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------+--------+ + +mgmt-notif-apihcnc-func +``````````````````````` + +**Honeycomb notifications test suite.** + ++--------------------------------------------------------------+--------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++==============================================================+==================================================================================================+========+ +| TC01: Honeycomb sends notification on interface state change | Check if Honeycomb sends a state-changed notification when the state of an interface is changed. | PASS | ++--------------------------------------------------------------+--------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb sends notification on interface deletion | Check if Honeycomb sends an interface-deleted notification when an interface is deleted. | PASS | ++--------------------------------------------------------------+--------------------------------------------------------------------------------------------------+--------+ + +mgmt-statepersist-apihc-func +```````````````````````````` + +**Honeycomb configuration persistence test suite.** + ++----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+ +| Name | Documentation | Status | ++==================================================================================+=====================================================================================================+========+ +| TC01: Honeycomb persists configuration through restart of both Honeycomb and VPP | Checks if Honeycomb maintains configuration after both Honeycomb and VPP are restarted. | FAIL | ++----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+ +| TC02: Honeycomb persists configuration through restart of Honeycomb | Checks if Honeycomb maintains configuration after it is restarted. | FAIL | ++----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+ +| TC03: Honeycomb persists configuration through restart of VPP | Checks if Honeycomb updates VPP settings after VPP is restarted. | FAIL | ++----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+ +| TC04: Honeycomb reverts to defaults if persistence files are invalid | Checks if Honeycomb reverts to default configuration when persistence files are damaged or invalid. | PASS | ++----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------+--------+ diff --git a/docs/report/honeycomb_functional_tests_virl/overview.rst b/docs/report/honeycomb_functional_tests_virl/overview.rst index 70a2777e7f..2bfc89a814 100644 --- a/docs/report/honeycomb_functional_tests_virl/overview.rst +++ b/docs/report/honeycomb_functional_tests_virl/overview.rst @@ -1,14 +1,17 @@ Overview
========
-Honeycombe Functional Test Topologies in VIRL
----------------------------------------------
+Tested Topologies VIRL
+----------------------
CSIT Honeycomb functional tests are executed on virtualized topologies created
using Virtual Internet Routing Lab (VIRL) simulation platform contributed by
Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project. All
-tests are executed in two node logical test topology - TG node and SUT node.
-Used logical test topology is shown in the figure below.::
+tests are executed in two node logical test topology - Traffic Generator (TG)
+node and Systems Under Test (SUT1) node connected in a loop. Logical test
+topology is shown in the figure below.
+
+::
+------------------------+
| |
@@ -28,12 +31,19 @@ Used logical test topology is shown in the figure below.:: +------------------> <------------------+
+-----------+
-Virtual testbeds are created dynamically whenever a verification job is
-started triggered by the patch submission to gerrit and destroyed upon
-completion of all functional tests. During test execution, all nodes are
-reachable thru the MGMT network connected to every node via dedicated NICs and
-links (not shown above for clarity). Each node is a Virtual Machine and each
-connection that is drawn on the diagram is available for use in any test case.
+SUT1 is a VM (Ubuntu or Centos, depending on the test suite), TG is a Traffic
+Generator (TG, another Ubuntu VM). SUTs run Honeycomb and VPP SW applications
+in Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy
+SW application as a packet Traffic Generator. Logical connectivity between
+SUTs and to TG is provided using virtual NICs using VMs' virtio driver.
+
+Virtual testbeds are created on-demand whenever a verification job is started
+(e.g. triggered by the gerrit patch submission) and destroyed upon completion
+of all functional tests. Each node is a Virtual Machine and each connection
+that is drawn on the diagram is available for use in any test case. During the
+test execution, all nodes are reachable thru the Management network connected
+to every node via dedicated virtual NICs and virtual links (not shown above
+for clarity).
Honeycomb Functional Tests Overview
-----------------------------------
diff --git a/docs/report/index.rst b/docs/report/index.rst index d1cc5913da..fe3fe55c5f 100644 --- a/docs/report/index.rst +++ b/docs/report/index.rst @@ -15,3 +15,4 @@ CSIT 17.01 vpp_functional_results_virl/index vpp_functional_results_vpptest/index honeycomb_functional_results_virl/index + diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index e3b343c815..dc86f8600e 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -1,16 +1,42 @@ Overview
========
-This is the Fast Data Project (FD.io) Continuous System Integration and Testing
-(CSIT) project report for CSIT |release| system testing of VPP-17.01 release.
-Initial sections of the report describe FD.io functional and performance tests
-and their continuous execution achieved in CSIT |release|. This is followed by
-high-level overview of FD.io test environments running in Linux Foundation
-(LF) FD.io Continuous Performance Labs), CSIT |release| release notes and
-result highlights. Subsequent sections list the functional and performance
-detailed test results generated by LF FD.io Jenkins system executing CSIT jobs
-running tests against the VPP-17.01 release and associated sub-systems
-(HoneyComb, DPDK).
+This is the **F**\ast **D**\ata **I**/**O** Project (FD.io) **C**\ontinuous
+**S**\ystem **I**\ntegration and **T**\esting (CSIT) project report for CSIT
+|release| system testing of VPP-17.01 release. Initial sections of the report
+describe FD.io functional and performance tests and their continuous execution
+achieved in CSIT |release|. This is followed by high-level overview of FD.io
+test environments running in Linux Foundation (LF) FD.io Continuous
+Performance Labs), CSIT |release| release notes and result highlights.
+Subsequent sections list the functional and performance detailed test results
+generated by LF FD.io Jenkins system executing CSIT jobs running tests
+against the VPP-17.01 release and associated sub-systems (HoneyComb, DPDK).
+
+The exact structure of the report is as follows:
+
+#. ``Introduction`` - Includes: ``Overview`` - this section;
+ ``CSIT Test Naming`` - describing CSIT naming convention for test suites and
+ test cases, important to understand to correctly interpret the reported
+ results; ``General Remarks`` - additional notes related to this report.
+#. ``VPP Performance Tests HW`` - Summary of VPP performance tests executed in
+ physical Hardware testbed. Includes: ``Overview`` - containing topology
+ description and list of executed tests; ``Performance Tests Naming`` - lists
+ CSIT naming specifics for performance tests; ``CSIT Release Notes`` - with
+ changes in CSIT |release| including major test additions, performance
+ improvements, changes in testing environment and methodology and known
+ issues; ``Performance Measurements Methodology`` - details testing
+ methodology followed for multi-thread multi-core, packet throughput and
+ latency, and KVM VM vhost tests; ``Report Addendum Targets`` - lists content
+ to be added to this report as part of CSIT |release|.
+#. ``Testpmd Performance Tests HW`` - ...
+#. ``VPP Functional Tests VIRL`` - ...
+#. ``VPP Functional Tests VPPtest`` - ...
+#. ``HoneyComb Functional Tests VIRL`` - ..
+#. ``VPP Performance Results HW`` - ...
+#. ``Testpmd Performance Results HW`` - ...
+#. ``VPP Functional Results VIRL`` - ...
+#. ``VPP Functional Results VPPtest`` - ...
+#. ``HoneyComb Functional Results VIRL`` - ...
FD.io CSIT system is developed using two main coding platforms: Robot
Framework (RF) and Python. CSIT |release| source code for the executed test
diff --git a/docs/report/testpmd_performance_tests_hw/csit_release_notes.rst b/docs/report/testpmd_performance_tests_hw/csit_release_notes.rst index 3b138a6f0c..cc1bf8dd1c 100644 --- a/docs/report/testpmd_performance_tests_hw/csit_release_notes.rst +++ b/docs/report/testpmd_performance_tests_hw/csit_release_notes.rst @@ -4,68 +4,10 @@ CSIT Release Notes Changes in CSIT |release| ------------------------- -#. Naming change for all Testpmd performance test suites and test cases. - #. Added Testpmd tests - new NICs - Intel x520 - -Performance Tests Naming ------------------------- - -CSIT |release| introduced a common structured naming convention for all -performance and functional tests. This change was driven by substantially -growing number and type of CSIT test cases. Firstly, the original practice did -not always follow any strict naming convention. Secondly test names did not -always clearly capture tested packet encapsulations, and the actual type or -content of the tests. Thirdly HW configurations in terms of NICs, ports and -their locality were not captured either. These were but few reasons that drove -the decision to change and define a new more complete and stricter test naming -convention, and to apply this to all existing and new test cases. - -The new naming should be intuitive for majority of the tests. The complete -description of CSIT test naming convention is provided on `CSIT test naming wiki -<https://wiki.fd.io/view/CSIT/csit-test-naming>`_. - -Here few illustrative examples of the new naming usage for performance test -suites: - -#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** - - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching - with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE - on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline - switching with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel - x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery. - - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput - discovery. - -#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, - P2V2P, NIC2VMchain2NIC, P2V2V2P** - - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation- - VirtPortConfig-VMconfig-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports - of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from four vhost interfaces and two VMs, NDR throughput - discovery. - Multi-Thread and Multi-Core Measurements ---------------------------------------- @@ -79,12 +21,12 @@ HyperThreading Enabled (requires BIOS settings change and hard reboot). **Multi-core Test** - CSIT |release| multi-core tests are executed in the following Testpmd thread and core configurations: -#. 1t1c - 1 Testpmd worker thread on 1 CPU physical core. -#. 2t2c - 2 Testpmd worker threads on 2 CPU physical cores. -#. 4t4c - 4 Testpmd threads on 4 CPU physical cores. +#. 1t1c - 1 Testpmd pmd thread on 1 CPU physical core. +#. 2t2c - 2 Testpmd pmd threads on 2 CPU physical cores. +#. 4t4c - 4 Testpmd pmd threads on 4 CPU physical cores. -Note that in quite a few test cases running Testpmd on 2 or 4 physical cores -hits the tested NIC I/O bandwidth or packets-per-second limit. +Note that in many tests running Testpmd reaches tested NIC I/O bandwidth +or packets-per-second limit. Packet Throughput Measurements ------------------------------ @@ -137,8 +79,8 @@ Reported latency values are measured using following methodology: interface line. -Report Addendum Tests - More NICs ---------------------------------- +Report Addendum Tests - Additional NICs +--------------------------------------- Adding test cases with more NIC types. Once the results become available, they will be published as an addendum to the current version of CSIT |release| diff --git a/docs/report/testpmd_performance_tests_hw/overview.rst b/docs/report/testpmd_performance_tests_hw/overview.rst index 5bd81abac2..4ef3982e49 100644 --- a/docs/report/testpmd_performance_tests_hw/overview.rst +++ b/docs/report/testpmd_performance_tests_hw/overview.rst @@ -1,11 +1,13 @@ Overview ======== -Testpmd Performance Test Topologies ------------------------------------ +Tested Topologies HW +-------------------- CSIT Testpmd performance tests are executed on physical baremetal servers hosted -by LF FD.io project. Testbed physical topology is shown in the figure below.:: +by LF FD.io project. Testbed physical topology is shown in the figure below. + +:: +------------------------+ +------------------------+ | | | | @@ -25,13 +27,14 @@ by LF FD.io project. Testbed physical topology is shown in the figure below.:: | | +-----------+ -SUT1 and SUT2 are two System Under Test servers (Cisco UCS C240, each with two -Intel XEON CPUs), TG is a Traffic Generator (TG, another Cisco UCS C240, with -two Intel XEON CPUs). SUTs run Testpmd SW application in Linux user-mode as a -Device Under Test (DUT). TG runs TRex SW application as a packet Traffic -Generator. Physical connectivity between SUTs and to TG is provided using -different NIC models that need to be tested for performance. Currently -installed and tested NIC models include: +SUT1 and SUT2 are two System Under Test servers (currently Cisco UCS C240, +each with two Intel XEON CPUs), TG is a Traffic Generator (TG, currently +another Cisco UCS C240, with two Intel XEON CPUs). SUTs run Testpmd SW +application in Linux user-mode as a Device Under Test (DUT). TG runs TRex SW +application as a packet Traffic Generator. Physical connectivity between SUTs +and to TG is provided using direct links (no L2 switches) connecting different +NIC models that need to be tested for performance. Currently installed and +tested NIC models include: #. 2port10GE X520-DA2 Intel. #. 2port10GE X710 Intel. @@ -39,26 +42,39 @@ installed and tested NIC models include: #. 2port40GE VIC1385 Cisco. #. 2port40GE XL710 Intel. -Detailed LF FD.io test bed specification and topology is described on `CSIT LF -testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_. +Detailed LF FD.io test bed specification and topology is described in +`wiki CSIT LF testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_. -Testpmd Performance Tests Overview ----------------------------------- +Testing Summary +--------------- -Performance tests are split into two main categories: +Performance tests are split into the two main categories: - Throughput discovery - discovery of packet forwarding rate using binary search - in accordance to RFC2544. + in accordance with RFC2544. - - NDR - discovery of Non Drop Rate, zero packet loss. - - PDR - discovery of Partial Drop Rate, with specified non-zero packet loss. + - NDR - discovery of Non Drop Rate packet throughput, at zero packet loss; + followed by packet one-way latency measurements at 10%, 50% and 100% of + discovered NDR throughput. + - PDR - discovery of Partial Drop Rate, with specified non-zero packet loss + currently set to 0.5%; followed by packet one-way latency measurements at + 100% of discovered PDR throughput. - Throughput verification - verification of packet forwarding rate against - previously discovered throughput rate. These tests are currently done against + previously discovered NDR throughput. These tests are currently done against 0.9 of reference NDR, with reference rates updated periodically. -CSIT |release| includes following performance test suites: +CSIT |release| includes following performance test suites, listed per NIC type: - 2port10GE X520-DA2 Intel - - **L2XC** - L2 Cross-Connect forwarding of untagged Ethernet frames. + - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between + two Interfaces. + +Execution of performance tests takes time, especially the throughput discovery +tests. Due to limited HW testbed resources available within FD.io labs hosted +by Linux Foundation, the number of tests for NICs other than X520 (a.k.a. +Niantic) has been limited to few baseline tests. Over time we expect the HW +testbed resources to grow, and will be adding complete set of performance +tests for all models of hardware to be executed regularly and(or) +continuously. diff --git a/docs/report/vpp_functional_results_virl/functional_results_virl.rst b/docs/report/vpp_functional_results_virl/functional_results_virl.rst index 2bf57c909f..5e823f5f6d 100644 --- a/docs/report/vpp_functional_results_virl/functional_results_virl.rst +++ b/docs/report/vpp_functional_results_virl/functional_results_virl.rst @@ -6,6 +6,13 @@ DHCP - Client and Proxy
```````````````````````
+.. note::
+
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4base-ip4dhcpclient-func
@@ -70,7 +77,13 @@ eth2p-ethip6-ip6base-ip6dhcpproxy-func GRE Overlay Tunnels
```````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4gre-ip4base-func
''''''''''''''''''''''''''''
@@ -106,7 +119,13 @@ eth2p-ethip4gre-ip4base-func L2BD Ethernet Switching
```````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-eth-l2bdbasemaclrn-eth-2vhost-1vm-func
''''''''''''''''''''''''''''''''''''''''''''
@@ -224,7 +243,13 @@ eth4p-eth-l2bdbasemaclrn-l2shg-func L2XC Ethernet Switching
```````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-eth-l2xcbase-eth-2vhost-1vm-func
''''''''''''''''''''''''''''''''''''''
@@ -275,7 +300,13 @@ eth2p-eth-l2xcbase-func LISP Overlay Tunnels
````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
api-crud-lisp-func
''''''''''''''''''
@@ -833,7 +864,13 @@ eth2p-ethip6lispgpe-ip6basevrf-func Softwire Tunnels
````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func
'''''''''''''''''''''''''''''''''''''''''''''''''''''''
@@ -925,7 +962,13 @@ eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func Cop Address Security
````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4base-copblklistbase-func
''''''''''''''''''''''''''''''''''''''''
@@ -1014,7 +1057,13 @@ eth2p-ethip6-ip6base-copwhlistbase-func IPSec - Tunnels and Transport
`````````````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4ipsectnl-ip4base-func
'''''''''''''''''''''''''''''''''
@@ -1175,7 +1224,13 @@ eth2p-ethip6ipsectpt-ip6base-func IPv6 Routed-Forwarding
``````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip6-ip6base-func
'''''''''''''''''''''''''
@@ -1237,7 +1292,13 @@ eth2p-ethip6-ip6base-ip6ra-func RPF Source Security
```````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4base-rpf-func
'''''''''''''''''''''''''''''
@@ -1261,7 +1322,13 @@ eth2p-ethip4-ip4base-rpf-func Tap Interface
`````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
api-crud-tap-func
'''''''''''''''''
@@ -1359,7 +1426,13 @@ Telemetry - IPFIX and SPAN IPFIX Netflow Statistics
''''''''''''''''''''''''
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4base-ip4ipfixbase-func
......................................
@@ -1426,7 +1499,13 @@ eth2p-ethip6-ip6base-ip6ipfixscale-func SPAN Port Mirroring
'''''''''''''''''''
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4base-spanrx-func
................................
@@ -1471,7 +1550,13 @@ eth2p-ethip6-ip6base-spanrx-func VRF Routed-Forwarding
`````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4basevrf-func
''''''''''''''''''''''''''''
@@ -1534,7 +1619,13 @@ eth2p-ethip6-ip6basevrf-func iACL Security
`````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-eth-l2xcbase-iaclbase-func
''''''''''''''''''''''''''''''''
@@ -1626,7 +1717,13 @@ eth2p-ethip6-ip6base-iaclbase-func IPv4 Routed-Forwarding
``````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4base-func
'''''''''''''''''''''''''
@@ -1711,7 +1808,13 @@ eth2p-ethip4-ip4base-ip4proxyarp-func QoS Policer Metering
````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-ethip4-ip4base-ipolicemarkbase-func
'''''''''''''''''''''''''''''''''''''''''
@@ -1766,7 +1869,13 @@ eth2p-ethip6-ip6base-ipolicemarkbase-func VLAN Tag Translation
````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-dot1ad--dot1q-l2bdbasemaclrn-vlantrans21-func
'''''''''''''''''''''''''''''''''''''''''''''''''''
@@ -1976,7 +2085,13 @@ eth2p-dot1q-l2xcbase-vlantrans11-func VXLAN Overlay Tunnels
`````````````````````
+.. note::
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-functional-1701-virl
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-functional-1701-virl/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
eth2p-dot1qip4vxlan-l2bdbasemaclrn-func
'''''''''''''''''''''''''''''''''''''''
diff --git a/docs/report/vpp_functional_tests_virl/overview.rst b/docs/report/vpp_functional_tests_virl/overview.rst index cf7a5d2e34..16c7ef7ba0 100644 --- a/docs/report/vpp_functional_tests_virl/overview.rst +++ b/docs/report/vpp_functional_tests_virl/overview.rst @@ -1,16 +1,18 @@ Overview
========
-VPP Functional Test Topologies in VIRL
---------------------------------------
+Tested Topologies VIRL
+----------------------
CSIT VPP functional tests are executed on virtualized topologies created using
Virtual Internet Routing Lab (VIRL) simulation platform contributed by Cisco.
VIRL runs on physical baremetal servers hosted by LF FD.io project. Majority
of the tests are executed in the three node logical test topology - Traffic
Generator (TG) node and two Systems Under Test (SUT) nodes connected in a
-loop. Some tests use two node logical test topology - TG node and SUT node.
-Both logical test topologies are shown in the figures below:::
+loop. Some tests use two node logical test topology - TG node and SUT1 node.
+Both logical test topologies are shown in the figures below.
+
+::
+------------------------+ +------------------------+
| | | |
@@ -48,18 +50,27 @@ Both logical test topologies are shown in the figures below::: +------------------> <------------------+
+-----------+
-Virtual testbeds are created dynamically whenever a verification job is
-started triggered by the patch submission to gerrit and destroyed upon
-completion of all functional tests. During test execution, all nodes are
-reachable thru the MGMT network connected to every node via dedicated NICs and
-links (not shown above for clarity). Each node is a Virtual Machine and each
-connection that is drawn on the diagram is available for use in any test case.
+SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG
+is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in
+Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW
+application as a packet Traffic Generator. Logical connectivity between SUTs
+and to TG is provided using virtual NICs using VMs' virtio driver.
+
+Virtual testbeds are created on-demand whenever a verification job is started
+(e.g. triggered by the gerrit patch submission) and destroyed upon completion
+of all functional tests. Each node is a Virtual Machine and each connection
+that is drawn on the diagram is available for use in any test case. During the
+test execution, all nodes are reachable thru the Management network connected
+to every node via dedicated virtual NICs and virtual links (not shown above
+for clarity).
+
+For the test cases that require DUT (VPP) to communicate with VM over the
+vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the
+duration of these particular test cases only. DUT (VPP) test topology with VM
+is shown in the figure below including the applicable packet flow thru the VM
+(marked in the figure with ``***``).
-For test cases that require DUT (VPP) to communicate with VM over vhost-user
-interfaces, a nested VM is created on SUT1 and/or SUT2 for the duration of
-that particular test case only. DUT (VPP) test topology with VM is shown in
-the figure below including the applicable packet flow thru the VM (marked with
-``*``).::
+::
+------------------------+ +------------------------+
| +----------+ | | +----------+ |
diff --git a/docs/report/vpp_performance_results_hw/performance_results_hw.rst b/docs/report/vpp_performance_results_hw/performance_results_hw.rst index 01ff0e8cbf..594f8911a9 100644 --- a/docs/report/vpp_performance_results_hw/performance_results_hw.rst +++ b/docs/report/vpp_performance_results_hw/performance_results_hw.rst @@ -6,6 +6,16 @@ L2 Ethernet Switching
=====================
+.. note::
+
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-perf-1701-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
+ `csit-vpp-perf-1701-long
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
+
10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc
``````````````````````````````````````
@@ -363,6 +373,16 @@ L2 Ethernet Switching IPv4 Routed-Forwarding
======================
+.. note::
+
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-perf-1701-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
+ `csit-vpp-perf-1701-long
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
+
10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc
````````````````````````````````````````````````````
@@ -792,6 +812,16 @@ IPv4 Routed-Forwarding IPv6 Routed-Forwarding
======================
+.. note::
+
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-perf-1701-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
+ `csit-vpp-perf-1701-long
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
+
10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrpdrdisc
````````````````````````````````````````````````````
@@ -1156,6 +1186,16 @@ IPv6 Routed-Forwarding IPv4 Overlay Tunnels
====================
+.. note::
+
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-perf-1701-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
+ `csit-vpp-perf-1701-long
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
+
10ge2p1x520-ethip4lispip4-ip4base-ndrpdrdisc
````````````````````````````````````````````
@@ -1379,6 +1419,16 @@ IPv4 Overlay Tunnels IPv6 Overlay Tunnels
====================
+.. note::
+
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-perf-1701-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
+ `csit-vpp-perf-1701-long
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
+
10ge2p1x520-ethip6lispip4-ip6base-ndrpdrdisc
````````````````````````````````````````````
@@ -1492,6 +1542,16 @@ IPv6 Overlay Tunnels VM Vhost Connections
====================
+.. note::
+
+ Data sources for reported test results: i) FD.io test executor jobs
+ `csit-vpp-perf-1701-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
+ `csit-vpp-perf-1701-long
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
+ , ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
+
10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc
``````````````````````````````````````````````````````````
diff --git a/docs/report/vpp_performance_tests_hw/csit_release_notes.rst b/docs/report/vpp_performance_tests_hw/csit_release_notes.rst index 4292a2cd10..dc7f36db09 100644 --- a/docs/report/vpp_performance_tests_hw/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests_hw/csit_release_notes.rst @@ -96,6 +96,110 @@ in VPP-17.01 for the following CSIT |release| tests: | IPv4 | 10ge2p1x520:64B-1t1c-ethip4-ip4base-ndrdisc | 8.7 Mpps | 9.7 Mpps | 12% |
+-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
+Non-drop rate search:
+
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | Relative Improvement |
++===================+=================================================================+===========+=================+======================+
+| L2XC | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-ndrdisc | 9.4 Mpps | 12.7 Mpps | 35% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC | 10ge2p1xl710: 64B-1t1c-eth-l2xcbase-ndrdisc | 9.5 Mpps | 12.2..12.4 Mpps | 28..30% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC dot1ad | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-ndrdisc | 7.4 Mpps | 8.8..9.0 Mpps | 19..23% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC dot1q | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-ndrdisc | 7.5 Mpps | 8.8..9.0 Mpps | 17..20% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC VxLAN | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc | 5.4 Mpps | 6.5 Mpps | 20% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc | 0.5 Mpps | 2.8..3.2 Mpps | 460..540% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2BD | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-ndrdisc | 7.8 Mpps | 10.4..10.6 Mpps | 33..36% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2BD-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc | 0.4 Mpps | 2.7..2.8 Mpps | 575..600% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ndrdisc | 8.7 Mpps | 9.7 Mpps | 11% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 COP | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc | 7.1 Mpps | 8.3..8.5 Mpps | 17..20% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 FIB 200k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-ndrdisc | 8.5 Mpps | 9.0 Mpps | 6% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 FIB 20k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-ndrdisc | 8.5 Mpps | 9.0..9.2 Mpps | 6..8% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 FIB 2M | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-ndrdisc | 8.5 Mpps | 7.8..8.1 Mpps | -8..-5% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 iAcl | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc | 6.9 Mpps | 7.6..7.8 Mpps | 10..13% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 Policer | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc | 6.9 Mpps | 7.4..7.6 Mpps | 7..10% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 LISP | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-ndrdisc | 4.4 Mpps | 4.8 Mpps | 9% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 vhost | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc | 0.3 Mpps | 2.6 Mpps | 767% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-ndrdisc | 3.0 Mpps | 7.3..7.7 Mpps | 143..157% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 COP | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc | 6.1 Mpps | 6.1..6.5 Mpps | 0..7% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 FIB 200k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-ndrdisc | 6.5 Mpps | 5.3..5.7 Mpps | -18..-12% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-ndrdisc | 6.9 Mpps | 6.5 Mpps | -6% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc | 5.3 Mpps | 4.2 Mpps | -21% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc | 6.5 Mpps | 6.1..6.5 Mpps | -6..0% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+
+Partial drop rate search, LT = 0.5%:
+
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | Relative Improvement |
++===================+=================================================================+===========+=================+======================+
+| L2XC | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-pdrdisc | 9.4 Mpps | 12.7..12.9 Mpps | 35..37% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC | 10ge2p1xl710: 64B-1t1c-eth-l2xcbase-pdrdisc | no data | no data | |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC dot1ad | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-pdrdisc | 7.4 Mpps | 8.8..9.1 Mpps | 19..23% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC dot1q | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-pdrdisc | 7.5 Mpps | 8.8..9.0 Mpps | 17..20% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC VxLAN | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-pdrdisc | 5.4 Mpps | 6.5 Mpps | 20% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2XC-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc | 2.6 Mpps | 3.2..3.3 Mpps | 23..26% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2BD | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-pdrdisc | 7.8 Mpps | 10.6 Mpps | 36% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| L2BD-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc | 2.1 Mpps | 2.9 Mpps | 38% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-pdrdisc | 8.7 Mpps | 9.7 Mpps | 11% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 COP | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc | 7.1 Mpps | 8.3..8.5 Mpps | 17..20% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 FIB 200k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-pdrdisc | 8.5 Mpps | 9.0 Mpps | 6% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 FIB 20k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-pdrdisc | 8.5 Mpps | 9.0..9.2 Mpps | 6..8% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 FIB 2M | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-pdrdisc | 8.3 Mpps | 8.1 Mpps | -2% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 iAcl | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc | 7.1 Mpps | 7.6..7.8 Mpps | 7..10% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 Policer | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc | 7.1 Mpps | 7.4..7.6 Mpps | 4..7% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 LISP | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-pdrdisc | 4.6 Mpps | 4.8 Mpps | 9% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv4 vhost | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc | 2.0 Mpps | 2.7 Mpps | 35% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-pdrdisc | 7.7 Mpps | 7.3..7.7 Mpps | -5..0% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 COP | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc | 6.1 Mpps | 6.1..6.5 Mpps | 0..7% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 FIB 200k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-pdrdisc | 6.9 Mpps | 5.3..5.7 Mpps | -23..-17% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-pdrdisc | 6.9 Mpps | 6.5 Mpps | -6% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc | 5.3 Mpps | 4.2 Mpps | -21% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+| IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc | 6.5 Mpps | 6.1..6.5 Mpps | -6..0% |
++-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
+
This is in line with VPP code optimizations listed in `VPP-17.01 release notes
<https://docs.fd.io/vpp/17.01/release_notes_1701.html>`_.
diff --git a/docs/report/vpp_performance_tests_hw/overview.rst b/docs/report/vpp_performance_tests_hw/overview.rst index a0425fa114..e1afabf1d8 100644 --- a/docs/report/vpp_performance_tests_hw/overview.rst +++ b/docs/report/vpp_performance_tests_hw/overview.rst @@ -1,11 +1,13 @@ Overview
========
-VPP Performance Test Topologies
--------------------------------
+Tested Topologies HW
+--------------------
CSIT VPP performance tests are executed on physical baremetal servers hosted by LF
-FD.io project. Testbed physical topology is shown in the figure below.::
+FD.io project. Testbed physical topology is shown in the figure below.
+
+::
+------------------------+ +------------------------+
| | | |
@@ -39,13 +41,15 @@ installed and tested NIC models include: #. 2port40GE VIC1385 Cisco.
#. 2port40GE XL710 Intel.
-Detailed LF FD.io test bed specification and topology is described on `CSIT LF
-testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
+Detailed LF FD.io test bed specification and physical topology are described
+in `wiki CSIT LF testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
For test cases that require DUT (VPP) to communicate with VM over vhost-user
interfaces, a VM is created on SUT1 and SUT2. DUT (VPP) test topology with VM
is shown in the figure below including applicable packet flow thru the VM
-(marked with \*\*\*).::
+(marked in the figure with ``***``).
+
+::
+------------------------+ +------------------------+
| +----------+ | | +----------+ |
@@ -70,25 +74,29 @@ is shown in the figure below including applicable packet flow thru the VM +-----------+
Note that for VM tests, packets are switched by DUT (VPP) twice, hence the
-throughput rates measured by TG must be multiplied by two to represent the
-actual DUT packet forwarding rate.
+throughput rates measured by TG (and listed in this report) must be multiplied
+by two to represent the actual DUT aggregate packet forwarding rate.
VPP Performance Tests Overview
------------------------------
-Performance tests are split into two main categories:
+Performance tests are split into the two main categories:
- Throughput discovery - discovery of packet forwarding rate using binary search
in accordance to RFC2544.
- - NDR - discovery of Non Drop Rate, zero packet loss.
- - PDR - discovery of Partial Drop Rate, with specified non-zero packet loss.
+ - NDR - discovery of Non Drop Rate packet throughput, at zero packet loss;
+ followed by packet one-way latency measurements at 10%, 50% and 100% of
+ discovered NDR throughput.
+ - PDR - discovery of Partial Drop Rate, with specified non-zero packet loss
+ currently set to 0.5%; followed by packet one-way latency measurements at
+ 100% of discovered PDR throughput.
- Throughput verification - verification of packet forwarding rate against
previously discovered throughput rate. These tests are currently done against
0.9 of reference NDR, with reference rates updated periodically.
-CSIT |release| includes following performance test suites:
+CSIT |release| includes following performance test suites, listed per NIC type:
- 2port10GE X520-DA2 Intel
@@ -138,3 +146,11 @@ CSIT |release| includes following performance test suites: - **L2BD** - L2 Bridge-Domain switched-forwarding of untagged Ethernet frames
with MAC learning.
+Execution of performance tests takes time, especially the throughput discovery
+tests. Due to limited HW testbed resources available within FD.io labs hosted
+by Linux Foundation, the number of tests for NICs other than X520 (a.k.a.
+Niantic) has been limited to few baseline tests. Over time we expect the HW
+testbed resources to grow, and will be adding complete set of performance
+tests for all models of hardware to be executed regularly and(or)
+continuously.
+
diff --git a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4.rst index 84fc827241..c46f0141a2 100644 --- a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4.rst @@ -1,99 +1,100 @@ -IPv4 Routed-Forwarding
-======================
-
-This section provides a summary of VPP Phy-to-Phy IPv4 Routed-Forwarding
-performance illustrating packet latency measured at 50% of discovered NDR
-throughput rate. Latency is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_
- and
- `csit-vpp-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-VPP packet latency - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc
-
-VPP packet latency - running in configuration of **two worker threads (2t) on two
-physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc
-
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
-
+IPv4 Routed-Forwarding +====================== + +This section provides a summary of VPP Phy-to-Phy IPv4 Routed-Forwarding +performance illustrating packet latency measured at 50% of discovered NDR +throughput rate. Latency is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +VPP packet latency - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-ndrdisc-lat50.html"></iframe> + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale200k-ndrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale20k-ndrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale2m-ndrdisc + 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc + +VPP packet latency - running in configuration of **two worker threads (2t) on two +physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-ndrdisc-lat50.html"></iframe> + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-iacldstbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale200k-ndrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale20k-ndrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc + 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc + +VPP packet latency - running in configuration of **four worker threads (4t) on four +physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc-lat50.html"></iframe> + +*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc + 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4_tunnels.rst index 19f314f4a5..d3293cc6aa 100644 --- a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv4_tunnels.rst @@ -1,87 +1,88 @@ -IPv4 Overlay Tunnels
-====================
-
-This section provides a summary of VPP Phy-to-Phy IPv4 Overlay Tunnels
-performance illustrating packet latency measured at 50% of discovered NDR
-throughput rate. Latency is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_
- and
- `csit-vpp-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-VPP packet latency - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc
-
-VPP packet latency - running in configuration of **two worker threads (2t) on two
-physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc
-
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc
-
+IPv4 Overlay Tunnels +==================== + +This section provides a summary of VPP Phy-to-Phy IPv4 Overlay Tunnels +performance illustrating packet latency measured at 50% of discovered NDR +throughput rate. Latency is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +VPP packet latency - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ndrdisc-lat50.html"></iframe> + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip4-ip4base-ndrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip6-ip4base-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc + +VPP packet latency - running in configuration of **two worker threads (2t) on two +physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ndrdisc-lat50.html"></iframe> + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip4-ip4base-ndrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip6-ip4base-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc + +VPP packet latency - running in configuration of **four worker threads (4t) on four +physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc-lat50.html"></iframe> + +*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6.rst index b05239b6bb..38bcf71de6 100644 --- a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6.rst @@ -1,97 +1,98 @@ -IPv6 Routed-Forwarding
-======================
-
-This section provides a summary of VPP Phy-to-Phy IPv6 Routed-Forwarding
-performance illustrating packet latency measured at 50% of discovered NDR
-throughput rate. Latency is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_
- and
- `csit-vpp-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-VPP packet latency - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc
-
-VPP packet latency - running in configuration of **two worker threads (2t) on two
-physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ipolicemarkbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc
-
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
-
+IPv6 Routed-Forwarding +====================== + +This section provides a summary of VPP Phy-to-Phy IPv6 Routed-Forwarding +performance illustrating packet latency measured at 50% of discovered NDR +throughput rate. Latency is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +VPP packet latency - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-ndrdisc-lat50.html"></iframe> + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale200k-ndrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale20k-ndrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale2m-ndrdisc + 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc + +VPP packet latency - running in configuration of **two worker threads (2t) on two +physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-ndrdisc-lat50.html"></iframe> + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-iacldstbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ipolicemarkbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale200k-ndrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale20k-ndrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc + 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc + +VPP packet latency - running in configuration of **four worker threads (4t) on four +physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc-lat50.html"></iframe> + +*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc + 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6_tunnels.rst index 20c31588f7..94bf9a811f 100644 --- a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/ipv6_tunnels.rst @@ -1,82 +1,83 @@ -IPv6 Overlay Tunnels
-====================
-
-This section provides a summary of VPP Phy-to-Phy IPv6 Overlay Tunnels
-performance illustrating packet latency measured at 50% of discovered NDR
-throughput rate. Latency is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_
- and
- `csit-vpp-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-VPP packet latency - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-1t1c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip6-ip6base-ndrdisc
-
-VPP packet latency - running in configuration of **two worker threads (2t) on two
-physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-2t2c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc
-
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc
-
-
+IPv6 Overlay Tunnels +==================== + +This section provides a summary of VPP Phy-to-Phy IPv6 Overlay Tunnels +performance illustrating packet latency measured at 50% of discovered NDR +throughput rate. Latency is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +VPP packet latency - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ndrdisc-lat50.html"></iframe> + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-1t1c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip4-ip6base-ndrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip6-ip6base-ndrdisc + +VPP packet latency - running in configuration of **two worker threads (2t) on two +physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ndrdisc-lat50.html"></iframe> + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-2t2c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc + +VPP packet latency - running in configuration of **four worker threads (4t) on four +physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc-lat50.html"></iframe> + +*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc + + diff --git a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/l2.rst index 38d055c41d..a487306dad 100644 --- a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/l2.rst +++ b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/l2.rst @@ -1,102 +1,103 @@ -L2 Ethernet Switching
-=====================
-
-This section provides a summary of VPP Phy-to-Phy L2 Ethernet switching
-performance illustrating packet latency measured at 50% of discovered NDR
-throughput rate. Latency is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_
- and
- `csit-vpp-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-VPP packet latency - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-ndrdisc-lat50.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy L2 Ethernet Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc
-
-VPP packet latency - running in configuration of **two worker threads (2t) on two
-physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-ndrdisc-lat50.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy L2 Ethernet Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
-
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy L2 Ethernet Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
-
+L2 Ethernet Switching +===================== + +This section provides a summary of VPP Phy-to-Phy L2 Ethernet switching +performance illustrating packet latency measured at 50% of discovered NDR +throughput rate. Latency is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +VPP packet latency - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-ndrdisc-lat50.html"></iframe> + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy L2 Ethernet Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + +VPP packet latency - running in configuration of **two worker threads (2t) on two +physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-ndrdisc-lat50.html"></iframe> + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy L2 Ethernet Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc + +VPP packet latency - running in configuration of **four worker threads (4t) on four +physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc-lat50.html"></iframe> + +*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy L2 Ethernet Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/vm_vhost.rst index 9478dfcc48..6421db1c59 100644 --- a/docs/report/vpp_performance_tests_hw/packet_latency_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests_hw/packet_latency_graphs/vm_vhost.rst @@ -1,99 +1,100 @@ -VM vhost Connections
-====================
-
-This section provides a summary of VPP Phy-to-VM-to-Phy VM vhost-user
-performance illustrating packet latency measured at 50% of discovered NDR
-throughput rate. Latency is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_
- and
- `csit-vpp-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-VPP packet latency - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-ndrdisc-lat50.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - packet latency for Phy-to-VM-to-Phy VM vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
-VPP packet latency - running in configuration of **two worker threads (2t) on two
-physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-ndrdisc-lat50.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
+VM vhost Connections +==================== + +This section provides a summary of VPP Phy-to-VM-to-Phy VM vhost-user +performance illustrating packet latency measured at 50% of discovered NDR +throughput rate. Latency is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +VPP packet latency - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-ndrdisc-lat50.html"></iframe> + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-VM-to-Phy VM vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-.*vhost.*-ndrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + +VPP packet latency - running in configuration of **two worker threads (2t) on two +physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-ndrdisc-lat50.html"></iframe> + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-.*vhost.*-ndrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + +VPP packet latency - running in configuration of **four worker threads (4t) on four +physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc-lat50.html"></iframe> + +*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4.rst index 5bbedc9f7b..c413adb51e 100644 --- a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4.rst @@ -1,187 +1,187 @@ -IPv4 Routed-Forwarding
-======================
-
-Following sections provide a summary of VPP Phy-to-Phy IPv4 Routed-Forwarding
-performance illustrating NDR throughput (zero packet loss) and PDR throughput
-(<0.5% packet loss). Performance is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-NDR Throughput
-~~~~~~~~~~~~~~
-
-VPP NDR Throughput - running in configuration of **one worker thread (1t) on
-one physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-ndrdisc.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc
-
-VPP NDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-ndrdisc.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv4
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc
-
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
-
-PDR Throughput
-~~~~~~~~~~~~~~
-
-VPP PDR Throughput - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below. PDR at below 0.5%
-packet loss ratio.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-pdrdisc.html"></iframe>
-
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4scale200k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4scale20k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4scale2m-pdrdisc
-
-VPP PDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-pdrdisc.html"></iframe>
-
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-ipolicemarkbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale200k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale20k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale2m-pdrdisc
-
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale200k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale20k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale2m-pdrdisc
-
+IPv4 Routed-Forwarding +====================== + +Following sections provide a summary of VPP Phy-to-Phy IPv4 Routed-Forwarding +performance illustrating NDR throughput (zero packet loss) and PDR throughput +(<0.5% packet loss). Performance is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR Throughput - running in configuration of **one worker thread (1t) on +one physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-ndrdisc.html"></iframe> + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale200k-ndrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale20k-ndrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4scale2m-ndrdisc + 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-ndrdisc + +VPP NDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-ndrdisc.html"></iframe> + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv4 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-iacldstbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale200k-ndrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale20k-ndrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc + 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc + +VPP NDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc.html"></iframe> + +*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc + 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR Throughput - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. PDR at below 0.5% +packet loss ratio. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-pdrdisc.html"></iframe> + +*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-pdrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4scale200k-pdrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4scale20k-pdrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4scale2m-pdrdisc + +VPP PDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-pdrdisc.html"></iframe> + +*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-copwhtlistbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-iacldstbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-ipolicemarkbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-pdrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale200k-pdrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale20k-pdrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale2m-pdrdisc + +VPP PDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-pdrdisc.html"></iframe> + +*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" * + + 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc + 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-pdrdisc + 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale200k-pdrdisc + 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale20k-pdrdisc + 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale2m-pdrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4_tunnels.rst index 3f610c1ebf..23ac3fc001 100644 --- a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv4_tunnels.rst @@ -1,165 +1,165 @@ -IPv4 Overlay Tunnels
-====================
-
-Following sections provide a summary of VPP Phy-to-Phy IPv4 Overlay Tunnels
-performance illustrating NDR throughput (zero packet loss) and PDR throughput
-(<0.5% packet loss). Performance is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-NDR Throughput
-~~~~~~~~~~~~~~
-
-VPP NDR Throughput - running in configuration of **one worker thread (1t) on
-one physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ndrdisc.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv4 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc
-
-VPP NDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ndrdisc.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc
-
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc
-
-VPP PDR Throughput
-~~~~~~~~~~~~~~~~~~
-
-VPP PDR Throughput - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below. PDR at below 0.5%
-packet loss ratio.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-pdrdisc.html"></iframe>
-
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc02-64B-1t1c-ethip4lispip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc02-64B-1t1c-ethip4lispip6-ip4base-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2xcbase-pdrdisc
-
-VPP PDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-pdrdisc.html"></iframe>
-
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc08-64B-2t2c-ethip4lispip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc08-64B-2t2c-ethip4lispip6-ip4base-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2xcbase-pdrdisc
-
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip6-ip4base-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2xcbase-pdrdisc
-
+IPv4 Overlay Tunnels +==================== + +Following sections provide a summary of VPP Phy-to-Phy IPv4 Overlay Tunnels +performance illustrating NDR throughput (zero packet loss) and PDR throughput +(<0.5% packet loss). Performance is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR Throughput - running in configuration of **one worker thread (1t) on +one physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ndrdisc.html"></iframe> + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv4 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip4-ip4base-ndrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc01-64B-1t1c-ethip4lispip6-ip4base-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc + +VPP NDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ndrdisc.html"></iframe> + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip4-ip4base-ndrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4lispip6-ip4base-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc + +VPP NDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc.html"></iframe> + +*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR Throughput - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. PDR at below 0.5% +packet loss ratio. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-pdrdisc.html"></iframe> + +*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc02-64B-1t1c-ethip4lispip4-ip4base-pdrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc02-64B-1t1c-ethip4lispip6-ip4base-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2xcbase-pdrdisc + +VPP PDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-pdrdisc.html"></iframe> + +*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc08-64B-2t2c-ethip4lispip4-ip4base-pdrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc08-64B-2t2c-ethip4lispip6-ip4base-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2xcbase-pdrdisc + +VPP PDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-pdrdisc.html"></iframe> + +*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" * + + 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip4-ip4base-pdrdisc + 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip6-ip4base-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2xcbase-pdrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6.rst index 5b80205578..e04f2c6822 100644 --- a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6.rst @@ -1,183 +1,183 @@ -IPv6 Routed-Forwarding
-======================
-
-Following sections provide a summary of VPP Phy-to-Phy IPv6 Routed-Forwarding
-performance illustrating NDR throughput (zero packet loss) and PDR throughput
-(<0.5% packet loss). Performance is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-NDR Throughput
-~~~~~~~~~~~~~~
-
-VPP NDR Throughput - running in configuration of **one worker thread (1t) on
-one physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-ndrdisc.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc
-
-VPP NDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-ndrdisc.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ipolicemarkbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc
-
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
-
-PDR Throughput
-~~~~~~~~~~~~~~
-
-VPP PDR Throughput - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below. PDR at below 0.5%
-packet loss ratio.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-pdrdisc.html"></iframe>
-
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6base-pdrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6scale200k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6scale20k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6scale2m-pdrdisc
-
-VPP PDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-pdrdisc.html"></iframe>
-
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6base-pdrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale200k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale20k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale2m-pdrdisc
-
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-pdrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale200k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale20k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale2m-pdrdisc
-
-
+IPv6 Routed-Forwarding +====================== + +Following sections provide a summary of VPP Phy-to-Phy IPv6 Routed-Forwarding +performance illustrating NDR throughput (zero packet loss) and PDR throughput +(<0.5% packet loss). Performance is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR Throughput - running in configuration of **one worker thread (1t) on +one physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-ndrdisc.html"></iframe> + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale200k-ndrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale20k-ndrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6scale2m-ndrdisc + 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6-ip6base-ndrdisc + +VPP NDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-ndrdisc.html"></iframe> + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-iacldstbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ipolicemarkbase-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ipolicemarkbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale200k-ndrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale20k-ndrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc + 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc + +VPP NDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc.html"></iframe> + +*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc + 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR Throughput - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. PDR at below 0.5% +packet loss ratio. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-pdrdisc.html"></iframe> + +*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6base-pdrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6scale200k-pdrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6scale20k-pdrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc02-78B-1t1c-ethip6-ip6scale2m-pdrdisc + +VPP PDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-pdrdisc.html"></iframe> + +*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6base-copwhtlistbase-pdrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6base-iacldstbase-pdrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6base-pdrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale200k-pdrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale20k-pdrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale2m-pdrdisc + +VPP PDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-pdrdisc.html"></iframe> + +*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" * + + 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-copwhtlistbase-pdrdisc + 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-iacldstbase-pdrdisc + 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-pdrdisc + 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale200k-pdrdisc + 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale20k-pdrdisc + 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale2m-pdrdisc + + diff --git a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6_tunnels.rst index aa66777de3..1a57998018 100644 --- a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/ipv6_tunnels.rst @@ -1,155 +1,155 @@ -IPv6 Overlay Tunnels
-====================
-
-Following sections provide a summary of VPP Phy-to-Phy IPv6 Overlay Tunnels
-performance illustrating NDR throughput (zero packet loss) and PDR throughput
-(<0.5% packet loss). Performance is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-NDR Throughput
-~~~~~~~~~~~~~~
-
-VPP NDR Throughput - running in configuration of **one worker thread (1t) on
-one physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ndrdisc.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-1t1c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip6-ip6base-ndrdisc
-
-VPP NDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ndrdisc.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-2t2c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc
-
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc
-
-PDR Throughput
-~~~~~~~~~~~~~~
-
-VPP PDR Throughput - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below. PDR at below 0.5%
-packet loss ratio.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-pdrdisc.html"></iframe>
-
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-1t1c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc02-78B-1t1c-ethip6lispip4-ip6base-pdrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc02-78B-1t1c-ethip6lispip6-ip6base-pdrdisc
-
-VPP PDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-pdrdisc.html"></iframe>
-
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-2t2c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip4-ip6base-pdrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip6-ip6base-pdrdisc
-
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip4-ip6base-pdrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip6-ip6base-pdrdisc
-
+IPv6 Overlay Tunnels +==================== + +Following sections provide a summary of VPP Phy-to-Phy IPv6 Overlay Tunnels +performance illustrating NDR throughput (zero packet loss) and PDR throughput +(<0.5% packet loss). Performance is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR Throughput - running in configuration of **one worker thread (1t) on +one physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ndrdisc.html"></iframe> + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy IPv6 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-1t1c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip4-ip6base-ndrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc01-78B-1t1c-ethip6lispip6-ip6base-ndrdisc + +VPP NDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ndrdisc.html"></iframe> + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy IPv6 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-2t2c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc + +VPP NDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc.html"></iframe> + +*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR Throughput - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. PDR at below 0.5% +packet loss ratio. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-pdrdisc.html"></iframe> + +*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-1t1c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc02-78B-1t1c-ethip6lispip4-ip6base-pdrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc02-78B-1t1c-ethip6lispip6-ip6base-pdrdisc + +VPP PDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-pdrdisc.html"></iframe> + +*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-2t2c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip4-ip6base-pdrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip6-ip6base-pdrdisc + +VPP PDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-pdrdisc.html"></iframe> + +*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 Overlay +Tunnels.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" * + + 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip4-ip6base-pdrdisc + 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip6-ip6base-pdrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/l2.rst index e3543f8bd0..89429a6192 100644 --- a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/l2.rst +++ b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/l2.rst @@ -1,190 +1,190 @@ -L2 Ethernet Switching
-=====================
-
-Following sections provide a summary of VPP Phy-to-Phy L2 Ethernet switching
-performance illustrating NDR throughput (zero packet loss) and PDR throughput
-(<0.5% packet loss). Performance is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-NDR Throughput
-~~~~~~~~~~~~~~
-
-VPP NDR Throughput - running in configuration of **one worker thread (1t) on
-one physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-ndrdisc.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc
-
-VPP NDR Throughput - running in configuration of **two worker threads (2t) on two
-physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-ndrdisc.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
-
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
-
-PDR Throughput
-~~~~~~~~~~~~~~
-
-VPP PDR Throughput - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below. PDR at below 0.5% packet
-loss ratio.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-pdrdisc.html"></iframe>
-
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-dot1ad-l2xcbase-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-dot1q-l2xcbase-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc
-
-VPP PDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-pdrdisc.html"></iframe>
-
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-dot1ad-l2xcbase-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-dot1q-l2xcbase-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc
-
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1ad-l2xcbase-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2xcbase-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc
-
+L2 Ethernet Switching +===================== + +Following sections provide a summary of VPP Phy-to-Phy L2 Ethernet switching +performance illustrating NDR throughput (zero packet loss) and PDR throughput +(<0.5% packet loss). Performance is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR Throughput - running in configuration of **one worker thread (1t) on +one physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-ndrdisc.html"></iframe> + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + +VPP NDR Throughput - running in configuration of **two worker threads (2t) on two +physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-ndrdisc.html"></iframe> + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc + +VPP NDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc.html"></iframe> + +*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR Throughput - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. PDR at below 0.5% packet +loss ratio. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-pdrdisc.html"></iframe> + +*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet +Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-dot1ad-l2xcbase-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-dot1q-l2xcbase-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + +VPP PDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-pdrdisc.html"></iframe> + +*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet +Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-dot1ad-l2xcbase-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-dot1q-l2xcbase-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc + +VPP PDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-pdrdisc.html"></iframe> + +*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet +Switching.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * + + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1ad-l2xcbase-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2xcbase-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc + diff --git a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/vm_vhost.rst index 4f43715dce..b5bc0dd5a3 100644 --- a/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests_hw/packet_throughput_graphs/vm_vhost.rst @@ -1,191 +1,191 @@ -VM vhost Connections
-====================
-
-Following sections provide a summary of VPP Phy-to-VM-to-Phy VM vhost-user
-performance illustrating NDR throughput (zero packet loss) and PDR throughput
-(<0.5% packet loss). Performance is reported for VPP running in multiple
-configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and
-their physical CPU core(s) placement.
-
-Title of each graph is a regex (regular expression) matching all plotted
-test case throughput measurements.
-
-.. note::
-
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-vpp-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
-
-NDR Throughput
-~~~~~~~~~~~~~~
-
-VPP NDR Throughput - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-ndrdisc.html"></iframe>
-
-*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
-VPP NDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-ndrdisc.html"></iframe>
-
-*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
-PDR Throughput
-~~~~~~~~~~~~~~
-
-VPP PDR Throughput - running in configuration of **one worker thread (1t) on one
-physical core (1c)** - is presented in the figure below. PDR at below 0.5%
-packet loss ratio.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-pdrdisc.html"></iframe>
-
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-1t1c-.*vhost.*-pdrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
-
-VPP PDR Throughput - running in configuration of **two worker threads (2t) on
-two physical cores (2c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-pdrdisc.html"></iframe>
-
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-2t2c-.*vhost.*-pdrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
-
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-.*vhost.*-pdrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
-
+VM vhost Connections +==================== + +Following sections provide a summary of VPP Phy-to-VM-to-Phy VM vhost-user +performance illustrating NDR throughput (zero packet loss) and PDR throughput +(<0.5% packet loss). Performance is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and +their physical CPU core(s) placement. + +Title of each graph is a regex (regular expression) matching all plotted +test case throughput measurements. + +.. note:: + + Data sources for reported test results: i) FD.io test executor jobs + `csit-vpp-perf-1701-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and + `csit-vpp-perf-1701-long + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ + , ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR Throughput - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-ndrdisc.html"></iframe> + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-VM-to-Phy VM +vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-.*vhost.*-ndrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + +VPP NDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-ndrdisc.html"></iframe> + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-VM-to-Phy VM +vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-.*vhost.*-ndrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + +VPP NDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc.html"></iframe> + +*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-VM-to-Phy VM +vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR Throughput - running in configuration of **one worker thread (1t) on one +physical core (1c)** - is presented in the figure below. PDR at below 0.5% +packet loss ratio. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-pdrdisc.html"></iframe> + +*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM +vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-1t1c-.*vhost.*-pdrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + +VPP PDR Throughput - running in configuration of **two worker threads (2t) on +two physical cores (2c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-pdrdisc.html"></iframe> + +*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM +vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-2t2c-.*vhost.*-pdrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + +VPP PDR Throughput - running in configuration of **four worker threads (4t) on +four physical cores (4c)** - is presented in the figure below. + +.. raw:: html + + <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-pdrdisc.html"></iframe> + +*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-VM-to-Phy VM +vhost-user.* + +CSIT test cases used to generate results presented above can be found in CSIT +git repository by filtering with specified regex as follows: + +.. code-block:: bash + + $ csit/tests/perf + $ grep -E "64B-4t4c-.*vhost.*-pdrdisc" * + + 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc + diff --git a/resources/tools/report_gen/conf.py b/resources/tools/report_gen/conf.py index e6fb23cc66..12944e47c4 100644 --- a/resources/tools/report_gen/conf.py +++ b/resources/tools/report_gen/conf.py @@ -44,9 +44,9 @@ source_suffix = ['.rst', '.md'] master_doc = 'index' # General information about the project. -project = u'CSIT' -copyright = u'2017, fd.io' -author = u'CSIT' +project = u'FD.io CSIT' +copyright = u'2017, FD.io' +author = u'FD.io CSIT' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -158,6 +158,3 @@ texinfo_documents = [ author, 'CSIT1701report', 'One line description of project.', 'Miscellaneous'), ] - - - diff --git a/resources/tools/report_gen/run_report.sh b/resources/tools/report_gen/run_report.sh index bbb048619e..2a88750464 100755 --- a/resources/tools/report_gen/run_report.sh +++ b/resources/tools/report_gen/run_report.sh @@ -6,7 +6,8 @@ SOURCE_DIR='../../../docs/report' STATIC_DIR="${BUILD_DIR}/_static" STATIC_DIR_VPP="${STATIC_DIR}/vpp" STATIC_DIR_TESTPMD="${STATIC_DIR}/testpmd" -CSS_PATCH_FILE="${BUILD_DIR}/_static/theme_overrides.css" +STATIC_DIR_ARCH="${STATIC_DIR}/archive" +CSS_PATCH_FILE="${STATIC_DIR}/theme_overrides.css" sudo apt-get install -y libxml2 libxml2-dev libxslt-dev build-essential zlib1g-dev @@ -31,7 +32,10 @@ pip install -r requirements.txt export PYTHONPATH=`pwd` # Generate the documentation: -sphinx-build -v -c . -a -b html -E -D release=$1 -D version=$1 ${SOURCE_DIR} ${BUILD_DIR}/ + +DATE=$(date -u '+%d-%b-%Y') + +sphinx-build -v -c . -a -b html -E -D release=$1 -D version="$1 report - $DATE" ${SOURCE_DIR} ${BUILD_DIR}/ # Patch the CSS for tables layout cat - > ${CSS_PATCH_FILE} <<"_EOF" @@ -54,34 +58,51 @@ _EOF echo Downloading raw outputs for plots ... mkdir -p ${STATIC_DIR_VPP} mkdir -p ${STATIC_DIR_TESTPMD} +mkdir -p ${STATIC_DIR_ARCH} + +JEN_URL='https://jenkins.fd.io/view/csit/job' +JEN_FILE_PERF='output_perf_data.xml' + +JEN_JOB='csit-vpp-perf-1701-all' +JEN_BUILD=(3 4 7) + +for i in "${JEN_BUILD[@]}"; do + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/${JEN_FILE_PERF} -O ${STATIC_DIR_VPP}/${JEN_JOB}-${i}.xml + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/\*zip\*/archive.zip -O ${STATIC_DIR_ARCH}/${JEN_JOB}-${i}.zip +done + +JEN_JOB='csit-vpp-perf-1701-long' +JEN_BUILD=(2 4) -JENKINS_URL='https://jenkins.fd.io/view/csit/job/' -JENKINS_DIR='/artifact/' +for i in "${JEN_BUILD[@]}"; do + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/${JEN_FILE_PERF} -O ${STATIC_DIR_VPP}/${JEN_JOB}-${i}.xml + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/\*zip\*/archive.zip -O ${STATIC_DIR_ARCH}/${JEN_JOB}-${i}.zip +done -PERF_JENKINS_JOB='csit-vpp-perf-1701-all' -PERF_JENKINS_BUILD=(3 4 7) -PERF_JENKINS_FILE='output_perf_data.xml' +JEN_JOB='csit-dpdk-perf-1701-all' +JEN_BUILD=(2 3) -for i in "${PERF_JENKINS_BUILD[@]}"; do - wget -q ${JENKINS_URL}${PERF_JENKINS_JOB}/${i}${JENKINS_DIR}${PERF_JENKINS_FILE} -O ${STATIC_DIR_VPP}/${PERF_JENKINS_JOB}-${i}.xml +for i in "${JEN_BUILD[@]}"; do + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/${JEN_FILE_PERF} -O ${STATIC_DIR_TESTPMD}/${JEN_JOB}-${i}.xml + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/\*zip\*/archive.zip -O ${STATIC_DIR_ARCH}/${JEN_JOB}-${i}.zip done -PERF_JENKINS_JOB='csit-vpp-perf-1701-long' -PERF_JENKINS_BUILD=(2 4) -PERF_JENKINS_FILE='output_perf_data.xml' +JEN_JOB='csit-vpp-functional-1701-virl' +JEN_BUILD=(18) -for i in "${PERF_JENKINS_BUILD[@]}"; do - wget -q ${JENKINS_URL}${PERF_JENKINS_JOB}/${i}${JENKINS_DIR}${PERF_JENKINS_FILE} -O ${STATIC_DIR_VPP}/${PERF_JENKINS_JOB}-${i}.xml +for i in "${JEN_BUILD[@]}"; do + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/\*zip\*/archive.zip -O ${STATIC_DIR_ARCH}/${JEN_JOB}-${i}.zip done -PERF_JENKINS_JOB='csit-dpdk-perf-1701-all' -PERF_JENKINS_BUILD=(2 3) -PERF_JENKINS_FILE='output_perf_data.xml' +JEN_URL='https://jenkins.fd.io/view/hc2vpp/job' +JEN_JOB='hc2vpp-csit-integration-1701-ubuntu1404' +JEN_BUILD=(1) -for i in "${PERF_JENKINS_BUILD[@]}"; do - wget -q ${JENKINS_URL}${PERF_JENKINS_JOB}/${i}${JENKINS_DIR}${PERF_JENKINS_FILE} -O ${STATIC_DIR_TESTPMD}/${PERF_JENKINS_JOB}-${i}.xml +for i in "${JEN_BUILD[@]}"; do + wget -q ${JEN_URL}/${JEN_JOB}/${i}/artifact/\*zip\*/archive.zip -O ${STATIC_DIR_ARCH}/${JEN_JOB}-${i}.zip done + # Plot packets per second python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-1t1c-l2-ndrdisc --title "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" --xpath '//*[@framesize="64B" and contains(@tags,"BASE") and contains(@tags,"NDRDISC") and contains(@tags,"1T1C") and (contains(@tags,"L2BDMACSTAT") or contains(@tags,"L2BDMACLRN") or contains(@tags,"L2XCFWD")) and not(contains(@tags,"VHOST"))]' --lower 0 --upper 16000000 @@ -112,13 +133,13 @@ python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-1t1c python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-2t2c-l2-pdrdisc --title "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" --xpath '//*[@framesize="64B" and contains(@tags,"BASE") and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"2T2C") and (contains(@tags,"L2BDMACSTAT") or contains(@tags,"L2BDMACLRN") or contains(@tags,"L2XCFWD")) and not(contains(@tags,"VHOST"))]' --lower 0 --upper 26000000 python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-4t4c-l2-pdrdisc --title "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" --xpath '//*[@framesize="64B" and contains(@tags,"BASE") and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"4T4C") and (contains(@tags,"L2BDMACSTAT") or contains(@tags,"L2BDMACLRN") or contains(@tags,"L2XCFWD")) and not(contains(@tags,"VHOST"))]' --lower 0 --upper 36000000 -python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-1t1c-ethip4-ip4-pdrdisc --title "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="64B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") and not(contains(@tags,"NDRDISC")) or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and contains(@tags,"1T1C") and contains(@tags,"IP4FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 16000000 -python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-2t2c-ethip4-ip4-pdrdisc --title "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="64B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") and not(contains(@tags,"NDRDISC")) or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and contains(@tags,"2T2C") and contains(@tags,"IP4FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 26000000 -python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-4t4c-ethip4-ip4-pdrdisc --title "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="64B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") and not(contains(@tags,"NDRDISC")) or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and contains(@tags,"4T4C") and contains(@tags,"IP4FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 36000000 +python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-1t1c-ethip4-ip4-pdrdisc --title "64B-1t1c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="64B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"1T1C") and contains(@tags,"IP4FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 16000000 +python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-2t2c-ethip4-ip4-pdrdisc --title "64B-2t2c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="64B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"2T2C") and contains(@tags,"IP4FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 26000000 +python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-4t4c-ethip4-ip4-pdrdisc --title "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="64B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"4T4C") and contains(@tags,"IP4FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 36000000 -python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/78B-1t1c-ethip6-ip6-pdrdisc --title "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="78B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") and not(contains(@tags,"NDRDISC")) or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and contains(@tags,"1T1C") and contains(@tags,"IP6FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 16000000 -python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/78B-2t2c-ethip6-ip6-pdrdisc --title "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="78B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") and not(contains(@tags,"NDRDISC")) or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and contains(@tags,"2T2C") and contains(@tags,"IP6FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 26000000 -python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/78B-4t4c-ethip6-ip6-pdrdisc --title "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="78B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") and not(contains(@tags,"NDRDISC")) or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and contains(@tags,"4T4C") and contains(@tags,"IP6FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 36000000 +python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/78B-1t1c-ethip6-ip6-pdrdisc --title "78B-1t1c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="78B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"1T1C") and contains(@tags,"IP6FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 16000000 +python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/78B-2t2c-ethip6-ip6-pdrdisc --title "78B-2t2c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="78B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"2T2C") and contains(@tags,"IP6FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 26000000 +python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/78B-4t4c-ethip6-ip6-pdrdisc --title "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" --xpath '//*[@framesize="78B" and (contains(@tags,"BASE") or contains(@tags,"SCALE") or contains(@tags,"FEATURE")) and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"4T4C") and contains(@tags,"IP6FWD") and not(contains(@tags,"VHOST"))]' --lower 0 --upper 36000000 python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-1t1c-ethip4-pdrdisc --title "64B-1t1c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" --xpath '//*[@framesize="64B" and contains(@tags,"ENCAP") and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"1T1C") and (contains(@tags,"VXLAN") or contains(@tags,"VXLANGPE") or contains(@tags,"LISP") or contains(@tags,"LISPGPE") or contains(@tags,"GRE")) and not(contains(@tags,"VHOST"))]' --lower 0 --upper 16000000 python run_plot.py --input ${STATIC_DIR_VPP} --output ${STATIC_DIR_VPP}/64B-2t2c-ethip4-pdrdisc --title "64B-2t2c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" --xpath '//*[@framesize="64B" and contains(@tags,"ENCAP") and contains(@tags,"PDRDISC") and not(contains(@tags,"NDRDISC")) and contains(@tags,"2T2C") and (contains(@tags,"VXLAN") or contains(@tags,"VXLANGPE") or contains(@tags,"LISP") or contains(@tags,"LISPGPE") or contains(@tags,"GRE")) and not(contains(@tags,"VHOST"))]' --lower 0 --upper 16000000 @@ -170,41 +191,6 @@ python run_plot.py --input ${STATIC_DIR_TESTPMD} --output ${STATIC_DIR_TESTPMD}/ python run_plot.py --input ${STATIC_DIR_TESTPMD} --output ${STATIC_DIR_TESTPMD}/64B-2t2c-l2-ndrdisc-lat50 --title "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" --xpath '//*[@framesize="64B" and contains(@tags,"BASE") and contains(@tags,"NDRDISC") and contains(@tags,"2T2C") and (contains(@tags,"L2BDMACSTAT") or contains(@tags,"L2BDMACLRN") or contains(@tags,"L2XCFWD")) and not(contains(@tags,"VHOST"))]' --latency lat_50 python run_plot.py --input ${STATIC_DIR_TESTPMD} --output ${STATIC_DIR_TESTPMD}/64B-4t4c-l2-ndrdisc-lat50 --title "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" --xpath '//*[@framesize="64B" and contains(@tags,"BASE") and contains(@tags,"NDRDISC") and contains(@tags,"4T4C") and (contains(@tags,"L2BDMACSTAT") or contains(@tags,"L2BDMACLRN") or contains(@tags,"L2XCFWD")) and not(contains(@tags,"VHOST"))]' --latency lat_50 -# Download raw outputs for archive - -#echo Downloading raw outputs for archive ... -#JENKINS_URL='https://jenkins.fd.io/view/csit/job/' -#JENKINS_DIR='/artifact/*zip*/' -#JENKINS_FILE='archive.zip' - -#PERF_JENKINS_JOB='csit-vpp-perf-1701-all' -#PERF_JENKINS_BUILD=(3 4 7) - -#for i in "${PERF_JENKINS_BUILD[@]}"; do -# wget -q ${JENKINS_URL}${PERF_JENKINS_JOB}/${i}${JENKINS_DIR}${JENKINS_FILE} -O ${STATIC_DIR}/${PERF_JENKINS_JOB}-${i}.zip -#done - -#PERF_JENKINS_JOB='csit-vpp-perf-1701-long' -#PERF_JENKINS_BUILD=(2 4) - -#for i in "${PERF_JENKINS_BUILD[@]}"; do -# wget -q ${JENKINS_URL}${PERF_JENKINS_JOB}/${i}${JENKINS_DIR}${JENKINS_FILE} -O ${STATIC_DIR}/${PERF_JENKINS_JOB}-${i}.zip -#done - -#FUNC_JENKINS_JOB='csit-vpp-functional-1701-virl' -#FUNC_JENKINS_BUILD=(18) - -#for i in "${FUNC_JENKINS_BUILD[@]}"; do -# wget -q ${JENKINS_URL}${FUNC_JENKINS_JOB}/${i}${JENKINS_DIR}${JENKINS_FILE} -O ${STATIC_DIR}/${FUNC_JENKINS_JOB}-${i}.zip -#done - -#PERF_JENKINS_JOB='csit-dpdk-perf-1701-all' -#PERF_JENKINS_BUILD=(2 3) - -#for i in "${PERF_JENKINS_BUILD[@]}"; do -# wget -q ${JENKINS_URL}${PERF_JENKINS_JOB}/${i}${JENKINS_DIR}${JENKINS_FILE} -O ${STATIC_DIR}/${PERF_JENKINS_JOB}-${i}.zip -#done - # Create archive echo Creating csit.report.tar.gz ... tar -czvf ./csit.report.tar.gz ${BUILD_DIR} diff --git a/resources/tools/robot_output_parser_publish.py b/resources/tools/report_gen/run_robot_data.py index e2bcfa2f0d..e2bcfa2f0d 100755 --- a/resources/tools/robot_output_parser_publish.py +++ b/resources/tools/report_gen/run_robot_data.py |