diff options
Diffstat (limited to 'docs')
50 files changed, 913 insertions, 2863 deletions
diff --git a/docs/report/detailed_test_results/index.rst b/docs/report/detailed_test_results/index.rst index d980a3532b..923f6c6f76 100644 --- a/docs/report/detailed_test_results/index.rst +++ b/docs/report/detailed_test_results/index.rst @@ -7,10 +7,9 @@ .. toctree:: vpp_performance_results/index + vpp_mrr_results/index cot_performance_results/index - dpdk_performance_results/index vpp_functional_results/index + vpp_functional_results_centos/index honeycomb_functional_results/index nshsfc_functional_results/index - vpp_unittest_results/index - vpp_http_server_perf_results/index diff --git a/docs/report/detailed_test_results/vpp_functional_results/index.rst b/docs/report/detailed_test_results/vpp_functional_results/index.rst index 4d7060bb54..5b82c489c4 100644 --- a/docs/report/detailed_test_results/vpp_functional_results/index.rst +++ b/docs/report/detailed_test_results/vpp_functional_results/index.rst @@ -1,5 +1,5 @@ -VPP Functional Results -====================== +VPP Functional Results - Ubuntu +=============================== .. note:: diff --git a/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst b/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst new file mode 100644 index 0000000000..e65b68b250 --- /dev/null +++ b/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst @@ -0,0 +1,12 @@ +VPP Functional Results - CentOS +=============================== + +.. note:: + + Data sources for reported test results: i) `FD.io test executor vpp + functional jobs`_, ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + + vpp_functional_results_centos diff --git a/docs/report/detailed_test_results/dpdk_performance_results/index.rst b/docs/report/detailed_test_results/vpp_mrr_results/index.rst index e6466d2e60..ce8262ba7c 100644 --- a/docs/report/detailed_test_results/dpdk_performance_results/index.rst +++ b/docs/report/detailed_test_results/vpp_mrr_results/index.rst @@ -1,12 +1,12 @@ -DPDK Performance Results -=========================== +VPP MRR Results +=============== .. note:: - Data sources for reported test results: i) `FD.io test executor dpdk + Data sources for reported test results: i) `FD.io test executor vpp performance jobs`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. .. toctree:: - dpdk_performance_results + vpp_mrr_results diff --git a/docs/report/detailed_test_results/vpp_unittest_results/index.rst b/docs/report/detailed_test_results/vpp_unittest_results/index.rst deleted file mode 100644 index f77153d6b9..0000000000 --- a/docs/report/detailed_test_results/vpp_unittest_results/index.rst +++ /dev/null @@ -1,7 +0,0 @@ -VPP Unit Test Results
-=====================
-
-.. toctree::
-
- vpp_unittest_results
-
diff --git a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst deleted file mode 100644 index 3e1fd5ac57..0000000000 --- a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst +++ /dev/null @@ -1,1178 +0,0 @@ -ACL Security Groups
-```````````````````
-::
-
- ==============================================================================
- ACL plugin Test Case
- ==============================================================================
- ACL plugin version check; learn MACs OK
- ACL create/delete test OK
- permit ACL apply test OK
- deny ACL apply test OK
- VPP_624 permit ICMPv4 OK
- VPP_624 permit ICMPv6 OK
- VPP_624 deny ICMPv4 OK
- VPP_624 deny ICMPv6 OK
- permit TCPv4 OK
- permit TCPv6 OK
- permit UDPv4 OK
- permit UDPv6 OK
- deny TCPv4/v6 OK
- deny UDPv4/v6 OK
- verify add/dump acls OK
- permit single TCPv4 OK
- permit single UDPv4 OK
- permit single TCPv6 OK
- permit single UPPv6 OK
- deny single TCPv4/v6 OK
- deny single UDPv4/v6 OK
- deny single UDPv4/v6, permit ip any, verify non-initial fragment blocked OK
- VPP-687 zero length udp ipv4 packet OK
- VPP-687 zero length udp ipv6 packet OK
- permit TCPv4 + non-match range OK
- permit TCPv6 + non-match range OK
- permit UDPv4 + non-match range OK
- permit UDPv6 + non-match range OK
- deny TCPv4/v6 + non-match range OK
- deny UDPv4/v6 + non-match range OK
-
- ==============================================================================
- IRB Test Case
- ==============================================================================
- ACL plugin prepare OK
- ACL IPv6 routed -> bridged, L2 ACL deny OK
- ACL IPv6 routed -> bridged, L3 ACL deny OK
- ACL IPv4 routed -> bridged, L2 ACL deny OK
- ACL IPv4 routed -> bridged, L3 ACL deny OK
- ACL IPv6 bridged -> routed, L2 ACL deny OK
- ACL IPv6 bridged -> routed, L3 ACL deny OK
- ACL IPv4 bridged -> routed, L2 ACL deny OK
- ACL IPv4 bridged -> routed, L3 ACL deny OK
- ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK
- ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK
- ACL IPv6+EH routed -> bridged, L2 ACL deny OK
- ACL IPv6+EH routed -> bridged, L3 ACL deny OK
- ACL IPv6+EH bridged -> routed, L2 ACL deny OK
- ACL IPv6+EH bridged -> routed, L3 ACL deny OK
- ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK
- ACL IPv4+MF routed -> bridged, L2 ACL deny OK
- ACL IPv4+MF routed -> bridged, L3 ACL deny OK
- ACL IPv4+MF bridged -> routed, L2 ACL deny OK
- ACL IPv4+MF bridged -> routed, L3 ACL deny OK
- ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK
-
- ==============================================================================
- ACL plugin connection-oriented extended testcases
- ==============================================================================
- Prepare the settings SKIP
- IPv4: Basic conn timeout test reflect on ingress SKIP
- IPv4: Basic conn timeout test reflect on egress SKIP
- IPv4: reflect egress, clear conn SKIP
- IPv4: reflect ingress, clear conn SKIP
- IPv4: Idle conn behind active conn, reflect on ingress SKIP
- IPv4: Idle conn behind active conn, reflect on egress SKIP
- IPv6: Basic conn timeout test reflect on ingress SKIP
- IPv6: Basic conn timeout test reflect on egress SKIP
- IPv6: reflect egress, clear conn SKIP
- IPv6: reflect ingress, clear conn SKIP
- IPv6: Idle conn behind active conn, reflect on ingress SKIP
- IPv6: Idle conn behind active conn, reflect on egress SKIP
- Prepare for TCP session tests SKIP
- IPv4: transient TCP session (incomplete 3WHS), ref. on ingress SKIP
- IPv4: transient TCP session (incomplete 3WHS), ref. on egress SKIP
- IPv4: established TCP session (complete 3WHS), ref. on ingress SKIP
- IPv4: established TCP session (complete 3WHS), ref. on egress SKIP
- IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP
- IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP
- IPv6: transient TCP session (incomplete 3WHS), ref. on ingress SKIP
- IPv6: transient TCP session (incomplete 3WHS), ref. on egress SKIP
- IPv6: established TCP session (complete 3WHS), ref. on ingress SKIP
- IPv6: established TCP session (complete 3WHS), ref. on egress SKIP
- IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP
- IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP
-
- ==============================================================================
- ACL on dot1q bridged subinterfaces Tests
- ==============================================================================
- IP4 ACL SubIf Dot1Q bridged traffic OK
- IP6 ACL SubIf Dot1Q bridged traffic OK
-
- ==============================================================================
- ACL on dot1ad bridged subinterfaces Tests
- ==============================================================================
- IP4 ACL SubIf Dot1AD bridged traffic OK
- IP6 ACL SubIf Dot1AD bridged traffic OK
-
- ==============================================================================
- ACL on dot1ad routed subinterfaces Tests
- ==============================================================================
- IP4 ACL SubIf Dot1AD routed traffic OK
- IP4 ACL SubIf wrong tags Dot1AD routed traffic OK
- IP6 ACL SubIf Dot1AD routed traffic OK
- IP6 ACL SubIf wrong tags Dot1AD routed traffic OK
-
- ==============================================================================
- ACL on dot1q routed subinterfaces Tests
- ==============================================================================
- IP4 ACL SubIf Dot1Q routed traffic OK
- IP4 ACL SubIf wrong tags Dot1Q routed traffic OK
- IP6 ACL SubIf Dot1Q routed traffic OK
- IP6 ACL SubIf wrong tags Dot1Q routed traffic OK
-
-APIs
-````
-::
-
- ==============================================================================
- VAPI test
- ==============================================================================
- run C VAPI tests SKIP
- run C++ VAPI tests SKIP
-
- ==============================================================================
- VPP Object Model Test
- ==============================================================================
- run C++ VOM tests SKIP
-
- ==============================================================================
- PAPI Test Case
- ==============================================================================
- show version OK
- show version - invalid parameters OK
- u8 array OK
-
- ==============================================================================
- PAPI Message parsing Test Case
- ==============================================================================
- New compound type with array OK
- Add new types OK
- Add new types 2 OK
- Add new message object OK
- New message with array OK
- Argument name OK
- VLA with aribtrary length field placement OK
- Message to byte encoding OK
- Nested array type OK
- Old style VLA array OK
- Old VLA compound type OK
- Old VLA array arbitrary placement OK
- Old VLA u32 OK
- Simple array OK
-
- ==============================================================================
- JVPP Core Test Case
- ==============================================================================
- JVPP Acl Callback Api Test Case OK
- JVPP Acl Future Api Test Case OK
- JVPP Core Callback Api Test Case OK
- JVPP Core Future Api Test Case OK
- JVPP Ioamexport Callback Api Test Case OK
- JVPP Ioamexport Future Api Test Case OK
- JVPP Ioampot Callback Api Test Case OK
- JVPP Ioampot Future Api Test Case OK
- JVPP Ioamtrace Callback Api Test Case OK
- JVPP Ioamtrace Future Api Test Case OK
- JVPP Snat Callback Api Test Case OK
- JVPP Snat Future Api Test Case OK
-
-ARP
-```
-::
-
- ==============================================================================
- ARP Test Case
- ==============================================================================
- ARP OK
- ARP Duplicates OK
- ARP Static OK
- ARP reply with VRRP virtual src hw addr OK
- MPLS OK
- Proxy ARP OK
- Interface Mirror Proxy ARP OK
-
- ==============================================================================
- L2BD arp termination Test Case
- ==============================================================================
- L2BD arp term - add 5 hosts, verify arp responses OK
- L2BD arp term - delete 3 hosts, verify arp responses OK
- L2BD arp term - recreate BD1, readd 3 hosts, verify arp responses OK
- L2BD arp term - 2 IP4 addrs per host OK
- L2BD arp term - create and update 10 IP4-mac pairs OK
- L2BD arp/ND term - hosts with both ip4/ip6 OK
- L2BD ND term - Add and Del hosts, verify ND replies OK
- L2BD ND term - Add and update IP+mac, verify ND replies OK
- L2BD arp term - send garps, verify arp event reports OK
- L2BD arp term - send duplicate garps, verify suppression OK
- L2BD arp term - disable ip4 arp events,send garps, verify no events OK
- L2BD ND term - send NS packets verify reports OK
- L2BD ND term - send duplicate ns, verify suppression OK
- L2BD ND term - disable ip4 arp events,send ns, verify no events OK
-
-BFD API
-````````
-::
-
- ==============================================================================
- Bidirectional Forwarding Detection (BFD) - API
- ==============================================================================
- activate SHA1 authentication SKIP
- create BFD session using non-existent SHA1 (negative case) SKIP
- create a BFD session SKIP
- create IPv6 BFD session SKIP
- create a BFD session (SHA1) SKIP
- add SHA1 keys SKIP
- change SHA1 key SKIP
- deactivate SHA1 authentication SKIP
- create the same BFD session twice (negative case) SKIP
- create the same BFD session twice (negative case) (SHA1) SKIP
- modify BFD session parameters SKIP
- share single SHA1 key between multiple BFD sessions SKIP
-
-BFD Authentication
-``````````````````
-::
-
- ==============================================================================
- Bidirectional Forwarding Detection (BFD) (SHA1 auth)
- ==============================================================================
- hold BFD session up SKIP
- hold BFD session up - meticulous auth SKIP
- session is not brought down by unauthenticated msg SKIP
- session is not brought down by msg with non-existent key-id SKIP
- session is not brought down by msg with wrong auth type SKIP
- simulate remote peer restart and resynchronization SKIP
- session is not kept alive by msgs with bad sequence numbers SKIP
- bring BFD session up SKIP
-
-BFD Authentication Change
-`````````````````````````
-::
-
- ==============================================================================
- Bidirectional Forwarding Detection (BFD) (changing auth)
- ==============================================================================
- change auth key without disturbing session state (delayed) SKIP
- change auth key without disturbing session state (immediate) SKIP
- turn auth off without disturbing session state (delayed) SKIP
- turn auth off without disturbing session state (immediate) SKIP
- turn auth on without disturbing session state (delayed) SKIP
- turn auth on without disturbing session state (immediate) SKIP
-
-BFD CLI
-````````
-::
-
- ==============================================================================
- Bidirectional Forwarding Detection (BFD) (CLI)
- ==============================================================================
- create/modify/delete IPv4 BFD UDP session SKIP
- create/modify/delete IPv6 BFD UDP session SKIP
- create/modify/delete IPv6 BFD UDP session (authenticated) SKIP
- create/modify/delete IPv4 BFD UDP session (authenticated) SKIP
- put session admin-up and admin-down SKIP
- turn authentication on and off SKIP
- turn authentication on and off (delayed) SKIP
- set/delete meticulous SHA1 auth key SKIP
- set/delete SHA1 auth key SKIP
- set/del udp echo source SKIP
- show commands SKIP
-
-BFD IPv4
-````````
-::
-
- ==============================================================================
- Bidirectional Forwarding Detection (BFD)
- ==============================================================================
- put session admin-up and admin-down SKIP
- configuration change while peer in demand mode SKIP
- verify session goes down after inactivity SKIP
- echo function SKIP
- session goes down if echo function fails SKIP
- echo packets looped back SKIP
- echo function stops if echo source is removed SKIP
- echo function stops if peer sets required min echo rx zero SKIP
- hold BFD session up SKIP
- immediately honor remote required min rx reduction SKIP
- interface with bfd session deleted SKIP
- echo packets with invalid checksum don't keep a session up SKIP
- large remote required min rx interval SKIP
- modify detect multiplier SKIP
- modify session - double required min rx SKIP
- modify session - halve required min rx SKIP
- no periodic frames outside poll sequence if remote demand set SKIP
- test correct response to control frame with poll bit set SKIP
- test poll sequence queueing SKIP
- bring BFD session down SKIP
- bring BFD session up SKIP
- bring BFD session up - first frame looked up by address pair SKIP
- verify slow periodic control frames while session down SKIP
- stale echo packets don't keep a session up SKIP
- no packets when zero remote required min rx interval SKIP
-
-BFD IPv6
-````````
-::
-
- ==============================================================================
- Bidirectional Forwarding Detection (BFD) (IPv6)
- ==============================================================================
- echo function used SKIP
- echo packets looped back SKIP
- hold BFD session up SKIP
- interface with bfd session deleted SKIP
- bring BFD session up SKIP
- bring BFD session up - first frame looked up by address pair SKIP
-
- ==============================================================================
- BFD-FIB interactions (IPv6)
- ==============================================================================
- BFD-FIB interactions SKIP
-
-BIER - Bit Indexed Explicit Replication
-```````````````````````````````````````
-::
-
- ==============================================================================
- BIER Test Case
- ==============================================================================
- BIER end-to-end OK
- BIER head OK
- BIER head over UDP OK
- BIER midpoint OK
- BIER Tail OK
- BIER Tail over UDP OK
-
- ==============================================================================
- BIER FIB Test Case
- ==============================================================================
- BFIB Unit Tests OK
-
-Classifier
-``````````
-::
-
- ==============================================================================
- Classifier Test Case
- ==============================================================================
- IP ACL test OK
- MAC ACL test OK
- IP PBR test OK
-
-Container Integration
-`````````````````````
-::
-
- ==============================================================================
- Container integration extended testcases
- ==============================================================================
- IPv4 basic connectivity test SKIP
- IPv6 basic connectivity test SKIP
- Create loopbacks overlapping with remote addresses SKIP
- IPv4 local-spoof connectivity test SKIP
- IPv6 local-spoof connectivity test SKIP
- Configure container commands SKIP
- IPv4 test after configuring container SKIP
- IPv6 test after configuring container SKIP
- Unconfigure container commands SKIP
- IPv4 local-spoof after unconfig test SKIP
- IPv6 local-spoof after unconfig test SKIP
-
-CRUD Loopback
-`````````````
-::
-
- ==============================================================================
- CRUD Loopback
- ==============================================================================
- test_crud (test_interface_crud.TestLoopbackInterfaceCRUD) OK
- test_down (test_interface_crud.TestLoopbackInterfaceCRUD) OK
-
-DHCP
-````
-::
-
- ==============================================================================
- DHCP Test Case
- ==============================================================================
- DHCPv6 Proxy OK
- DHCP Client OK
- DHCPv4 Proxy OK
-
-Distributed Virtual Router
-``````````````````````````
-::
-
- ==============================================================================
- Distributed Virtual Router
- ==============================================================================
- Distributed Virtual Router OK
- L2 Emulation OK
-
-DS-Lite Softwire
-````````````````
-::
-
- ==============================================================================
- DS-Lite Test Cases
- ==============================================================================
- Test DS-Lite OK
-
-FIB
-```
-::
-
- ==============================================================================
- FIB Test Case
- ==============================================================================
- FIB Unit Tests OK
-
-Flowprobe
-`````````
-::
-
- ==============================================================================
- Re-enable Flowprobe feature
- ==============================================================================
- disable flowprobe feature after first packets and re-enable SKIP
-
- ==============================================================================
- collect information on Ethernet, IP4 and IP6 datapath (no timers)
- ==============================================================================
- no timers, one CFLOW packet, 9 Flows inside OK
- no timers, two CFLOW packets (mtu=256), 3 Flows in each OK
- L2 data on IP4 datapath OK
- L2 data on IP6 datapath OK
- L2 data on L2 datapath OK
- L3 data on IP4 datapath OK
- L3 data on IP6 datapath OK
- L3 data on L2 datapath OK
- L4 data on IP4 datapath OK
- L4 data on IP6 datapath OK
- L4 data on L2 datapath OK
- verify templates on IP6 datapath OK
- verify templates on IP4 datapath OK
- verify template on L2 datapath OK
-
- ==============================================================================
- Disable Flowprobe feature
- ==============================================================================
- disable flowprobe feature after first packets SKIP
-
- ==============================================================================
- Re-enable IPFIX
- ==============================================================================
- disable IPFIX after first packets and re-enable after few packets SKIP
-
- ==============================================================================
- Disable IPFIX
- ==============================================================================
- disable IPFIX after first packets SKIP
-
-Geneve Tunnels
-``````````````
-::
-
- ==============================================================================
- GENEVE Test Case
- ==============================================================================
- Decapsulation test OK
- Encapsulation test OK
- Multicast flood test OK
- Multicast receive test OK
- Unicast flood test OK
-
-GRE Tunnels
-```````````
-::
-
- ==============================================================================
- GRE Test Case
- ==============================================================================
- GRE IPv4 tunnel Tests OK
- GRE IPv6 tunnel Tests OK
- GRE tunnel L2 Tests OK
- GRE tunnel VRF Tests OK
-
-GTPU Tunnels
-````````````
-::
-
- ==============================================================================
- GTPU Test Case
- ==============================================================================
- Decapsulation test OK
- Encapsulation test OK
- Multicast flood test OK
- Multicast receive test OK
- Unicast flood test OK
-
-IP Multicast Routing
-````````````````````
-::
-
- ==============================================================================
- IP Multicast Test Case
- ==============================================================================
- IP Multicast Bi-directional OK
- IPv6 Multicast Replication OK
- IPv6 Multicast Replication in non-default table OK
- IP Multicast Replication OK
- IP Multicast Connected Source check OK
- IP Multicast Signal OK
- IP Multicast Replication in non-default table OK
-
-IPSec
-`````
-::
-
- ==============================================================================
- Basic test for IPSEC using AH transport and Tunnel mode
- ==============================================================================
- ipsec ah v4 transport basic test OK
- ipsec ah v4 transport burst test OK
- ipsec ah 4o4 tunnel basic test OK
- ipsec ah 4o4 tunnel burst test OK
-
- ==============================================================================
- Basic test for ipsec esp sanity - tunnel and transport modes.
- ==============================================================================
- ipsec esp v4 transport basic test OK
- ipsec esp v4 transport burst test OK
- ipsec esp 4o4 tunnel basic test OK
- ipsec esp 4o4 tunnel burst test OK
-
-IPv4 FIB CRUD
-`````````````
-::
-
- ==============================================================================
- FIB - add/update/delete - ip4 routes
- ==============================================================================
- Add 1k routes OK
- Delete 100 routes OK
- Add 1k routes OK
- Delete 1.5k routes OK
-
-IPv4 Routing
-````````````
-::
-
- ==============================================================================
- IPv4 Test Case
- ==============================================================================
- IPv4 FIB test OK
-
- ==============================================================================
- IPv4 routes via NULL
- ==============================================================================
- IP NULL route OK
-
- ==============================================================================
- IPv4 disabled
- ==============================================================================
- IP Disabled OK
-
- ==============================================================================
- IPv4 Subnets
- ==============================================================================
- IP Sub Nets OK
-
- ==============================================================================
- IPv4 VLAN-0
- ==============================================================================
- IP VLAN-0 OK
-
- ==============================================================================
- IPv4 Load-Balancing
- ==============================================================================
- IP Load-Balancing OK
-
- ==============================================================================
- IPv4 Deaggregate Routes
- ==============================================================================
- IP Deag Routes OK
-
- ==============================================================================
- IPv4 Input Exceptions
- ==============================================================================
- IP Input Exceptions OK
-
- ==============================================================================
- IPv4 Punt Police/Redirect
- ==============================================================================
- IP punt police and redirect OK
-
-IPv4 VRF Multi-instance
-```````````````````````
-::
-
- ==============================================================================
- IP4 VRF Multi-instance Test Case
- ==============================================================================
- IP4 VRF Multi-instance test 1 - create 5 BDs OK
- IP4 VRF Multi-instance test 2 - delete 2 VRFs OK
- IP4 VRF Multi-instance 3 - add 2 VRFs OK
- IP4 VRF Multi-instance test 4 - delete 4 VRFs OK
-
-IPv6 Routing
-````````````
-::
-
- ==============================================================================
- IPv6 Test Case
- ==============================================================================
- IPv6 FIB test OK
- IPv6 Neighbour Solicitation Exceptions OK
- ND Duplicates OK
- IPv6 Router Solicitation Exceptions OK
-
- ==============================================================================
- IPv6 Punt Police/Redirect
- ==============================================================================
- IP6 punt police and redirect OK
-
- ==============================================================================
- IPv6 disabled
- ==============================================================================
- IP Disabled OK
-
- ==============================================================================
- IPv6 ND ProxyTest Case
- ==============================================================================
- IPv6 Proxy ND OK
-
- ==============================================================================
- IPv6 Load-Balancing
- ==============================================================================
- IPv6 Load-Balancing OK
-
- ==============================================================================
- IPv6 routes via NULL
- ==============================================================================
- IP NULL route OK
-
- ==============================================================================
- IPv6 Input Exceptions
- ==============================================================================
- IP6 Input Exceptions OK
-
-IPv6 VRF Multi-instance
-```````````````````````
-::
-
- ==============================================================================
- IP6 VRF Multi-instance Test Case
- ==============================================================================
- IP6 VRF Multi-instance test 1 - create 4 VRFs OK
- IP6 VRF Multi-instance test 2 - reset 2 VRFs OK
- IP6 VRF Multi-instance 3 - add 2 VRFs OK
- IP6 VRF Multi-instance test 4 - reset 4 VRFs OK
-
-IRB Integrated Routing-Bridging
-```````````````````````````````
-::
-
- ==============================================================================
- IRB Test Case
- ==============================================================================
- IPv4 IRB test 1 OK
- IPv4 IRB test 2 OK
-
-Kube-proxy
-``````````
-::
-
- ==============================================================================
- Kube-proxy Test Case
- ==============================================================================
- Kube-proxy NAT44 OK
- Kube-proxy NAT46 SKIP
- Kube-proxy NAT64 SKIP
- Kube-proxy NAT66 SKIP
-
-L2 FIB CRUD
-```````````
-::
-
- ==============================================================================
- L2 FIB Test Case
- ==============================================================================
- L2 FIB - program 100 + 100 MACs OK
- L2 FIB - program 100 + delete 12 MACs OK
- L2 FIB - flush all OK
- L2 FIB - flush BD OK
- L2 FIB - flush interface OK
- L2 FIB - mac learning events OK
- L2 FIB - mac learning max macs in event OK
- L2 FIB - program 100 MACs OK
- L2 FIB - Program 10 MACs, learn 10 OK
-
-L2BD Multi-instance
-```````````````````
-::
-
- ==============================================================================
- L2BD Multi-instance Test Case
- ==============================================================================
- L2BD Multi-instance test 1 - create 5 BDs OK
- L2BD Multi-instance test 2 - update data of 5 BDs OK
- L2BD Multi-instance test 3 - delete 2 BDs OK
- L2BD Multi-instance test 4 - add 2 BDs OK
- L2BD Multi-instance test 5 - delete 5 BDs SKIP
-
-L2BD Switching
-``````````````
-::
-
- ==============================================================================
- L2BD Test Case
- ==============================================================================
- L2BD MAC learning dual-loop test OK
- L2BD MAC learning single-loop test OK
-
-L2XC Multi-instance
-```````````````````
-::
-
- ==============================================================================
- L2XC Multi-instance Test Case
- ==============================================================================
- L2XC Multi-instance test 1 - create 10 cross-connects OK
- L2XC Multi-instance test 2 - delete 4 cross-connects OK
- L2BD Multi-instance 3 - add new 4 cross-connects OK
- L2XC Multi-instance test 4 - delete 10 cross-connects OK
-
-L2XC Switching
-``````````````
-::
-
- ==============================================================================
- L2XC Test Case
- ==============================================================================
- L2XC dual-loop test OK
- L2XC single-loop test OK
-
-LISP Tunnels
-````````````
-::
-
- ==============================================================================
- Basic LISP test
- ==============================================================================
- Test case for basic encapsulation OK
-
-Load Balancer
-`````````````
-::
-
- ==============================================================================
- Load Balancer Test Case
- ==============================================================================
- Load Balancer IP4 GRE4 OK
- Load Balancer IP4 GRE6 OK
- Load Balancer IP6 GRE4 OK
- Load Balancer IP6 GRE6 OK
-
-MACIP Access Control
-````````````````````
-::
-
- ==============================================================================
- MACIP Tests
- ==============================================================================
- MACIP 10 ACLs each with 100+ entries OK
- MACIP 10 ACLs each with 100+ entries with IP4 traffic OK
- MACIP 10 ACLs each with 100+ entries with IP6 traffic OK
- MACIP ACL with 10 entries OK
- MACIP ACL with 100 entries OK
- MACIP ACL with 2 entries OK
- MACIP ACL with 20 entries OK
- MACIP ACL with 5 entries OK
- MACIP ACL with 50 entries OK
- MACIP 2 ACLs each with 100+ entries OK
- MACIP replace ACL OK
- MACIP ACL delete intf with acl OK
-
- ==============================================================================
- MACIP with IP6 traffic
- ==============================================================================
- IP6 MACIP exactMAC|exactIP ACL bridged traffic OK
- IP6 MACIP exactMAC|subnetIP ACL bridged traffic OK
- IP6 MACIP exactMAC|wildIP ACL bridged traffic OK
- IP6 MACIP oui_MAC|exactIP ACL bridged traffic OK
- IP6 MACIP ouiMAC|subnetIP ACL bridged traffic OK
- IP6 MACIP ouiMAC|wildIP ACL bridged traffic OK
- IP6 MACIP wildcardMAC|exactIP ACL bridged traffic OK
- IP6 MACIP wildcardMAC|subnetIP ACL bridged traffic OK
- IP6 MACIP wildcardMAC|wildIP ACL bridged traffic OK
- MACIP replace ACL with IP6 traffic OK
- IP6 MACIP exactMAC|exactIP ACL routed traffic OK
- IP6 MACIP exactMAC|subnetIP ACL routed traffic OK
- IP6 MACIP exactMAC|wildIP ACL routed traffic OK
- IP6 MACIP ouiMAC|exactIP ACL routed traffic OK
- IP6 MACIP ouiMAC|subnetIP ACL routed traffic OK
- IP6 MACIP ouiMAC|wildIP ACL routed traffic OK
- IP6 MACIP wildcardMAC|exactIP ACL routed traffic OK
- IP6 MACIP wildcardMAC|subnetIP ACL routed traffic OK
- IP6 MACIP wildcardMAC|wildIP ACL OK
-
- ==============================================================================
- MACIP with IP4 traffic
- ==============================================================================
- IP4 MACIP wildcardMAC|exactIP ACL bridged traffic OK
- IP4 MACIP exactMAC|exactIP ACL bridged traffic OK
- IP4 MACIP exactMAC|subnetIP ACL bridged traffic OK
- IP4 MACIP exactMAC|wildIP ACL bridged traffic OK
- IP4 MACIP ouiMAC|exactIP ACL bridged traffic OK
- IP4 MACIP ouiMAC|subnetIP ACL bridged traffic OK
- IP4 MACIP ouiMAC|wildIP ACL bridged traffic OK
- IP4 MACIP wildcardMAC|subnetIP ACL bridged traffic OK
- IP4 MACIP wildcardMAC|wildIP ACL bridged traffic OK
- MACIP replace ACL with IP4 traffic OK
- IP4 MACIP exactMAC|exactIP ACL routed traffic OK
- IP4 MACIP exactMAC|subnetIP ACL routed traffic OK
- IP4 MACIP exactMAC|wildIP ACL routed traffic OK
- IP4 MACIP ouiMAC|exactIP ACL routed traffic OK
- IP4 MACIP ouiMAC|subnetIP ACL routed traffic OK
- IP4 MACIP ouiMAC|wildIP ACL routed traffic OK
- IP4 MACIP wildcardMAC|exactIP ACL routed traffic OK
- IP4 MACIP wildcardMAC|subnetIP ACL routed traffic OK
- IP4 MACIP wildcardMAC|wildIP ACL OK
-
-MAP Softwires
-`````````````
-::
-
- ==============================================================================
- MAP Test Case
- ==============================================================================
- MAP-E OK
-
-MFIB Multicast FIB
-``````````````````
-::
-
- ==============================================================================
- MFIB Test Case
- ==============================================================================
- MFIB Unit Tests OK
-
-MPLS Switching
-``````````````
-::
-
- ==============================================================================
- MPLS-L2
- ==============================================================================
- Virtual Private LAN Service OK
- Virtual Private Wire Service OK
-
- ==============================================================================
- MPLS Test Case
- ==============================================================================
- MPLS Local Label Binding test OK
- MPLS Deagg OK
- MPLS label imposition test OK
- MPLS Interface Receive OK
- MPLS Multicast Head-end OK
- MPLS IPv4 Multicast Tail OK
- MPLS IPv6 Multicast Tail OK
- MPLS Multicast Mid Point OK
- MPLS label swap tests OK
- MPLS Tunnel Tests OK
- MPLS V4 Explicit NULL test OK
- MPLS V6 Explicit NULL test OK
-
- ==============================================================================
- MPLS PIC edge convergence
- ==============================================================================
- MPLS eBGP PIC edge convergence OK
- MPLS iBGP PIC edge convergence OK
- MPLSv6 eBGP PIC edge convergence OK
-
- ==============================================================================
- MPLS disabled
- ==============================================================================
- MPLS Disabled OK
-
-NAT44
-`````
-::
-
- ==============================================================================
- NAT44 Test Cases
- ==============================================================================
- Delete NAT44 session OK
- NAT44 dynamic translation test OK
- NAT44 handling of client packets with TTL=1 OK
- NAT44 handling of error responses to client packets with TTL=2 OK
- NAT44 handling of server packets with TTL=1 OK
- NAT44 handling of error responses to server packets with TTL=2 OK
- NAT44 interfaces without configured IP address OK
- NAT44 forwarding test OK
- NAT44 translate fragments arriving in order OK
- NAT44 translate fragments arriving out of order OK
- NAT44 hairpinning - 1:1 NAPT OK
- NAT44 hairpinning - 1:1 NAT OK
- 1:1 NAT translate packet with unknown protocol - hairpinning OK
- NAT44 translate packet with unknown protocol - hairpinning OK
- Identity NAT OK
- NAT44 multiple inside interfaces with overlapping address space OK
- Acquire NAT44 addresses from interface OK
- Identity NAT with addresses from interface OK
- Static mapping with addresses from interface OK
- IPFIX logging NAT addresses exhausted OK
- IPFIX logging NAT44 session created/delted OK
- MAX translations per user - recycle the least recently used OK
- NAT44 multiple non-overlapping address space inside interfaces OK
- One armed NAT44 OK
- NAT44 interface output feature (in2out postrouting) OK
- NAT44 interface output feature hairpinning (in2out postrouting) OK
- NAT44 interface output feature VRF aware (in2out postrouting) OK
- Ping internal host from outside network OK
- Ping NAT44 out interface from outside network OK
- NAT44 add pool addresses to FIB OK
- Port restricted NAT44 (MAP-E CE) OK
- NAT44 fragments hairpinning OK
- NAT44 set/get virtual fragmentation reassembly OK
- 1:1 NAT initialized from inside network OK
- NAT44 interfaces without configured IP address - 1:1 NAT OK
- NAT44 local service load balancing OK
- 1:1 NAT initialized from outside network OK
- 1:1 NAT translate packet with unknown protocol OK
- 1:1 NAT VRF awareness OK
- 1:1 NAPT initialized from inside network OK
- NAT44 interfaces without configured IP address - 1:1 NAPT OK
- 1:1 NAPT initialized from outside network OK
- Twice NAT44 OK
- Acquire twice NAT44 addresses from interface OK
- Twice NAT44 local service load balancing OK
- NAT44 translate packet with unknown protocol OK
- NAT44 tenant VRF independent address pool mode OK
- NAT44 tenant VRF aware address pool mode OK
-
- ==============================================================================
- Deterministic NAT Test Cases
- ==============================================================================
- Deterministic NAT translation test (TCP, UDP, ICMP) OK
- NAT plugin run deterministic mode OK
- Deterministic NAT multiple users OK
- Deterministic NAT maximum sessions per user limit SKIP
- Deterministic NAT session timeouts SKIP
- Set deterministic NAT timeouts OK
- Deterministic NAT TCP session close from inside network OK
- Deterministic NAT TCP session close from outside network OK
-
-NAT64
-`````
-::
-
- ==============================================================================
- NAT64 Test Cases
- ==============================================================================
- NAT64 dynamic translation test OK
- NAT64 translate fragments arriving in order OK
- NAT64 translate fragments arriving out of order OK
- NAT64 hairpinning OK
- NAT64 translate packet with unknown protocol - hairpinning OK
- NAT64 ICMP Error message translation OK
- Enable/disable NAT64 feature on the interface OK
- Acquire NAT64 pool addresses from interface OK
- One armed NAT64 OK
- Add/delete address to NAT64 pool OK
- NAT64 Network-Specific Prefix OK
- NAT64 fragments hairpinning OK
- NAT64 session timeout SKIP
- Set NAT64 timeouts OK
- NAT64 static translation test OK
- Add/delete static BIB entry OK
- NAT64 translate packet with unknown protocol OK
-
-P2P Ethernet Subinterface
-`````````````````````````
-::
-
- ==============================================================================
- P2P Ethernet tests
- ==============================================================================
- delete/create p2p subif OK
- create 1k of p2p subifs OK
-
- ==============================================================================
- P2P Ethernet IPv4 tests
- ==============================================================================
- receive ipv4 packet via p2p subinterface OK
- route rx packet not matching p2p subinterface OK
- send ip4 packet via p2p subinterface OK
- drop tx ip4 packet not matching p2p subinterface OK
-
- ==============================================================================
- P2P Ethernet IPv6 tests
- ==============================================================================
- receive ipv6 packet via p2p subinterface OK
- drop rx packet not matching p2p subinterface OK
- route rx ip6 packet not matching p2p subinterface OK
- send packet via p2p subinterface OK
- drop tx ip6 packet not matching p2p subinterface OK
- standard routing without p2p subinterfaces OK
-
-PPPoE Encapsulation
-```````````````````
-::
-
- ==============================================================================
- PPPoE Test Case
- ==============================================================================
- PPPoE Add Same Session Twice Test OK
- PPPoE Decap Test OK
- PPPoE Decap Multiple Sessions Test OK
- PPPoE Delete Same Session Twice Test OK
- PPPoE Encap Test OK
- PPPoE Encap Multiple Sessions Test OK
-
-SPAN Switch Port Analyzer
-`````````````````````````
-::
-
- ==============================================================================
- SPAN Test Case
- ==============================================================================
- SPAN device rx mirror OK
- SPAN l2 broadcast mirror OK
- SPAN l2 rx tx mirror OK
- SPAN l2 tx mirror OK
- SPAN l2 rx mirror OK
- SPAN l2 rx mirror into 1ad subif+vtr OK
- SPAN l2 rx mirror into 1q subif+vtr OK
- SPAN l2 rx mirror into gre-subif+vtr OK
- SPAN l2 rx mirror into vxlan OK
-
-SRv6 Routing
-````````````
-::
-
- ==============================================================================
- SRv6 Test Case
- ==============================================================================
- Test SRv6 End (without PSP) behavior. OK
- Test SRv6 End.DT4 behavior. OK
- Test SRv6 End.DT6 behavior. OK
- Test SRv6 End.DX2 behavior. OK
- Test SRv6 End.DX4 behavior. OK
- Test SRv6 End.DX6 behavior. OK
- Test SRv6 End.X (without PSP) behavior. OK
- Test SRv6 End.X with PSP behavior. OK
- Test SRv6 End with PSP behavior. OK
- Test SRv6 Transit.Encaps behavior for IPv6. OK
- Test SRv6 Transit.Encaps behavior for IPv4. OK
- Test SRv6 Transit.Encaps behavior for L2. SKIP
- Test SRv6 Transit.Insert behavior (IPv6 only). OK
- Test SRv6 Transit.Insert behavior (IPv6 only). OK
-
-TCP/IP Stack
-````````````
-::
-
- ==============================================================================
- TCP Test Case
- ==============================================================================
- TCP builtin client/server transfer OK
- TCP Unit Tests OK
-
-UDP Stack
-`````````
-::
-
- ==============================================================================
- UDP Encap Test Case
- ==============================================================================
- UDP Encap test OK
-
-VTR VLAN Tag Rewrites
-`````````````````````
-::
-
- ==============================================================================
- VTR Test Case
- ==============================================================================
- 1AD VTR pop 1 test OK
- 1AD VTR pop 2 test OK
- 1AD VTR push 1 1AD test OK
- 1AD VTR push 1 1Q test OK
- 1AD VTR push 2 1AD test OK
- 1AD VTR push 2 1Q test OK
- 1AD VTR translate 1 -> 1 1AD test OK
- 1AD VTR translate 1 -> 1 1Q test OK
- 1AD VTR translate 1 -> 2 1AD test OK
- 1AD VTR translate 1 -> 2 1Q test OK
- 1AD VTR translate 2 -> 1 1AD test OK
- 1AD VTR translate 2 -> 1 1Q test OK
- 1AD VTR translate 2 -> 2 1AD test OK
- 1AD VTR translate 2 -> 2 1Q test OK
- 1Q VTR pop 1 test OK
- 1Q VTR push 1 test OK
- 1Q VTR push 2 test OK
- 1Q VTR translate 1 -> 1 test OK
- 1Q VTR translate 1 -> 2 test OK
-
-VXLAN Tunnels
-`````````````
-::
-
- ==============================================================================
- VXLAN Test Case
- ==============================================================================
- Decapsulation test OK
- Encapsulation test OK
- Multicast flood test OK
- Multicast receive test OK
- Unicast flood test OK
-
-VXLAN-GPE Tunnels
-`````````````````
-::
-
- ==============================================================================
- VXLAN-GPE Test Case
- ==============================================================================
- Decapsulation test SKIP
- Encapsulation test SKIP
- Multicast flood test SKIP
- Multicast receive test SKIP
- Unicast flood test SKIP
-
-Other Tests
-```````````
-::
-
- ==============================================================================
- Ping Test Case
- ==============================================================================
- basic ping test OK
- burst ping test OK
-
- ==============================================================================
- Session Test Case
- ==============================================================================
- Session Unit Tests OK
-
- ==============================================================================
- Template verification, timer tests
- ==============================================================================
- timer less than template timeout OK
- timer greater than template timeout OK
- verify cflow packet fields OK
-
diff --git a/docs/report/dpdk_performance_tests/csit_release_notes.rst b/docs/report/dpdk_performance_tests/csit_release_notes.rst deleted file mode 100644 index 413c7c3cea..0000000000 --- a/docs/report/dpdk_performance_tests/csit_release_notes.rst +++ /dev/null @@ -1,23 +0,0 @@ -CSIT Release Notes -================== - -Changes in CSIT |release| -------------------------- - -No code changes apart from bug fixes. - -Known Issues ------------- - -Here is the list of known issues in CSIT |release| for Testpmd performance tests: - -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ -| # | Issue | Jira ID | Description | -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ -| 1 | Testpmd in 1t1c and 2t2c setups - large variation | CSIT-569 | Suspected NIC firmware or DPDK driver issue affecting NDR | -| | of discovered NDR throughput values across | | throughput. Applies to XL710 and X710 NICs, no issues observed | -| | multiple test runs with xl710 and x710 NICs. | | on x520 NICs. | -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ -| 2 | Lower than expected NDR throughput with xl710 | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR | -| | and x710 NICs, compared to x520 NICs. | | throughput. Applies to XL710 and X710 NICs. | -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ diff --git a/docs/report/dpdk_performance_tests/documentation.rst b/docs/report/dpdk_performance_tests/documentation.rst deleted file mode 100644 index 47ec7cbe2a..0000000000 --- a/docs/report/dpdk_performance_tests/documentation.rst +++ /dev/null @@ -1,5 +0,0 @@ -Documentation
-=============
-
-`CSIT DPDK Performance Tests Documentation`_ contains detailed
-functional description and input parameters for each test case.
diff --git a/docs/report/dpdk_performance_tests/index.rst b/docs/report/dpdk_performance_tests/index.rst deleted file mode 100644 index 44d34d3d4d..0000000000 --- a/docs/report/dpdk_performance_tests/index.rst +++ /dev/null @@ -1,12 +0,0 @@ -DPDK Performance Tests -====================== - -.. toctree:: - - overview - csit_release_notes - packet_throughput_graphs/index - packet_latency_graphs/index - test_environment - documentation - diff --git a/docs/report/dpdk_performance_tests/overview.rst b/docs/report/dpdk_performance_tests/overview.rst deleted file mode 100644 index 02651d7e53..0000000000 --- a/docs/report/dpdk_performance_tests/overview.rst +++ /dev/null @@ -1,240 +0,0 @@ -Overview -======== - -Tested Physical Topologies --------------------------- - -CSIT DPDK performance tests are executed on physical baremetal servers hosted -by :abbr:`LF (Linux Foundation)` FD.io project. Testbed physical topology is -shown in the figure below.:: - - +------------------------+ +------------------------+ - | | | | - | +------------------+ | | +------------------+ | - | | | | | | | | - | | <-----------------> | | - | | DUT1 | | | | DUT2 | | - | +--^---------------+ | | +---------------^--+ | - | | | | | | - | | SUT1 | | SUT2 | | - +------------------------+ +------------------^-----+ - | | - | | - | +-----------+ | - | | | | - +------------------> TG <------------------+ - | | - +-----------+ - -SUT1 and SUT2 are two System Under Test servers (Cisco UCS C240, each with two -Intel XEON CPUs), TG is a Traffic Generator (TG, another Cisco UCS C240, with -two Intel XEON CPUs). SUTs run Testpmd/L3FWD SW SW application in Linux -user-mode as a Device Under Test (DUT). TG runs TRex SW application as a packet -Traffic Generator. Physical connectivity between SUTs and to TG is provided -using different NIC models that need to be tested for performance. Currently -installed and tested NIC models include: - -#. 2port10GE X520-DA2 Intel. -#. 2port10GE X710 Intel. -#. 2port10GE VIC1227 Cisco. -#. 2port40GE VIC1385 Cisco. -#. 2port40GE XL710 Intel. - -From SUT and DUT perspective, all performance tests involve forwarding packets -between two physical Ethernet ports (10GE or 40GE). Due to the number of -listed NIC models tested and available PCI slot capacity in SUT servers, in -all of the above cases both physical ports are located on the same NIC. In -some test cases this results in measured packet throughput being limited not -by VPP DUT but by either the physical interface or the NIC capacity. - -Going forward CSIT project will be looking to add more hardware into FD.io -performance labs to address larger scale multi-interface and multi-NIC -performance testing scenarios. - -Note that reported DUT (DPDK) performance results are specific to the SUTs -tested. Current :abbr:`LF (Linux Foundation)` FD.io SUTs are based on Intel -XEON E5-2699v3 2.3GHz CPUs. SUTs with other CPUs are likely to yield different -results. A good rule of thumb, that can be applied to estimate DPDK packet -thoughput for Phy-to-Phy (NIC-to-NIC, PCI-to-PCI) topology, is to expect -the forwarding performance to be proportional to CPU core frequency, -assuming CPU is the only limiting factor and all other SUT parameters -equivalent to FD.io CSIT environment. The same rule of thumb can be also -applied for Phy-to-VM/LXC-to-Phy (NIC-to-VM/LXC-to-NIC) topology, but due to -much higher dependency on intensive memory operations and sensitivity to Linux -kernel scheduler settings and behaviour, this estimation may not always yield -good enough accuracy. - -For detailed :abbr:`LF (Linux Foundation)` FD.io test bed specification and -physical topology please refer to `LF FD.io CSIT testbed wiki page -<https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_. - -Performance Tests Coverage --------------------------- - -Performance tests are split into two main categories: - -- Throughput discovery - discovery of packet forwarding rate using binary search - in accordance to :rfc:`2544`. - - - NDR - discovery of Non Drop Rate packet throughput, at zero packet loss; - followed by one-way packet latency measurements at 10%, 50% and 100% of - discovered NDR throughput. - - PDR - discovery of Partial Drop Rate, with specified non-zero packet loss - currently set to 0.5%; followed by one-way packet latency measurements at - 100% of discovered PDR throughput. - -- Throughput verification - verification of packet forwarding rate against - previously discovered throughput rate. These tests are currently done against - 0.9 of reference NDR, with reference rates updated periodically. - -CSIT |release| includes following performance test suites, listed per NIC type: - -- 2port10GE X520-DA2 Intel - - - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between - two Interfaces. - -- 2port40GE XL710 Intel - - - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between - two Interfaces. - -- 2port10GE X520-DA2 Intel - - - **IPv4 Routed Forwarding** - L3 IP forwarding of Ethernet frames between - two Interfaces. - -Execution of performance tests takes time, especially the throughput discovery -tests. Due to limited HW testbed resources available within FD.io labs hosted -by Linux Foundation, the number of tests for NICs other than X520 (a.k.a. -Niantic) has been limited to few baseline tests. Over time we expect the HW -testbed resources to grow, and will be adding complete set of performance -tests for all models of hardware to be executed regularly and(or) -continuously. - -Performance Tests Naming ------------------------- - -CSIT |release| follows a common structured naming convention for all performance -and system functional tests, introduced in CSIT |release-1|. - -The naming should be intuitive for majority of the tests. Complete description -of CSIT test naming convention is provided on `CSIT test naming wiki -<https://wiki.fd.io/view/CSIT/csit-test-naming>`_. - -Methodology: Multi-Core and Multi-Threading -------------------------------------------- - -**Intel Hyper-Threading** - CSIT |release| performance tests are executed with -SUT servers' Intel XEON processors configured in Intel Hyper-Threading Disabled -mode (BIOS setting). This is the simplest configuration used to establish -baseline single-thread single-core application packet processing and forwarding -performance. Subsequent releases of CSIT will add performance tests with Intel -Hyper-Threading Enabled (requires BIOS settings change and hard reboot of -server). - -**Multi-core Tests** - CSIT |release| multi-core tests are executed in the -following VPP thread and core configurations: - -#. 1t1c - 1 pmd worker thread on 1 CPU physical core. -#. 2t2c - 2 pmd worker threads on 2 CPU physical cores. - -Note that in many tests running Testpmd/L3FWD reaches tested NIC I/O bandwidth -or packets-per-second limit. - -Methodology: Packet Throughput ------------------------------- - -Following values are measured and reported for packet throughput tests: - -- NDR binary search per :rfc:`2544`: - - - Packet rate: "RATE: <aggregate packet rate in packets-per-second> pps - (2x <per direction packets-per-second>)" - - Aggregate bandwidth: "BANDWIDTH: <aggregate bandwidth in Gigabits per - second> Gbps (untagged)" - -- PDR binary search per :rfc:`2544`: - - - Packet rate: "RATE: <aggregate packet rate in packets-per-second> pps (2x - <per direction packets-per-second>)" - - Aggregate bandwidth: "BANDWIDTH: <aggregate bandwidth in Gigabits per - second> Gbps (untagged)" - - Packet loss tolerance: "LOSS_ACCEPTANCE <accepted percentage of packets - lost at PDR rate>"" - -- NDR and PDR are measured for the following L2 frame sizes: - - - IPv4: 64B, 1518B, 9000B. - -All rates are reported from external Traffic Generator perspective. - - -Methodology: Packet Latency ---------------------------- - -TRex Traffic Generator (TG) is used for measuring latency of Testpmd DUTs. -Reported latency values are measured using following methodology: - -- Latency tests are performed at 10%, 50% of discovered NDR rate (non drop rate) - for each NDR throughput test and packet size (except IMIX). -- TG sends dedicated latency streams, one per direction, each at the rate of - 10kpps at the prescribed packet size; these are sent in addition to the main - load streams. -- TG reports min/avg/max latency values per stream direction, hence two sets - of latency values are reported per test case; future release of TRex is - expected to report latency percentiles. -- Reported latency values are aggregate across two SUTs due to three node - topology used for all performance tests; for per SUT latency, reported value - should be divided by two. -- 1usec is the measurement accuracy advertised by TRex TG for the setup used in - FD.io labs used by CSIT project. -- TRex setup introduces an always-on error of about 2*2usec per latency flow - - additonal Tx/Rx interface latency induced by TRex SW writing and reading - packet timestamps on CPU cores without HW acceleration on NICs closer to the - interface line. - -Methodology: TRex Traffic Generator Usage ------------------------------------------ - -The `TRex traffic generator <https://wiki.fd.io/view/TRex>`_ is used for all -CSIT performance tests. TRex stateless mode is used to measure NDR and PDR -throughputs using binary search (NDR and PDR discovery tests) and for quick -checks of DUT performance against the reference NDRs (NDR check tests) for -specific configuration. - -TRex is installed and run on the TG compute node. The typical procedure is: - -- If the TRex is not already installed on TG, it is installed in the - suite setup phase - see `TRex intallation`_. -- TRex configuration is set in its configuration file - :: - - /etc/trex_cfg.yaml - -- TRex is started in the background mode - :: - - $ sh -c 'cd <t-rex-install-dir>/scripts/ && sudo nohup ./t-rex-64 -i -c 7 --iom 0 > /tmp/trex.log 2>&1 &' > /dev/null - -- There are traffic streams dynamically prepared for each test, based on traffic - profiles. The traffic is sent and the statistics obtained using - :command:`trex_stl_lib.api.STLClient`. - -**Measuring packet loss** - -- Create an instance of STLClient -- Connect to the client -- Add all streams -- Clear statistics -- Send the traffic for defined time -- Get the statistics - -If there is a warm-up phase required, the traffic is sent also before test and -the statistics are ignored. - -**Measuring latency** - -If measurement of latency is requested, two more packet streams are created (one -for each direction) with TRex flow_stats parameter set to STLFlowLatencyStats. In -that case, returned statistics will also include min/avg/max latency values. diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst deleted file mode 100644 index b99bc2b1f6..0000000000 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst +++ /dev/null @@ -1,28 +0,0 @@ -Packet Latency Graphs -===================== - -Plotted results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Median and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by -`FD.io test executor dpdk performance jobs`_ that created result output file -used as data source for the graph, *Y-axis labels* are measured packet Latency -[uSec] values, and the *Graph legend* lists the plotted test suites and their -indices. Latency is reported for concurrent symmetric bi-directional flows, -separately for each direction: i) West-to-East: -TGint1-to-SUT1-to-SUT2-to-TGint2, and ii) East-to-West: -TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by - `FD.io test executor dpdk performance jobs`_ with Robot Framework result - files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_. - -.. toctree:: - - l2 - ip4 diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/ip4.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/ip4.rst deleted file mode 100644 index 0bccc4b920..0000000000 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/ip4.rst +++ /dev/null @@ -1,73 +0,0 @@ -IPv4 Routed-Forwarding -====================== - -This section includes summary graphs of L3FWD Phy-to-Phy performance with packet -routed forwarding measured at 50% of discovered NDR throughput rate. Latency is -reported for L3FWD running in multiple configurations of L3FWD pmd thread(s), -a.k.a. L3FWD data plane thread(s), and their physical CPU core(s) placement. - -L3FWD packet latency - running in configuration of **one worker thread (1t) on one -physical core (1c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-1t1c-ipv4-ndrdisc-lat50.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-ipv4-ndrdisc-lat50} - \label{fig:64B-1t1c-ipv4-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 1. L3FWD 1thread 1core - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -E '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -E '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - -Testpmd packet latency - running in configuration of **two worker threads (2t) -on two physical cores (2c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-2t2c-ipv4-ndrdisc-lat50.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-ipv4-ndrdisc-lat50} - \label{fig:64B-2t2c-ipv4-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 2. L3FWD 2thread 2core - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -E '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -E '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/l2.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/l2.rst deleted file mode 100644 index bcb7c4468c..0000000000 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/l2.rst +++ /dev/null @@ -1,74 +0,0 @@ -L2 Ethernet Interface Loop -========================== - -This section includes summary graphs of Testpmd Phy-to-Phy packet -latency with L2 Ethernet Interface Loop measured at 50% of discovered -NDR throughput rate. Latency is reported for Testpmd running in multiple -configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane -thread(s), and their physical CPU core(s) placement. - -Testpmd packet latency - running in configuration of **one worker thread (1t) on one -physical core (1c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-1t1c-l2-ndrdisc-lat50.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-l2-ndrdisc-lat50} - \label{fig:64B-1t1c-l2-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 1. Testpmd 1thread 1core - packet latency for Phy-to-Phy L2 Ethernet Looping.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - -Testpmd packet latency - running in configuration of **two worker threads (2t) -on two physical cores (2c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-2t2c-l2-ndrdisc-lat50.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-l2-ndrdisc-lat50} - \label{fig:64B-2t2c-l2-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 2. Testpmd 2thread 2core - packet latency for Phy-to-Phy L2 Ethernet Looping.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst deleted file mode 100644 index c1fc3f1bbc..0000000000 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst +++ /dev/null @@ -1,39 +0,0 @@ -Packet Throughput Graphs -======================== - -Plotted results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. - -For each plotted test case, Box-and-Whisker plots show the quartiles -(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / -mean, 3rd quartile / 75th percentile, Max) across collected data set -(data set size stated in the note below). Outliers are plotted as -individual points. Min and max values are plotted as bottom and top -Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and -top edge of the box. If multiple samples match only two values, and all -samples fall between them, then no whiskers are plotted. If all samples -have the same value, only a horizontal line is plotted. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by -`FD.io test executor dpdk performance jobs`_ jobs that created result output -files used as data sources for the graph, *Y-axis labels* are measured Packets -Per Second [pps] values, and the *Graph legend* lists the plotted test suites -and their indices. - -.. note:: - - Test results have been generated by - `FD.io test executor dpdk performance jobs`_ with Robot Framework result - files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_. - Plotted data set size per test case is equal to the number of job executions - presented in this report version: **10**. - -.. toctree:: - - l2 - ip4 diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/ip4.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/ip4.rst deleted file mode 100644 index 1ad08992bc..0000000000 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/ip4.rst +++ /dev/null @@ -1,150 +0,0 @@ -IPv4 Routed-Forwarding -====================== - -Following sections include summary graphs ofL3FWD Phy-to-Phy performance with -packet routed forwarding, including NDR throughput (zero packet loss) -and PDR throughput (<0.5% packet loss). Performance is reported for L3FWD -running in multiple configurations of L3FWD pmd thread(s), a.k.a. L3FWD -data plane thread(s), and their physical CPU core(s) placement. - -NDR Throughput -~~~~~~~~~~~~~~ - -Testpmd NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-1t1c-ipv4-ndrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-ipv4-ndrdisc} - \label{fig:64B-1t1c-ipv4-ndrdisc-dpdk} - \end{figure} - -*Figure 1. L3FWD 1thread 1core - NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - -Testpmd NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-2t2c-ipv4-ndrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-ipv4-ndrdisc} - \label{fig:64B-2t2c-ipv4-ndrdisc-dpdk} - \end{figure} - -*Figure 2. L3FWD 2threads 2cores - NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * - -PDR Throughput -~~~~~~~~~~~~~~ - -L3FWD PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-1t1c-ipv4-pdrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-ipv4-pdrdisc} - \label{fig:64B-1t1c-ipv4-pdrdisc-dpdk} - \end{figure} - -*Figure 3. L3FWD 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-ethip4-ip4base-l3fwd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-ethip4-ip4base-l3fwd-pdrdisc' * - -L3FWD PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-2t2c-ipv4-pdrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-ipv4-pdrdisc} - \label{fig:64B-2t2c-ipv4-pdrdisc-dpdk} - \end{figure} - -*Figure 4. L3FWD 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-ethip4-ip4base-l3fwd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-ethip4-ip4base-l3fwd-pdrdisc' * diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/l2.rst deleted file mode 100644 index d96bf9f37d..0000000000 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/l2.rst +++ /dev/null @@ -1,150 +0,0 @@ -L2 Ethernet Interface Loop -========================== - -Following sections include summary graphs of DPDK Testpmd Phy-to-Phy performance -with L2 Ethernet Interface Loop, including NDR throughput (zero packet loss) -and PDR throughput (<0.5% packet loss). Performance is reported for Testpmd -running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd -data plane thread(s), and their physical CPU core(s) placement. - -NDR Throughput -~~~~~~~~~~~~~~ - -Testpmd NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-1t1c-l2-ndrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-l2-ndrdisc} - \label{fig:64B-1t1c-l2-ndrdisc-dpdk} - \end{figure} - -*Figure 1. Testpmd 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - -Testpmd NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-2t2c-l2-ndrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-l2-ndrdisc} - \label{fig:64B-2t2c-l2-ndrdisc-dpdk} - \end{figure} - -*Figure 2. Testpmd 2threads 2cores - NDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * - -PDR Throughput -~~~~~~~~~~~~~~ - -Testpmd PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-1t1c-l2-pdrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-l2-pdrdisc} - \label{fig:64B-1t1c-l2-pdrdisc-dpdk} - \end{figure} - -*Figure 3. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-eth-l2xcbase-testpmd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-eth-l2xcbase-testpmd-pdrdisc' * - -Testpmd PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/dpdk/64B-2t2c-l2-pdrdisc.html"></iframe> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-l2-pdrdisc} - \label{fig:64B-2t2c-l2-pdrdisc-dpdk} - \end{figure} - -*Figure 4. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-eth-l2xcbase-testpmd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-eth-l2xcbase-testpmd-pdrdisc' * diff --git a/docs/report/dpdk_performance_tests/test_environment.rst b/docs/report/dpdk_performance_tests/test_environment.rst deleted file mode 100644 index eb6617b11c..0000000000 --- a/docs/report/dpdk_performance_tests/test_environment.rst +++ /dev/null @@ -1,68 +0,0 @@ -.. include:: ../vpp_performance_tests/test_environment_intro.rst
-
-.. include:: ../vpp_performance_tests/test_environment_sut_conf_1.rst
-
-.. include:: ../vpp_performance_tests/test_environment_sut_conf_3.rst
-
-
-DUT Configuration - DPDK
-------------------------
-
-**DPDK Version**
-
-|dpdk-release|
-
-**DPDK Compile Parameters**
-
-.. code-block:: bash
-
- make install T=x86_64-native-linuxapp-gcc -j
-
-**Testpmd Startup Configuration**
-
-Testpmd startup configuration changes per test case with different settings for CPU
-cores, rx-queues. Startup config is aligned with applied test case tag:
-
-Tagged by **1T1C**
-
-.. code-block:: bash
-
- testpmd -c 0x3 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=1 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start
-
-Tagged by **2T2C**
-
-.. code-block:: bash
-
- testpmd -c 0x403 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=2 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start
-
-Tagged by **4T4C**
-
-.. code-block:: bash
-
- testpmd -c 0xc07 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=4 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=2 --txq=2 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start
-
-**L3FWD Startup Configuration**
-
-L3FWD startup configuration changes per test case with different settings for CPU
-cores, rx-queues. Startup config is aligned with applied test case tag:
-
-Tagged by **1T1C**
-
-.. code-block:: bash
-
- l3fwd -l 1 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype
-
-Tagged by **2T2C**
-
-.. code-block:: bash
-
- l3fwd -l 1,2 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype
-
-Tagged by **4T4C**
-
-.. code-block:: bash
-
- l3fwd -l 1,2,3,4 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype
-
-
-.. include:: ../vpp_performance_tests/test_environment_tg.rst
diff --git a/docs/report/honeycomb_performance_tests/csit_release_notes.rst b/docs/report/honeycomb_performance_tests/csit_release_notes.rst deleted file mode 100644 index d79907d4ca..0000000000 --- a/docs/report/honeycomb_performance_tests/csit_release_notes.rst +++ /dev/null @@ -1,20 +0,0 @@ -CSIT Release Notes
-==================
-
-Changes in CSIT |release|
--------------------------
-
-#. No changes since previous release.
-
-Known Issues
-------------
-
-Here is the list of known issues in CSIT |release| for Honeycomb performance
-tests in VIRL:
-
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
-| # | Issue | Jira ID | Description |
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
-| 1 | Intermittent failures in Honeycomb startup | HC2VPP-199 | During test setup Honeycomb sometimes fails to start. Does not appear to |
-| | | | affect measured results, only fails test execution ocassionally. |
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
diff --git a/docs/report/honeycomb_performance_tests/documentation.rst b/docs/report/honeycomb_performance_tests/documentation.rst deleted file mode 100644 index 6b15bde6ee..0000000000 --- a/docs/report/honeycomb_performance_tests/documentation.rst +++ /dev/null @@ -1,5 +0,0 @@ -Documentation
-=============
-
-`CSIT Honeycomb Performance Tests Documentation`_ contains detailed
-functional description and input parameters for each test case.
diff --git a/docs/report/honeycomb_performance_tests/index.rst b/docs/report/honeycomb_performance_tests/index.rst deleted file mode 100644 index 3177494395..0000000000 --- a/docs/report/honeycomb_performance_tests/index.rst +++ /dev/null @@ -1,11 +0,0 @@ -Honeycomb Performance Tests -=========================== - -.. toctree:: - - overview - csit_release_notes - test_environment - documentation - test_result_data - diff --git a/docs/report/honeycomb_performance_tests/overview.rst b/docs/report/honeycomb_performance_tests/overview.rst deleted file mode 100644 index ee9788ed26..0000000000 --- a/docs/report/honeycomb_performance_tests/overview.rst +++ /dev/null @@ -1,122 +0,0 @@ -Overview -======== - -Tested Physical Topologies --------------------------- - -CSIT VPP performance tests are executed on physical baremetal servers hosted by -LF FD.io project. Testbed physical topology is shown in the figure below.:: - - +------------------------+ +------------------------+ - | | | | - | +------------------+ | | +------------------+ | - | | | | | | | | - | | <-----------------> | | - | | DUT1 | | | | DUT2 | | - | +--^---------------+ | | +---------------^--+ | - | | | | | | - | | SUT1 | | SUT2 | | - +------------------------+ +------------------^-----+ - | | - | | - | +-----------+ | - | | | | - +------------------> TG <------------------+ - | | - +-----------+ - -SUT1 runs VPP SW application in Linux user-mode as a -Device Under Test (DUT), and a python script to generate traffic. SUT2 and TG -are unused. -sical connectivity between SUTs and to TG is provided using -different NIC model. Currently installed NIC models include: - -Performance tests involve sending Netconf requests over localhost to the -Honeycomb listener port, and measuring response time. - -Note that reported performance results are specific to the SUTs tested. -Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with -other CPUs are likely to yield different results. - -For detailed LF FD.io test bed specification and physical topology please refer -to `LF FDio CSIT testbed wiki page -<https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_. - -Performance Tests Coverage --------------------------- - -As of right now, there is only a single Honeycomb performance test. Measuring -response time for a simple read operation, performed synchronously and using -single (not batch) requests. - -Currently the tests do not trigger automatically, but can be run on-demand from -the hc2vpp project. - -Performance Tests Naming ------------------------- - -CSIT |release| follows a common structured naming convention for all -performance and system functional tests, introduced in CSIT |release-1|. - -The naming should be intuitive for majority of the tests. Complete -description of CSIT test naming convention is provided on `CSIT test naming wiki -<https://wiki.fd.io/view/CSIT/csit-test-naming>`_. - -Here few illustrative examples of the new naming usage for performance test -suites: - -#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** - - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching - with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE - on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline - switching with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel - x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery. - - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput - discovery. - -#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, - P2V2P, NIC2VMchain2NIC, P2V2V2P** - - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation- - VirtPortConfig-VMconfig-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports - of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from four vhost interfaces and two VMs, NDR throughput - discovery. - -Methodology: Multi-Core ------------------------ - -**Multi-core Test** - CSIT |release| multi-core tests are executed in the -following thread and core configurations: - -#. 1t - 1 Honeycomb Netconf thread on 1 CPU physical core. -#. 8t - 8 Honeycomb Netconf thread on 8 CPU physical core. -#. 16t - 16 Honeycomb Netconf thread on 16 CPU physical core. - -Traffic generator also uses multiple threads/cores, to simulate multiple -Netconf clients accessing the Honeycomb server. - -Methodology: Performance measurement ------------------------------------- - -The following values are measured and reported in tests: - -- Average request rate. Averaged over the entire test duration, over all client - threads. Negative replies (if any) are not counted and are reported separately. diff --git a/docs/report/honeycomb_performance_tests/test_environment.rst b/docs/report/honeycomb_performance_tests/test_environment.rst deleted file mode 100644 index db187b2c1b..0000000000 --- a/docs/report/honeycomb_performance_tests/test_environment.rst +++ /dev/null @@ -1,22 +0,0 @@ -Test Environment -================ - -To execute performance tests, there are three identical testbeds, each testbed -consists of two DUTs and one TG. - -Server HW Configuration ------------------------ - -See `Performance HW Configuration <../vpp_performance_tests/test_environment.html>`_ - -Additionally, configuration for the Honeycomb client: - - -**Honeycomb Startup Command** - -Use the server mode JIT compiler, increase the default memory size, -metaspace size, and enable NUMA optimizations for the JVM. - -:: - - $ java -server -Xms128m -Xmx512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=512m -XX:+UseNUMA -XX:+UseParallelGC diff --git a/docs/report/honeycomb_performance_tests/test_result_data.rst b/docs/report/honeycomb_performance_tests/test_result_data.rst deleted file mode 100644 index 9e791d6e88..0000000000 --- a/docs/report/honeycomb_performance_tests/test_result_data.rst +++ /dev/null @@ -1,19 +0,0 @@ -Test Result Data -================ - -This section includes summary of Netconf read operation performance. -Performance is reported for Honeycomb running in multiple configurations of -netconf thread(s) and their physical CPU core(s) placement, and for different -read operation targets. - -.. note:: - - Test results have been generated by - `FD.io test executor honeycomb performance jobs`_ with Robot Framework - result files csit-vpp-perf-\*.zip `archived here <../../_static/archive/>`_. - -Honeycomb + Netconf -=================== - -No new data has been generated in the course of this release. Results from -the last successful run are available in `17.07 release report <https://docs.fd.io/csit/rls1707/report/honeycomb_performance_tests/test_result_data.html>`_.
\ No newline at end of file diff --git a/docs/report/index.rst b/docs/report/index.rst index bbcd5cbab8..abd4178833 100644 --- a/docs/report/index.rst +++ b/docs/report/index.rst @@ -1,18 +1,15 @@ -CSIT 18.01 -========== +CSIT 18.01.2 +============ .. toctree:: :numbered: introduction/index vpp_performance_tests/index - dpdk_performance_tests/index - honeycomb_performance_tests/index vpp_functional_tests/index honeycomb_functional_tests/index - vpp_unit_tests/index nsh_sfc_functional_tests/index detailed_test_results/index test_configuration/index test_operational_data/index - csit_framework_documentation/index
\ No newline at end of file + csit_framework_documentation/index diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst index 994a53e6d0..48c20ad132 100644 --- a/docs/report/introduction/general_notes.rst +++ b/docs/report/introduction/general_notes.rst @@ -40,20 +40,6 @@ is listed separately, as follows: orchestrated by Kubernetes, with `Ligato <https://github.com/ligato>`_ for container networking. TRex is used as a traffic generator. -#. **DPDK Performance** - VPP is using DPDK code to control and drive - the NICs and physical interfaces. Tests are used as a baseline to - profile performance of the DPDK sub-system. DPDK tests are executed in - physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for - Phy-to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE - interfaces, range of multi-thread and multi-core configurations. - Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic - generator. - -#. **Honeycomb Performance** - Honeycomb performance tests are executed in - physical FD.io testbeds, focusing on the performance of Honeycomb management - and programming functionality of VPP. Tests cover a range of CRUD operations - executed against VPP. - #. **VPP Functional** - VPP functional tests are executed in virtual FD.io testbeds focusing on VPP packet processing functionality, including network data plane and in -line control plane. Tests cover vNIC-to-vNIC diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index 22bc29f071..4e290639ff 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -36,21 +36,6 @@ CSIT |release| report contains following main sections and sub-sections: test job executions; *Test Environment* - environment description; *Documentation* - CSIT source code documentation for VPP performance tests. -#. **DPDK Performance Tests** - DPDK performance tests executed in - physical FD.io testbeds; *Overview* - tested topologies, test coverage; - *CSIT Release Notes* - changes in CSIT |release|, any known CSIT issues; - *Packet Throughput Graphs* and *Packet Latency Graphs* - - plotted NDR, PDR throughput and latency results from multiple test job - executions; *Test Environment* - environment description; *Documentation* - - CSIT source code documentation for DPDK performance tests. - -#. **Honeycomb Performance Tests** - Honeycomb performance tests executed in - physical FD.io testbeds; *Overview* - tested topologies, test coverage - and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, - added tests, environment or methodology changes, known CSIT issues; - *Test Environment* - environment description; *Documentation* - source - code documentation for Honeycomb performance tests. - #. **VPP Functional Tests** - VPP functional tests executed in virtual FD.io testbeds; *Overview* - tested virtual topologies, test coverage and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, added @@ -65,12 +50,6 @@ CSIT |release| report contains following main sections and sub-sections: *Test Environment* - environment description ; *Documentation* - source code documentation for Honeycomb functional tests. -#. **VPP Unit Tests** - refers to VPP functional unit tests executed as - part of vpp make test verify option within the FD.io VPP project; listed in - this report to give a more complete view about executed VPP functional tests; - *Overview* - short overview of unit test framework and executed tests; - *Documentation* - source code documentation of VPP unit tests. - #. **NSH_SFC Functional Tests** - NSH_SFC functional tests executed in virtual FD.io testbeds; *Overview* - tested virtual topologies, test coverage and naming specifics; *CSIT Release Notes* - changes in CSIT @@ -80,7 +59,7 @@ CSIT |release| report contains following main sections and sub-sections: #. **Detailed Test Results** - auto-generated results from CSIT jobs executions using CSIT Robot Framework output files as source data; *VPP - Performance Results*, *DPDK Performance Results*, *VPP Functional + Performance Results*, *VPP Functional Results*, *Honeycomb Functional Results*, *VPPtest Functional Results*. #. **Test Configuration** - auto-generated DUT configuration data from CSIT jobs diff --git a/docs/report/test_configuration/index.rst b/docs/report/test_configuration/index.rst index 3963896b82..8c231a85cd 100644 --- a/docs/report/test_configuration/index.rst +++ b/docs/report/test_configuration/index.rst @@ -7,4 +7,6 @@ .. toctree:: vpp_performance_configuration/index + vpp_mrr_configuration/index vpp_functional_configuration/index + vpp_functional_configuration_centos/index diff --git a/docs/report/test_configuration/vpp_functional_configuration/index.rst b/docs/report/test_configuration/vpp_functional_configuration/index.rst index aab94bd31c..36f72eba04 100644 --- a/docs/report/test_configuration/vpp_functional_configuration/index.rst +++ b/docs/report/test_configuration/vpp_functional_configuration/index.rst @@ -1,5 +1,5 @@ -VPP Functional Test Configs -=========================== +VPP Functional Test Configs - Ubuntu +==================================== .. note:: diff --git a/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst new file mode 100644 index 0000000000..cf416ac48e --- /dev/null +++ b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst @@ -0,0 +1,12 @@ +VPP Functional Test Configs - CentOS +==================================== + +.. note:: + + Data sources for reported test results: i) `FD.io test executor vpp + functional jobs`_, ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + + vpp_functional_configuration_centos diff --git a/docs/report/test_configuration/vpp_mrr_configuration/index.rst b/docs/report/test_configuration/vpp_mrr_configuration/index.rst new file mode 100644 index 0000000000..72b6ec01cd --- /dev/null +++ b/docs/report/test_configuration/vpp_mrr_configuration/index.rst @@ -0,0 +1,12 @@ +VPP MRR Test Configs +==================== + +.. note:: + + Data sources for reported test results: i) `FD.io test executor vpp + performance jobs`_, ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + + vpp_mrr_configuration diff --git a/docs/report/vpp_performance_tests/impact_meltdown/index.rst b/docs/report/vpp_performance_tests/impact_meltdown/index.rst deleted file mode 100644 index 35ecb7453c..0000000000 --- a/docs/report/vpp_performance_tests/impact_meltdown/index.rst +++ /dev/null @@ -1,164 +0,0 @@ -Impact of Meltdown Patches -========================== - -Following sections list changes to VPP throughput performance after applying -patches addressing security vulnerabilities referred to as Meltdown -(Variant3: Rogue Data Cache Load). Incremental kernel patches are -applied for Ubuntu 16.04LTS as documented on -`Ubuntu SpectreAndMeltdown page <https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown>`_. -Detailed listing of used software versions and patches is documented in -:ref:`test_environment`. - -NDR and PDR packet throughput results are compared for 1-core/1-thread, -2-cores/2-threads and 4-cores/4-threads VPP configurations, with -reference performance numbers coming from tests without the Meltdown -patches. Tables show test results grouped into Best 20 changes (minimal -performance impact), followed by Worst 20 changes (maximal performance -impact). All results are also provided in downloadable CSV and pretty -ASCII formats. - -NDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-ndr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-ndr-1t1c-top.csv} - } - -NDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-ndr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-ndr-1t1c-bottom.csv} - } - -.. only:: html - - NDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all NDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-impact-ndr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-impact-ndr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-impact-ndr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-impact-ndr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-impact-ndr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-impact-ndr-4t4c-full.txt>`_. - -PDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-pdr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-pdr-1t1c-top.csv} - } - -PDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-pdr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-pdr-1t1c-bottom.csv} - } - -.. only:: html - -PDR Throughput: All Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all PDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-impact-pdr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-impact-pdr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-impact-pdr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-impact-pdr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-impact-pdr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-impact-pdr-4t4c-full.txt>`_. diff --git a/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst b/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst deleted file mode 100644 index eec38e8698..0000000000 --- a/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst +++ /dev/null @@ -1,167 +0,0 @@ -Impact of SpectreAndMeltdown Patches -==================================== - -Following sections list changes to VPP throughput performance after -applying patches addressing security vulnerabilities referred to as: -Meltdown (Variant3: Rogue Data Cache Load) and Spectre (Variant1: Bounds -Check Bypass; Variant2: Branch Target Injection) security -vulnerabilities. Incremental kernel patches for Ubuntu 16.04 LTS as -documented on -`Ubuntu SpectreAndMeltdown page <https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown>`_. -For Spectre additional Processor microcode and BIOS firmware changes are -applied. Detailed listing of used software versions and patches is -documented in :ref:`test_environment`. - -NDR and PDR packet throughput results are compared for 1-core/1-thread, -2-cores/2-threads and 4-cores/4-threads VPP configurations, with -reference performance numbers coming from tests without the Meltdown -patches. Tables show test results grouped into Best 20 changes (minimal -performance impact), followed by Worst 20 changes (maximal performance -impact). All results are also provided in downloadable CSV and pretty -ASCII formats. - -NDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-top.csv} - } - -NDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-bottom.csv} - } - -.. only:: html - - NDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all NDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-spectre-impact-ndr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-spectre-impact-ndr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-spectre-impact-ndr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-spectre-impact-ndr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-spectre-impact-ndr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-spectre-impact-ndr-4t4c-full.txt>`_. - -PDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-top.csv} - } - -PDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-bottom.csv} - } - -.. only:: html - - PDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all PDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-spectre-impact-pdr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-spectre-impact-pdr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-spectre-impact-pdr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-spectre-impact-pdr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-spectre-impact-pdr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-spectre-impact-pdr-4t4c-full.txt>`_. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst index 50074e1b6a..4507978b66 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst @@ -27,6 +27,7 @@ TGint2-to-SUT2-to-SUT1-to-TGint1. l2 ip4 ip6 + srv6 ip4_tunnels ip6_tunnels vm_vhost diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/srv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/srv6.rst new file mode 100644 index 0000000000..163d892cde --- /dev/null +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/srv6.rst @@ -0,0 +1,48 @@ +SRv6 +==== + +This section includes summary graphs of VPP Phy-to-Phy packet latency +with SRv6 measured at 50% of discovered NDR throughput +rate. Latency is reported for VPP running in multiple configurations of +VPP worker thread(s), a.k.a. VPP data plane thread(s), and their +physical CPU core(s) placement. + +VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-srv6-ndrdisc-lat50.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-1t1c-ethip6-srv6-ndrdisc-lat50} + \label{fig:78B-1t1c-ethip6-srv6-ndrdisc-lat50} + \end{figure} + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. + +VPP packet latency in 2t2c setup (2thread, 2core) is presented in the graph below. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-srv6-ndrdisc-lat50.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-2t2c-ethip6-srv6-ndrdisc-lat50} + \label{fig:78B-2t2c-ethip6-srv6-ndrdisc-lat50} + \end{figure} + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst index 6bfe4fe1af..ec3c9f9173 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst @@ -39,6 +39,7 @@ and their indices. l2 ip4 ip6 + srv6 ip4_tunnels ip6_tunnels vm_vhost diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/srv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/srv6.rst new file mode 100644 index 0000000000..0df40ac636 --- /dev/null +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/srv6.rst @@ -0,0 +1,99 @@ +SRv6 +==== + +Following sections include summary graphs of VPP Phy-to-Phy performance +with SRv6, including NDR throughput (zero packet loss) +and PDR throughput (<0.5% packet loss). Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR 78B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-srv6-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-1t1c-ethip6-srv6-ndrdisc} + \label{fig:78B-1t1c-ethip6-srv6-ndrdisc} + \end{figure} + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. + +VPP NDR 78B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-srv6-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-2t2c-ethip6-srv6-ndrdisc} + \label{fig:78B-2t2c-ethip6-srv6-ndrdisc} + \end{figure} + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR 78B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-srv6-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-1t1c-ethip6-srv6-pdrdisc} + \label{fig:78B-1t1c-ethip6-srv6-pdrdisc} + \end{figure} + +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. + +VPP PDR 78B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-srv6-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-2t2c-ethip6-srv6-pdrdisc} + \label{fig:78B-2t2c-ethip6-srv6-pdrdisc} + \end{figure} + +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.rst new file mode 100644 index 0000000000..4fb8791dff --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.rst @@ -0,0 +1,66 @@ +Container memif Connections +=========================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-container-memif-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-memif-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-container-memif-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy L2 Ethernet Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/container_memif?h=rls1804>`_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-container-memif-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-memif-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-container-memif-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy L2 Ethernet Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/container_memif?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst new file mode 100644 index 0000000000..4e8ff4cd25 --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst @@ -0,0 +1,110 @@ +Container Orchestrated Topologies +================================= + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 and 10ge2p1x710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-container-orchestrated-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-orchestrated-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-container-orchestrated-tsa-ndrdisc} + \end{figure} + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/kubernetes/perf/container_memif?h=rls1804>`_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x710-64B-container-orchestrated-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-container-orchestrated-tsa-ndrdisc} + \label{fig:10ge2p1x710-64B-container-orchestrated-tsa-ndrdisc} + \end{figure} + +*Figure 2. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/kubernetes/perf/container_memif?h=rls1804>`_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 and 10ge2p1x710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-container-orchestrated-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-orchestrated-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-container-orchestrated-tsa-pdrdisc} + \end{figure} + +*Figure 3. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/kubernetes/perf/container_memif?h=rls1804>`_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x710-64B-container-orchestrated-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-container-orchestrated-tsa-pdrdisc} + \label{fig:10ge2p1x710-64B-container-orchestrated-tsa-pdrdisc} + \end{figure} + +*Figure 4. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/kubernetes/perf/container_memif?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst index ddce548b6c..dca77c5b7b 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst @@ -1,3 +1,5 @@ +.. _throughput_speedup_multi_core: + Throughput Speedup Multi-Core ============================= @@ -18,3 +20,10 @@ threaded VPP configurations relative to 1-core configurations. l2 ip4 ip6 + srv6 + ip4_tunnels + ip6_tunnels + vm_vhost + container_memif + container_orchestrated + ipsec diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst index af9bd255be..0b2b7ef791 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst @@ -31,24 +31,12 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-ip4-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip4 && grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip4 - $ grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - *Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip4?h=rls1804>`_. + NIC 40ge2p1xl710 ~~~~~~~~~~~~~~~~ @@ -65,24 +53,12 @@ NIC 40ge2p1xl710 \label{fig:40ge2p1xl710-64B-ip4-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip4 && grep -P '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip4 - $ grep -P '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - *Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip4?h=rls1804>`_. + PDR Throughput -------------- @@ -106,20 +82,8 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-ip4-tsa-pdrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip4 && grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip4 - $ grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - *Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip4?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4_tunnels.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4_tunnels.rst new file mode 100644 index 0000000000..001584b856 --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4_tunnels.rst @@ -0,0 +1,66 @@ +IPv4 Overlay Tunnels +==================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-ethip4-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-ethip4-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-ethip4-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip4_tunnels?h=rls1804>`_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-ethip4-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-ethip4-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-ethip4-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip4_tunnels?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst index 077b330652..e8c70d0c18 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst @@ -32,24 +32,12 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-78B-ip6-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip6 && grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip6 - $ grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - *Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip6?h=rls1804>`_. + NIC 40ge2p1xl710 ~~~~~~~~~~~~~~~~ @@ -66,24 +54,12 @@ NIC 40ge2p1xl710 \label{fig:40ge2p1xl710-78B-ip6-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip6 && grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip6 - $ grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - *Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip6?h=rls1804>`_. + PDR Throughput -------------- @@ -107,20 +83,8 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-78B-ip6-tsa-pdrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip6 && grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-pdrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip6 - $ grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-pdrdisc' 10ge2p1x520* - *Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized PDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip6?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6_tunnels.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6_tunnels.rst new file mode 100644 index 0000000000..3965242e8c --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6_tunnels.rst @@ -0,0 +1,66 @@ +IPv6 Overlay Tunnels +==================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-78B-ethip6-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-ethip6-tsa-ndrdisc} + \label{fig:10ge2p1x520-78B-ethip6-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip6_tunnels?h=rls1804>`_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-78B-ethip6-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-ethip6-tsa-pdrdisc} + \label{fig:10ge2p1x520-78B-ethip6-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/ip6_tunnels?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ipsec.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ipsec.rst new file mode 100644 index 0000000000..a5a4d7d1bc --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ipsec.rst @@ -0,0 +1,71 @@ +IPSec Crypto HW: IP4 Routed-Forwarding +====================================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +VPP IPSec encryption is accelerated using DPDK cryptodev +library driving Intel Quick Assist (QAT) crypto PCIe hardware cards. +Performance is reported for VPP running in multiple configurations of +VPP worker thread(s), a.k.a. VPP data plane thread(s), and their +physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 40ge2p1xl710 network interface card. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/40ge2p1xl710-64B-ipsechw-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-ipsechw-tsa-ndrdisc} + \label{fig:40ge2p1xl710-64B-ipsechw-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy IPSEC HW.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/crypto?h=rls1804>`_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 40ge2p1xl710 network interface card. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +VPP PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/40ge2p1xl710-64B-ipsechw-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-ipsechw-tsa-pdrdisc} + \label{fig:40ge2p1xl710-64B-ipsechw-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy IPSEC HW.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/crypto?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst index 2df3fb6229..5db8eba307 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst @@ -31,23 +31,12 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-l2-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/l2 && grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" tests/vpp/perf/l2/10ge2p1x520* - *Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy L2 Ethernet Switching.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/l2?h=rls1804>`_. + NIC 40ge2p1xl710 ~~~~~~~~~~~~~~~~ @@ -64,23 +53,12 @@ NIC 40ge2p1xl710 \label{fig:40ge2p1xl710-64B-l2-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/l2 && grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" 40ge2p1xl710* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" tests/vpp/perf/l2/40ge2p1xl710* - *Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy L2 Ethernet Switching.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/l2?h=rls1804>`_. + PDR Throughput -------------- @@ -104,19 +82,8 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-l2-tsa-pdrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/l2 && grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*pdrdisc" 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*pdrdisc" tests/vpp/perf/l2/10ge2p1x520* - *Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/l2?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/srv6.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/srv6.rst new file mode 100644 index 0000000000..4ece4d538e --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/srv6.rst @@ -0,0 +1,68 @@ +SRv6 +==== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. Input data +used for the graphs comes from Phy-to-Phy 78B performance tests with VPP +SRv6, including NDR throughput (zero packet loss) and +PDR throughput (<0.5% packet loss). + +NDR Throughput +-------------- + +VPP NDR 78B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-78B-srv6-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-srv6-tsa-ndrdisc} + \label{fig:10ge2p1x520-78B-srv6-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. + +PDR Throughput +-------------- + +VPP PDR 78B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. PDR +measured for 0.5% packet loss ratio. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-78B-srv6-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-srv6-tsa-pdrdisc} + \label{fig:10ge2p1x520-78B-srv6-tsa-pdrdisc} + \end{figure} + +*Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/srv6?h=rls1804>`_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/vm_vhost.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/vm_vhost.rst new file mode 100644 index 0000000000..9bd49b42ab --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/vm_vhost.rst @@ -0,0 +1,229 @@ +VM vhost Connections +==================== +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. Input data +used for the graphs comes from Phy-to-Phy 64B performance tests with +VM vhost-user, including NDR throughput (zero packet loss) and +PDR throughput (<0.5% packet loss). + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520, 10ge2p1x710 and 40ge2p1xl710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-vhost-sel1-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel1-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel1-tsa-ndrdisc} + \end{figure} + +*Figure 1a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-vhost-sel2-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel2-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel2-tsa-ndrdisc} + \end{figure} + +*Figure 1b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x710-64B-vhost-sel2-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-vhost-sel2-tsa-ndrdisc} + \label{fig:10ge2p1x710-64B-vhost-sel2-tsa-ndrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/40ge2p1xl710-64B-vhost-sel1-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel1-tsa-ndrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel1-tsa-ndrdisc} + \end{figure} + +*Figure 3a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/40ge2p1xl710-64B-vhost-sel2-tsa-ndrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel2-tsa-ndrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel2-tsa-ndrdisc} + \end{figure} + +*Figure 3b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520, 10ge2p1x710 and 40ge2p1xl710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-vhost-sel1-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel1-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel1-tsa-pdrdisc} + \end{figure} + +*Figure 4a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x520-64B-vhost-sel2-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel2-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel2-tsa-pdrdisc} + \end{figure} + +*Figure 4b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/10ge2p1x710-64B-vhost-sel2-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-vhost-sel2-tsa-pdrdisc} + \label{fig:10ge2p1x710-64B-vhost-sel2-tsa-pdrdisc} + \end{figure} + +*Figure 5. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/40ge2p1xl710-64B-vhost-sel1-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel1-tsa-pdrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel1-tsa-pdrdisc} + \end{figure} + +*Figure 6a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. + +.. raw:: html + + <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/40ge2p1xl710-64B-vhost-sel2-tsa-pdrdisc.html"></iframe> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel2-tsa-pdrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel2-tsa-pdrdisc} + \end{figure} + +*Figure 6b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1804>`_. diff --git a/docs/report/vpp_unit_tests/documentation.rst b/docs/report/vpp_unit_tests/documentation.rst deleted file mode 100644 index 304db1121b..0000000000 --- a/docs/report/vpp_unit_tests/documentation.rst +++ /dev/null @@ -1,6 +0,0 @@ -Documentation
-=============
-
-For complete description of the VPP Test Framework including anatomy of a test
-case and detailed documentation of existing VPP unit test cases please refer
-to the `VPP test framework documentation`_.
diff --git a/docs/report/vpp_unit_tests/index.rst b/docs/report/vpp_unit_tests/index.rst deleted file mode 100644 index 525eb1230d..0000000000 --- a/docs/report/vpp_unit_tests/index.rst +++ /dev/null @@ -1,8 +0,0 @@ -VPP Unit Tests -============== - -.. toctree:: - - overview - documentation - diff --git a/docs/report/vpp_unit_tests/overview.rst b/docs/report/vpp_unit_tests/overview.rst deleted file mode 100644 index 9472680aa5..0000000000 --- a/docs/report/vpp_unit_tests/overview.rst +++ /dev/null @@ -1,87 +0,0 @@ -Overview
-========
-
-.. note::
-
- This section includes an abbreviated version of the VPP Test Framework
- overview maintained within the VPP project. Complete overview can be found
- in `VPP test framework documentation`_.
-
-VPP Unit Test Framework
------------------------
-
-VPP Test Framework is used to ease writing, running and debugging unit tests
-for the VPP. It is based on python as a high level language to allow rapid
-test development. scapy\_ is used as a tool for creating and dissecting
-packets.
-
-VPP Test Framework does not send any packets to VPP directly. Traffic is
-instead injected using VPP packet-generator interfaces. Packets are written
-into a temporary .pcap file, which is then read by the VPP code with packets
-getting injected into the VPP processing nodes.
-
-Similarly, VPP does not send any packets to VPP Test Framework directly.
-Instead, VPP packet capture feature is used to capture and write packets to a
-temporary .pcap file, which is then read and analyzed by the VPP Test
-Framework.
-
-For complete description of the VPP Test Framework including anatomy of a test
-case and detailed documentation of existing VPP unit test cases please refer
-to the `VPP test framework documentation`_
-
-Unit Tests Coverage
--------------------
-
-Following VPP functional test areas are covered in VPP unit test code included
-in VPP rls1710 with results listed in this report:
-
-- ACL Security - stateful and stateless security-groups access-control-lists.
-- APIs - VAPI, VOM, PAPI, JVPP.
-- ARP - ARP, proxy ARP, static arp.
-- BFD - API, Authentication, Authentication Change, CLI.
-- BFD IPv4 - sessions operation.
-- BFD IPv6 - sessions operation.
-- BIER - Bit Indexed Explicit Replication.
-- Classifier - classification with IP ACL, MAC ACL, IP PBR.
-- Container Integration - IPv4, IPv6 local-spoof connectivity tests.
-- CRUD Loopback - create, read, update, delete Loopback interfaces.
-- DHCP - DHCPv4/v6 Client and Proxy.
-- Distributed Virtual Router.
-- DS-Lite Softwire - softwire termination.
-- FIB - baseline and scale tests.
-- Flowprobe.
-- Geneve Tunnels.
-- GRE Tunnels - GRE IPv4/IPv6 tunnel, L2, VRF tests.
-- GTPU Tunnels - baseline GTPU tests.
-- IP Multicast Routing - IPv4/IPv6 multicast replication, connected source check.
-- IPSec - baseline IPSec sanity tests.
-- IPv4 FIB CRUD - add/update/delete IPv4 routes.
-- IPv4 Routing.
-- IP4 VRF Multi-instance - create, read, update, delete and verify IPv4 VRFs.
-- IPv6 Routing - baseline FIB operations, NS/RS exception handling.
-- IP6 VRF Multi-instance - create, read, update, delete and verify IPv6 VRFs.
-- IRB Integrated Routing-Bridging.
-- Kube-proxy - data plane NAT tests.
-- L2 FIB CRUD - add/update/delete L2 MAC entries.
-- L2BD Multi-instance.
-- L2BD Switching - L2 Bridge-Domain baseline tests incl. single- and dual-loop.
-- L2XC Multi-instance - L2 cross-connect multi-instance tests.
-- L2XC Switching - L2 cross-connect baseline tests incl. single- and dual-loop.
-- LISP Tunnels - basic LISP tests.
-- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6.
-- MACIP Access Control - ingress access control for IPv4, IPv6 with L2BDP and IP routing.
-- MAP Softwires - softwire termination.
-- MFIB Multicast FIB.
-- MPLS Switching - MPLS baseline, prefix independent convergence for MPLS PE.
-- NAT44 - NAT44 tests, IPFIX logging, VRF awareness, deterministic CGNAT.
-- NAT64 - NAT64 static and dynamic translation tests.
-- P2P Ethernet Subinterface.
-- PPPoE Encapsulation.
-- SPAN Switch Port Analyzer - packet mirroring.
-- SRv6 Routing - Segment Routing IPv6 tests.
-- TCP/IP Stack - unit tests, builtin client/server transfers.
-- UDP Stack - unit tests.
-- VTR VLAN Tag Rewrites - VLAN tag rewrite tests.
-- VXLAN Tunnels - baseline VXLAN tests including multicast.
-- VXLAN-GPE Tunnels - baseline VXLAN-GPE tunneling including multicast.
-- Other Tests - ping, session, template verification, timer tests.
|