From e01470ec9038338409a494a2652eecabf4394578 Mon Sep 17 00:00:00 2001 From: Tibor Frank Date: Tue, 29 May 2018 10:45:47 +0200 Subject: CSIT-1105: Prepare and generate 18.01.2 report Change-Id: Iebda4fd10701c27512b443c14b2aeef314003d58 Signed-off-by: Tibor Frank --- .../dpdk_performance_results/index.rst | 12 - docs/report/detailed_test_results/index.rst | 5 +- .../vpp_functional_results/index.rst | 4 +- .../vpp_functional_results_centos/index.rst | 12 + .../vpp_mrr_results/index.rst | 12 + .../vpp_unittest_results/index.rst | 7 - .../vpp_unittest_results/vpp_unittest_results.rst | 1178 -------------------- .../dpdk_performance_tests/csit_release_notes.rst | 23 - .../dpdk_performance_tests/documentation.rst | 5 - docs/report/dpdk_performance_tests/index.rst | 12 - docs/report/dpdk_performance_tests/overview.rst | 240 ---- .../packet_latency_graphs/index.rst | 28 - .../packet_latency_graphs/ip4.rst | 73 -- .../packet_latency_graphs/l2.rst | 74 -- .../packet_throughput_graphs/index.rst | 39 - .../packet_throughput_graphs/ip4.rst | 150 --- .../packet_throughput_graphs/l2.rst | 150 --- .../dpdk_performance_tests/test_environment.rst | 68 -- .../csit_release_notes.rst | 20 - .../honeycomb_performance_tests/documentation.rst | 5 - docs/report/honeycomb_performance_tests/index.rst | 11 - .../honeycomb_performance_tests/overview.rst | 122 -- .../test_environment.rst | 22 - .../test_result_data.rst | 19 - docs/report/index.rst | 9 +- docs/report/introduction/general_notes.rst | 14 - docs/report/introduction/overview.rst | 23 +- docs/report/test_configuration/index.rst | 2 + .../vpp_functional_configuration/index.rst | 4 +- .../vpp_functional_configuration_centos/index.rst | 12 + .../vpp_mrr_configuration/index.rst | 12 + .../impact_meltdown/index.rst | 164 --- .../impact_spectreandmeltdown/index.rst | 167 --- .../packet_latency_graphs/index.rst | 1 + .../packet_latency_graphs/srv6.rst | 48 + .../packet_throughput_graphs/index.rst | 1 + .../packet_throughput_graphs/srv6.rst | 99 ++ .../container_memif.rst | 66 ++ .../container_orchestrated.rst | 110 ++ .../throughput_speedup_multi_core/index.rst | 9 + .../throughput_speedup_multi_core/ip4.rst | 54 +- .../throughput_speedup_multi_core/ip4_tunnels.rst | 66 ++ .../throughput_speedup_multi_core/ip6.rst | 54 +- .../throughput_speedup_multi_core/ip6_tunnels.rst | 66 ++ .../throughput_speedup_multi_core/ipsec.rst | 71 ++ .../throughput_speedup_multi_core/l2.rst | 51 +- .../throughput_speedup_multi_core/srv6.rst | 68 ++ .../throughput_speedup_multi_core/vm_vhost.rst | 229 ++++ docs/report/vpp_unit_tests/documentation.rst | 6 - docs/report/vpp_unit_tests/index.rst | 8 - docs/report/vpp_unit_tests/overview.rst | 87 -- 51 files changed, 921 insertions(+), 2871 deletions(-) delete mode 100644 docs/report/detailed_test_results/dpdk_performance_results/index.rst create mode 100644 docs/report/detailed_test_results/vpp_functional_results_centos/index.rst create mode 100644 docs/report/detailed_test_results/vpp_mrr_results/index.rst delete mode 100644 docs/report/detailed_test_results/vpp_unittest_results/index.rst delete mode 100644 docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst delete mode 100644 docs/report/dpdk_performance_tests/csit_release_notes.rst delete mode 100644 docs/report/dpdk_performance_tests/documentation.rst delete mode 100644 docs/report/dpdk_performance_tests/index.rst delete mode 100644 docs/report/dpdk_performance_tests/overview.rst delete mode 100644 docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst delete mode 100644 docs/report/dpdk_performance_tests/packet_latency_graphs/ip4.rst delete mode 100644 docs/report/dpdk_performance_tests/packet_latency_graphs/l2.rst delete mode 100644 docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst delete mode 100644 docs/report/dpdk_performance_tests/packet_throughput_graphs/ip4.rst delete mode 100644 docs/report/dpdk_performance_tests/packet_throughput_graphs/l2.rst delete mode 100644 docs/report/dpdk_performance_tests/test_environment.rst delete mode 100644 docs/report/honeycomb_performance_tests/csit_release_notes.rst delete mode 100644 docs/report/honeycomb_performance_tests/documentation.rst delete mode 100644 docs/report/honeycomb_performance_tests/index.rst delete mode 100644 docs/report/honeycomb_performance_tests/overview.rst delete mode 100644 docs/report/honeycomb_performance_tests/test_environment.rst delete mode 100644 docs/report/honeycomb_performance_tests/test_result_data.rst create mode 100644 docs/report/test_configuration/vpp_functional_configuration_centos/index.rst create mode 100644 docs/report/test_configuration/vpp_mrr_configuration/index.rst delete mode 100644 docs/report/vpp_performance_tests/impact_meltdown/index.rst delete mode 100644 docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst create mode 100644 docs/report/vpp_performance_tests/packet_latency_graphs/srv6.rst create mode 100644 docs/report/vpp_performance_tests/packet_throughput_graphs/srv6.rst create mode 100644 docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.rst create mode 100644 docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst create mode 100644 docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4_tunnels.rst create mode 100644 docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6_tunnels.rst create mode 100644 docs/report/vpp_performance_tests/throughput_speedup_multi_core/ipsec.rst create mode 100644 docs/report/vpp_performance_tests/throughput_speedup_multi_core/srv6.rst create mode 100644 docs/report/vpp_performance_tests/throughput_speedup_multi_core/vm_vhost.rst delete mode 100644 docs/report/vpp_unit_tests/documentation.rst delete mode 100644 docs/report/vpp_unit_tests/index.rst delete mode 100644 docs/report/vpp_unit_tests/overview.rst (limited to 'docs/report') diff --git a/docs/report/detailed_test_results/dpdk_performance_results/index.rst b/docs/report/detailed_test_results/dpdk_performance_results/index.rst deleted file mode 100644 index e6466d2e60..0000000000 --- a/docs/report/detailed_test_results/dpdk_performance_results/index.rst +++ /dev/null @@ -1,12 +0,0 @@ -DPDK Performance Results -=========================== - -.. note:: - - Data sources for reported test results: i) `FD.io test executor dpdk - performance jobs`_, ii) archived FD.io jobs test result `output files - <../../_static/archive/>`_. - -.. toctree:: - - dpdk_performance_results diff --git a/docs/report/detailed_test_results/index.rst b/docs/report/detailed_test_results/index.rst index d980a3532b..923f6c6f76 100644 --- a/docs/report/detailed_test_results/index.rst +++ b/docs/report/detailed_test_results/index.rst @@ -7,10 +7,9 @@ .. toctree:: vpp_performance_results/index + vpp_mrr_results/index cot_performance_results/index - dpdk_performance_results/index vpp_functional_results/index + vpp_functional_results_centos/index honeycomb_functional_results/index nshsfc_functional_results/index - vpp_unittest_results/index - vpp_http_server_perf_results/index diff --git a/docs/report/detailed_test_results/vpp_functional_results/index.rst b/docs/report/detailed_test_results/vpp_functional_results/index.rst index 4d7060bb54..5b82c489c4 100644 --- a/docs/report/detailed_test_results/vpp_functional_results/index.rst +++ b/docs/report/detailed_test_results/vpp_functional_results/index.rst @@ -1,5 +1,5 @@ -VPP Functional Results -====================== +VPP Functional Results - Ubuntu +=============================== .. note:: diff --git a/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst b/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst new file mode 100644 index 0000000000..e65b68b250 --- /dev/null +++ b/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst @@ -0,0 +1,12 @@ +VPP Functional Results - CentOS +=============================== + +.. note:: + + Data sources for reported test results: i) `FD.io test executor vpp + functional jobs`_, ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + + vpp_functional_results_centos diff --git a/docs/report/detailed_test_results/vpp_mrr_results/index.rst b/docs/report/detailed_test_results/vpp_mrr_results/index.rst new file mode 100644 index 0000000000..ce8262ba7c --- /dev/null +++ b/docs/report/detailed_test_results/vpp_mrr_results/index.rst @@ -0,0 +1,12 @@ +VPP MRR Results +=============== + +.. note:: + + Data sources for reported test results: i) `FD.io test executor vpp + performance jobs`_, ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + + vpp_mrr_results diff --git a/docs/report/detailed_test_results/vpp_unittest_results/index.rst b/docs/report/detailed_test_results/vpp_unittest_results/index.rst deleted file mode 100644 index f77153d6b9..0000000000 --- a/docs/report/detailed_test_results/vpp_unittest_results/index.rst +++ /dev/null @@ -1,7 +0,0 @@ -VPP Unit Test Results -===================== - -.. toctree:: - - vpp_unittest_results - diff --git a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst deleted file mode 100644 index 3e1fd5ac57..0000000000 --- a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst +++ /dev/null @@ -1,1178 +0,0 @@ -ACL Security Groups -``````````````````` -:: - - ============================================================================== - ACL plugin Test Case - ============================================================================== - ACL plugin version check; learn MACs OK - ACL create/delete test OK - permit ACL apply test OK - deny ACL apply test OK - VPP_624 permit ICMPv4 OK - VPP_624 permit ICMPv6 OK - VPP_624 deny ICMPv4 OK - VPP_624 deny ICMPv6 OK - permit TCPv4 OK - permit TCPv6 OK - permit UDPv4 OK - permit UDPv6 OK - deny TCPv4/v6 OK - deny UDPv4/v6 OK - verify add/dump acls OK - permit single TCPv4 OK - permit single UDPv4 OK - permit single TCPv6 OK - permit single UPPv6 OK - deny single TCPv4/v6 OK - deny single UDPv4/v6 OK - deny single UDPv4/v6, permit ip any, verify non-initial fragment blocked OK - VPP-687 zero length udp ipv4 packet OK - VPP-687 zero length udp ipv6 packet OK - permit TCPv4 + non-match range OK - permit TCPv6 + non-match range OK - permit UDPv4 + non-match range OK - permit UDPv6 + non-match range OK - deny TCPv4/v6 + non-match range OK - deny UDPv4/v6 + non-match range OK - - ============================================================================== - IRB Test Case - ============================================================================== - ACL plugin prepare OK - ACL IPv6 routed -> bridged, L2 ACL deny OK - ACL IPv6 routed -> bridged, L3 ACL deny OK - ACL IPv4 routed -> bridged, L2 ACL deny OK - ACL IPv4 routed -> bridged, L3 ACL deny OK - ACL IPv6 bridged -> routed, L2 ACL deny OK - ACL IPv6 bridged -> routed, L3 ACL deny OK - ACL IPv4 bridged -> routed, L2 ACL deny OK - ACL IPv4 bridged -> routed, L3 ACL deny OK - ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK - ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK - ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK - ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK - ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK - ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK - ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK - ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK - ACL IPv6+EH routed -> bridged, L2 ACL deny OK - ACL IPv6+EH routed -> bridged, L3 ACL deny OK - ACL IPv6+EH bridged -> routed, L2 ACL deny OK - ACL IPv6+EH bridged -> routed, L3 ACL deny OK - ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK - ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK - ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK - ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK - ACL IPv4+MF routed -> bridged, L2 ACL deny OK - ACL IPv4+MF routed -> bridged, L3 ACL deny OK - ACL IPv4+MF bridged -> routed, L2 ACL deny OK - ACL IPv4+MF bridged -> routed, L3 ACL deny OK - ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK - ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK - ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK - ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK - - ============================================================================== - ACL plugin connection-oriented extended testcases - ============================================================================== - Prepare the settings SKIP - IPv4: Basic conn timeout test reflect on ingress SKIP - IPv4: Basic conn timeout test reflect on egress SKIP - IPv4: reflect egress, clear conn SKIP - IPv4: reflect ingress, clear conn SKIP - IPv4: Idle conn behind active conn, reflect on ingress SKIP - IPv4: Idle conn behind active conn, reflect on egress SKIP - IPv6: Basic conn timeout test reflect on ingress SKIP - IPv6: Basic conn timeout test reflect on egress SKIP - IPv6: reflect egress, clear conn SKIP - IPv6: reflect ingress, clear conn SKIP - IPv6: Idle conn behind active conn, reflect on ingress SKIP - IPv6: Idle conn behind active conn, reflect on egress SKIP - Prepare for TCP session tests SKIP - IPv4: transient TCP session (incomplete 3WHS), ref. on ingress SKIP - IPv4: transient TCP session (incomplete 3WHS), ref. on egress SKIP - IPv4: established TCP session (complete 3WHS), ref. on ingress SKIP - IPv4: established TCP session (complete 3WHS), ref. on egress SKIP - IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP - IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP - IPv6: transient TCP session (incomplete 3WHS), ref. on ingress SKIP - IPv6: transient TCP session (incomplete 3WHS), ref. on egress SKIP - IPv6: established TCP session (complete 3WHS), ref. on ingress SKIP - IPv6: established TCP session (complete 3WHS), ref. on egress SKIP - IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP - IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP - - ============================================================================== - ACL on dot1q bridged subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1Q bridged traffic OK - IP6 ACL SubIf Dot1Q bridged traffic OK - - ============================================================================== - ACL on dot1ad bridged subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1AD bridged traffic OK - IP6 ACL SubIf Dot1AD bridged traffic OK - - ============================================================================== - ACL on dot1ad routed subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1AD routed traffic OK - IP4 ACL SubIf wrong tags Dot1AD routed traffic OK - IP6 ACL SubIf Dot1AD routed traffic OK - IP6 ACL SubIf wrong tags Dot1AD routed traffic OK - - ============================================================================== - ACL on dot1q routed subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1Q routed traffic OK - IP4 ACL SubIf wrong tags Dot1Q routed traffic OK - IP6 ACL SubIf Dot1Q routed traffic OK - IP6 ACL SubIf wrong tags Dot1Q routed traffic OK - -APIs -```` -:: - - ============================================================================== - VAPI test - ============================================================================== - run C VAPI tests SKIP - run C++ VAPI tests SKIP - - ============================================================================== - VPP Object Model Test - ============================================================================== - run C++ VOM tests SKIP - - ============================================================================== - PAPI Test Case - ============================================================================== - show version OK - show version - invalid parameters OK - u8 array OK - - ============================================================================== - PAPI Message parsing Test Case - ============================================================================== - New compound type with array OK - Add new types OK - Add new types 2 OK - Add new message object OK - New message with array OK - Argument name OK - VLA with aribtrary length field placement OK - Message to byte encoding OK - Nested array type OK - Old style VLA array OK - Old VLA compound type OK - Old VLA array arbitrary placement OK - Old VLA u32 OK - Simple array OK - - ============================================================================== - JVPP Core Test Case - ============================================================================== - JVPP Acl Callback Api Test Case OK - JVPP Acl Future Api Test Case OK - JVPP Core Callback Api Test Case OK - JVPP Core Future Api Test Case OK - JVPP Ioamexport Callback Api Test Case OK - JVPP Ioamexport Future Api Test Case OK - JVPP Ioampot Callback Api Test Case OK - JVPP Ioampot Future Api Test Case OK - JVPP Ioamtrace Callback Api Test Case OK - JVPP Ioamtrace Future Api Test Case OK - JVPP Snat Callback Api Test Case OK - JVPP Snat Future Api Test Case OK - -ARP -``` -:: - - ============================================================================== - ARP Test Case - ============================================================================== - ARP OK - ARP Duplicates OK - ARP Static OK - ARP reply with VRRP virtual src hw addr OK - MPLS OK - Proxy ARP OK - Interface Mirror Proxy ARP OK - - ============================================================================== - L2BD arp termination Test Case - ============================================================================== - L2BD arp term - add 5 hosts, verify arp responses OK - L2BD arp term - delete 3 hosts, verify arp responses OK - L2BD arp term - recreate BD1, readd 3 hosts, verify arp responses OK - L2BD arp term - 2 IP4 addrs per host OK - L2BD arp term - create and update 10 IP4-mac pairs OK - L2BD arp/ND term - hosts with both ip4/ip6 OK - L2BD ND term - Add and Del hosts, verify ND replies OK - L2BD ND term - Add and update IP+mac, verify ND replies OK - L2BD arp term - send garps, verify arp event reports OK - L2BD arp term - send duplicate garps, verify suppression OK - L2BD arp term - disable ip4 arp events,send garps, verify no events OK - L2BD ND term - send NS packets verify reports OK - L2BD ND term - send duplicate ns, verify suppression OK - L2BD ND term - disable ip4 arp events,send ns, verify no events OK - -BFD API -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) - API - ============================================================================== - activate SHA1 authentication SKIP - create BFD session using non-existent SHA1 (negative case) SKIP - create a BFD session SKIP - create IPv6 BFD session SKIP - create a BFD session (SHA1) SKIP - add SHA1 keys SKIP - change SHA1 key SKIP - deactivate SHA1 authentication SKIP - create the same BFD session twice (negative case) SKIP - create the same BFD session twice (negative case) (SHA1) SKIP - modify BFD session parameters SKIP - share single SHA1 key between multiple BFD sessions SKIP - -BFD Authentication -`````````````````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (SHA1 auth) - ============================================================================== - hold BFD session up SKIP - hold BFD session up - meticulous auth SKIP - session is not brought down by unauthenticated msg SKIP - session is not brought down by msg with non-existent key-id SKIP - session is not brought down by msg with wrong auth type SKIP - simulate remote peer restart and resynchronization SKIP - session is not kept alive by msgs with bad sequence numbers SKIP - bring BFD session up SKIP - -BFD Authentication Change -````````````````````````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (changing auth) - ============================================================================== - change auth key without disturbing session state (delayed) SKIP - change auth key without disturbing session state (immediate) SKIP - turn auth off without disturbing session state (delayed) SKIP - turn auth off without disturbing session state (immediate) SKIP - turn auth on without disturbing session state (delayed) SKIP - turn auth on without disturbing session state (immediate) SKIP - -BFD CLI -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (CLI) - ============================================================================== - create/modify/delete IPv4 BFD UDP session SKIP - create/modify/delete IPv6 BFD UDP session SKIP - create/modify/delete IPv6 BFD UDP session (authenticated) SKIP - create/modify/delete IPv4 BFD UDP session (authenticated) SKIP - put session admin-up and admin-down SKIP - turn authentication on and off SKIP - turn authentication on and off (delayed) SKIP - set/delete meticulous SHA1 auth key SKIP - set/delete SHA1 auth key SKIP - set/del udp echo source SKIP - show commands SKIP - -BFD IPv4 -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) - ============================================================================== - put session admin-up and admin-down SKIP - configuration change while peer in demand mode SKIP - verify session goes down after inactivity SKIP - echo function SKIP - session goes down if echo function fails SKIP - echo packets looped back SKIP - echo function stops if echo source is removed SKIP - echo function stops if peer sets required min echo rx zero SKIP - hold BFD session up SKIP - immediately honor remote required min rx reduction SKIP - interface with bfd session deleted SKIP - echo packets with invalid checksum don't keep a session up SKIP - large remote required min rx interval SKIP - modify detect multiplier SKIP - modify session - double required min rx SKIP - modify session - halve required min rx SKIP - no periodic frames outside poll sequence if remote demand set SKIP - test correct response to control frame with poll bit set SKIP - test poll sequence queueing SKIP - bring BFD session down SKIP - bring BFD session up SKIP - bring BFD session up - first frame looked up by address pair SKIP - verify slow periodic control frames while session down SKIP - stale echo packets don't keep a session up SKIP - no packets when zero remote required min rx interval SKIP - -BFD IPv6 -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (IPv6) - ============================================================================== - echo function used SKIP - echo packets looped back SKIP - hold BFD session up SKIP - interface with bfd session deleted SKIP - bring BFD session up SKIP - bring BFD session up - first frame looked up by address pair SKIP - - ============================================================================== - BFD-FIB interactions (IPv6) - ============================================================================== - BFD-FIB interactions SKIP - -BIER - Bit Indexed Explicit Replication -``````````````````````````````````````` -:: - - ============================================================================== - BIER Test Case - ============================================================================== - BIER end-to-end OK - BIER head OK - BIER head over UDP OK - BIER midpoint OK - BIER Tail OK - BIER Tail over UDP OK - - ============================================================================== - BIER FIB Test Case - ============================================================================== - BFIB Unit Tests OK - -Classifier -`````````` -:: - - ============================================================================== - Classifier Test Case - ============================================================================== - IP ACL test OK - MAC ACL test OK - IP PBR test OK - -Container Integration -````````````````````` -:: - - ============================================================================== - Container integration extended testcases - ============================================================================== - IPv4 basic connectivity test SKIP - IPv6 basic connectivity test SKIP - Create loopbacks overlapping with remote addresses SKIP - IPv4 local-spoof connectivity test SKIP - IPv6 local-spoof connectivity test SKIP - Configure container commands SKIP - IPv4 test after configuring container SKIP - IPv6 test after configuring container SKIP - Unconfigure container commands SKIP - IPv4 local-spoof after unconfig test SKIP - IPv6 local-spoof after unconfig test SKIP - -CRUD Loopback -````````````` -:: - - ============================================================================== - CRUD Loopback - ============================================================================== - test_crud (test_interface_crud.TestLoopbackInterfaceCRUD) OK - test_down (test_interface_crud.TestLoopbackInterfaceCRUD) OK - -DHCP -```` -:: - - ============================================================================== - DHCP Test Case - ============================================================================== - DHCPv6 Proxy OK - DHCP Client OK - DHCPv4 Proxy OK - -Distributed Virtual Router -`````````````````````````` -:: - - ============================================================================== - Distributed Virtual Router - ============================================================================== - Distributed Virtual Router OK - L2 Emulation OK - -DS-Lite Softwire -```````````````` -:: - - ============================================================================== - DS-Lite Test Cases - ============================================================================== - Test DS-Lite OK - -FIB -``` -:: - - ============================================================================== - FIB Test Case - ============================================================================== - FIB Unit Tests OK - -Flowprobe -````````` -:: - - ============================================================================== - Re-enable Flowprobe feature - ============================================================================== - disable flowprobe feature after first packets and re-enable SKIP - - ============================================================================== - collect information on Ethernet, IP4 and IP6 datapath (no timers) - ============================================================================== - no timers, one CFLOW packet, 9 Flows inside OK - no timers, two CFLOW packets (mtu=256), 3 Flows in each OK - L2 data on IP4 datapath OK - L2 data on IP6 datapath OK - L2 data on L2 datapath OK - L3 data on IP4 datapath OK - L3 data on IP6 datapath OK - L3 data on L2 datapath OK - L4 data on IP4 datapath OK - L4 data on IP6 datapath OK - L4 data on L2 datapath OK - verify templates on IP6 datapath OK - verify templates on IP4 datapath OK - verify template on L2 datapath OK - - ============================================================================== - Disable Flowprobe feature - ============================================================================== - disable flowprobe feature after first packets SKIP - - ============================================================================== - Re-enable IPFIX - ============================================================================== - disable IPFIX after first packets and re-enable after few packets SKIP - - ============================================================================== - Disable IPFIX - ============================================================================== - disable IPFIX after first packets SKIP - -Geneve Tunnels -`````````````` -:: - - ============================================================================== - GENEVE Test Case - ============================================================================== - Decapsulation test OK - Encapsulation test OK - Multicast flood test OK - Multicast receive test OK - Unicast flood test OK - -GRE Tunnels -``````````` -:: - - ============================================================================== - GRE Test Case - ============================================================================== - GRE IPv4 tunnel Tests OK - GRE IPv6 tunnel Tests OK - GRE tunnel L2 Tests OK - GRE tunnel VRF Tests OK - -GTPU Tunnels -```````````` -:: - - ============================================================================== - GTPU Test Case - ============================================================================== - Decapsulation test OK - Encapsulation test OK - Multicast flood test OK - Multicast receive test OK - Unicast flood test OK - -IP Multicast Routing -```````````````````` -:: - - ============================================================================== - IP Multicast Test Case - ============================================================================== - IP Multicast Bi-directional OK - IPv6 Multicast Replication OK - IPv6 Multicast Replication in non-default table OK - IP Multicast Replication OK - IP Multicast Connected Source check OK - IP Multicast Signal OK - IP Multicast Replication in non-default table OK - -IPSec -````` -:: - - ============================================================================== - Basic test for IPSEC using AH transport and Tunnel mode - ============================================================================== - ipsec ah v4 transport basic test OK - ipsec ah v4 transport burst test OK - ipsec ah 4o4 tunnel basic test OK - ipsec ah 4o4 tunnel burst test OK - - ============================================================================== - Basic test for ipsec esp sanity - tunnel and transport modes. - ============================================================================== - ipsec esp v4 transport basic test OK - ipsec esp v4 transport burst test OK - ipsec esp 4o4 tunnel basic test OK - ipsec esp 4o4 tunnel burst test OK - -IPv4 FIB CRUD -````````````` -:: - - ============================================================================== - FIB - add/update/delete - ip4 routes - ============================================================================== - Add 1k routes OK - Delete 100 routes OK - Add 1k routes OK - Delete 1.5k routes OK - -IPv4 Routing -```````````` -:: - - ============================================================================== - IPv4 Test Case - ============================================================================== - IPv4 FIB test OK - - ============================================================================== - IPv4 routes via NULL - ============================================================================== - IP NULL route OK - - ============================================================================== - IPv4 disabled - ============================================================================== - IP Disabled OK - - ============================================================================== - IPv4 Subnets - ============================================================================== - IP Sub Nets OK - - ============================================================================== - IPv4 VLAN-0 - ============================================================================== - IP VLAN-0 OK - - ============================================================================== - IPv4 Load-Balancing - ============================================================================== - IP Load-Balancing OK - - ============================================================================== - IPv4 Deaggregate Routes - ============================================================================== - IP Deag Routes OK - - ============================================================================== - IPv4 Input Exceptions - ============================================================================== - IP Input Exceptions OK - - ============================================================================== - IPv4 Punt Police/Redirect - ============================================================================== - IP punt police and redirect OK - -IPv4 VRF Multi-instance -``````````````````````` -:: - - ============================================================================== - IP4 VRF Multi-instance Test Case - ============================================================================== - IP4 VRF Multi-instance test 1 - create 5 BDs OK - IP4 VRF Multi-instance test 2 - delete 2 VRFs OK - IP4 VRF Multi-instance 3 - add 2 VRFs OK - IP4 VRF Multi-instance test 4 - delete 4 VRFs OK - -IPv6 Routing -```````````` -:: - - ============================================================================== - IPv6 Test Case - ============================================================================== - IPv6 FIB test OK - IPv6 Neighbour Solicitation Exceptions OK - ND Duplicates OK - IPv6 Router Solicitation Exceptions OK - - ============================================================================== - IPv6 Punt Police/Redirect - ============================================================================== - IP6 punt police and redirect OK - - ============================================================================== - IPv6 disabled - ============================================================================== - IP Disabled OK - - ============================================================================== - IPv6 ND ProxyTest Case - ============================================================================== - IPv6 Proxy ND OK - - ============================================================================== - IPv6 Load-Balancing - ============================================================================== - IPv6 Load-Balancing OK - - ============================================================================== - IPv6 routes via NULL - ============================================================================== - IP NULL route OK - - ============================================================================== - IPv6 Input Exceptions - ============================================================================== - IP6 Input Exceptions OK - -IPv6 VRF Multi-instance -``````````````````````` -:: - - ============================================================================== - IP6 VRF Multi-instance Test Case - ============================================================================== - IP6 VRF Multi-instance test 1 - create 4 VRFs OK - IP6 VRF Multi-instance test 2 - reset 2 VRFs OK - IP6 VRF Multi-instance 3 - add 2 VRFs OK - IP6 VRF Multi-instance test 4 - reset 4 VRFs OK - -IRB Integrated Routing-Bridging -``````````````````````````````` -:: - - ============================================================================== - IRB Test Case - ============================================================================== - IPv4 IRB test 1 OK - IPv4 IRB test 2 OK - -Kube-proxy -`````````` -:: - - ============================================================================== - Kube-proxy Test Case - ============================================================================== - Kube-proxy NAT44 OK - Kube-proxy NAT46 SKIP - Kube-proxy NAT64 SKIP - Kube-proxy NAT66 SKIP - -L2 FIB CRUD -``````````` -:: - - ============================================================================== - L2 FIB Test Case - ============================================================================== - L2 FIB - program 100 + 100 MACs OK - L2 FIB - program 100 + delete 12 MACs OK - L2 FIB - flush all OK - L2 FIB - flush BD OK - L2 FIB - flush interface OK - L2 FIB - mac learning events OK - L2 FIB - mac learning max macs in event OK - L2 FIB - program 100 MACs OK - L2 FIB - Program 10 MACs, learn 10 OK - -L2BD Multi-instance -``````````````````` -:: - - ============================================================================== - L2BD Multi-instance Test Case - ============================================================================== - L2BD Multi-instance test 1 - create 5 BDs OK - L2BD Multi-instance test 2 - update data of 5 BDs OK - L2BD Multi-instance test 3 - delete 2 BDs OK - L2BD Multi-instance test 4 - add 2 BDs OK - L2BD Multi-instance test 5 - delete 5 BDs SKIP - -L2BD Switching -`````````````` -:: - - ============================================================================== - L2BD Test Case - ============================================================================== - L2BD MAC learning dual-loop test OK - L2BD MAC learning single-loop test OK - -L2XC Multi-instance -``````````````````` -:: - - ============================================================================== - L2XC Multi-instance Test Case - ============================================================================== - L2XC Multi-instance test 1 - create 10 cross-connects OK - L2XC Multi-instance test 2 - delete 4 cross-connects OK - L2BD Multi-instance 3 - add new 4 cross-connects OK - L2XC Multi-instance test 4 - delete 10 cross-connects OK - -L2XC Switching -`````````````` -:: - - ============================================================================== - L2XC Test Case - ============================================================================== - L2XC dual-loop test OK - L2XC single-loop test OK - -LISP Tunnels -```````````` -:: - - ============================================================================== - Basic LISP test - ============================================================================== - Test case for basic encapsulation OK - -Load Balancer -````````````` -:: - - ============================================================================== - Load Balancer Test Case - ============================================================================== - Load Balancer IP4 GRE4 OK - Load Balancer IP4 GRE6 OK - Load Balancer IP6 GRE4 OK - Load Balancer IP6 GRE6 OK - -MACIP Access Control -```````````````````` -:: - - ============================================================================== - MACIP Tests - ============================================================================== - MACIP 10 ACLs each with 100+ entries OK - MACIP 10 ACLs each with 100+ entries with IP4 traffic OK - MACIP 10 ACLs each with 100+ entries with IP6 traffic OK - MACIP ACL with 10 entries OK - MACIP ACL with 100 entries OK - MACIP ACL with 2 entries OK - MACIP ACL with 20 entries OK - MACIP ACL with 5 entries OK - MACIP ACL with 50 entries OK - MACIP 2 ACLs each with 100+ entries OK - MACIP replace ACL OK - MACIP ACL delete intf with acl OK - - ============================================================================== - MACIP with IP6 traffic - ============================================================================== - IP6 MACIP exactMAC|exactIP ACL bridged traffic OK - IP6 MACIP exactMAC|subnetIP ACL bridged traffic OK - IP6 MACIP exactMAC|wildIP ACL bridged traffic OK - IP6 MACIP oui_MAC|exactIP ACL bridged traffic OK - IP6 MACIP ouiMAC|subnetIP ACL bridged traffic OK - IP6 MACIP ouiMAC|wildIP ACL bridged traffic OK - IP6 MACIP wildcardMAC|exactIP ACL bridged traffic OK - IP6 MACIP wildcardMAC|subnetIP ACL bridged traffic OK - IP6 MACIP wildcardMAC|wildIP ACL bridged traffic OK - MACIP replace ACL with IP6 traffic OK - IP6 MACIP exactMAC|exactIP ACL routed traffic OK - IP6 MACIP exactMAC|subnetIP ACL routed traffic OK - IP6 MACIP exactMAC|wildIP ACL routed traffic OK - IP6 MACIP ouiMAC|exactIP ACL routed traffic OK - IP6 MACIP ouiMAC|subnetIP ACL routed traffic OK - IP6 MACIP ouiMAC|wildIP ACL routed traffic OK - IP6 MACIP wildcardMAC|exactIP ACL routed traffic OK - IP6 MACIP wildcardMAC|subnetIP ACL routed traffic OK - IP6 MACIP wildcardMAC|wildIP ACL OK - - ============================================================================== - MACIP with IP4 traffic - ============================================================================== - IP4 MACIP wildcardMAC|exactIP ACL bridged traffic OK - IP4 MACIP exactMAC|exactIP ACL bridged traffic OK - IP4 MACIP exactMAC|subnetIP ACL bridged traffic OK - IP4 MACIP exactMAC|wildIP ACL bridged traffic OK - IP4 MACIP ouiMAC|exactIP ACL bridged traffic OK - IP4 MACIP ouiMAC|subnetIP ACL bridged traffic OK - IP4 MACIP ouiMAC|wildIP ACL bridged traffic OK - IP4 MACIP wildcardMAC|subnetIP ACL bridged traffic OK - IP4 MACIP wildcardMAC|wildIP ACL bridged traffic OK - MACIP replace ACL with IP4 traffic OK - IP4 MACIP exactMAC|exactIP ACL routed traffic OK - IP4 MACIP exactMAC|subnetIP ACL routed traffic OK - IP4 MACIP exactMAC|wildIP ACL routed traffic OK - IP4 MACIP ouiMAC|exactIP ACL routed traffic OK - IP4 MACIP ouiMAC|subnetIP ACL routed traffic OK - IP4 MACIP ouiMAC|wildIP ACL routed traffic OK - IP4 MACIP wildcardMAC|exactIP ACL routed traffic OK - IP4 MACIP wildcardMAC|subnetIP ACL routed traffic OK - IP4 MACIP wildcardMAC|wildIP ACL OK - -MAP Softwires -````````````` -:: - - ============================================================================== - MAP Test Case - ============================================================================== - MAP-E OK - -MFIB Multicast FIB -`````````````````` -:: - - ============================================================================== - MFIB Test Case - ============================================================================== - MFIB Unit Tests OK - -MPLS Switching -`````````````` -:: - - ============================================================================== - MPLS-L2 - ============================================================================== - Virtual Private LAN Service OK - Virtual Private Wire Service OK - - ============================================================================== - MPLS Test Case - ============================================================================== - MPLS Local Label Binding test OK - MPLS Deagg OK - MPLS label imposition test OK - MPLS Interface Receive OK - MPLS Multicast Head-end OK - MPLS IPv4 Multicast Tail OK - MPLS IPv6 Multicast Tail OK - MPLS Multicast Mid Point OK - MPLS label swap tests OK - MPLS Tunnel Tests OK - MPLS V4 Explicit NULL test OK - MPLS V6 Explicit NULL test OK - - ============================================================================== - MPLS PIC edge convergence - ============================================================================== - MPLS eBGP PIC edge convergence OK - MPLS iBGP PIC edge convergence OK - MPLSv6 eBGP PIC edge convergence OK - - ============================================================================== - MPLS disabled - ============================================================================== - MPLS Disabled OK - -NAT44 -````` -:: - - ============================================================================== - NAT44 Test Cases - ============================================================================== - Delete NAT44 session OK - NAT44 dynamic translation test OK - NAT44 handling of client packets with TTL=1 OK - NAT44 handling of error responses to client packets with TTL=2 OK - NAT44 handling of server packets with TTL=1 OK - NAT44 handling of error responses to server packets with TTL=2 OK - NAT44 interfaces without configured IP address OK - NAT44 forwarding test OK - NAT44 translate fragments arriving in order OK - NAT44 translate fragments arriving out of order OK - NAT44 hairpinning - 1:1 NAPT OK - NAT44 hairpinning - 1:1 NAT OK - 1:1 NAT translate packet with unknown protocol - hairpinning OK - NAT44 translate packet with unknown protocol - hairpinning OK - Identity NAT OK - NAT44 multiple inside interfaces with overlapping address space OK - Acquire NAT44 addresses from interface OK - Identity NAT with addresses from interface OK - Static mapping with addresses from interface OK - IPFIX logging NAT addresses exhausted OK - IPFIX logging NAT44 session created/delted OK - MAX translations per user - recycle the least recently used OK - NAT44 multiple non-overlapping address space inside interfaces OK - One armed NAT44 OK - NAT44 interface output feature (in2out postrouting) OK - NAT44 interface output feature hairpinning (in2out postrouting) OK - NAT44 interface output feature VRF aware (in2out postrouting) OK - Ping internal host from outside network OK - Ping NAT44 out interface from outside network OK - NAT44 add pool addresses to FIB OK - Port restricted NAT44 (MAP-E CE) OK - NAT44 fragments hairpinning OK - NAT44 set/get virtual fragmentation reassembly OK - 1:1 NAT initialized from inside network OK - NAT44 interfaces without configured IP address - 1:1 NAT OK - NAT44 local service load balancing OK - 1:1 NAT initialized from outside network OK - 1:1 NAT translate packet with unknown protocol OK - 1:1 NAT VRF awareness OK - 1:1 NAPT initialized from inside network OK - NAT44 interfaces without configured IP address - 1:1 NAPT OK - 1:1 NAPT initialized from outside network OK - Twice NAT44 OK - Acquire twice NAT44 addresses from interface OK - Twice NAT44 local service load balancing OK - NAT44 translate packet with unknown protocol OK - NAT44 tenant VRF independent address pool mode OK - NAT44 tenant VRF aware address pool mode OK - - ============================================================================== - Deterministic NAT Test Cases - ============================================================================== - Deterministic NAT translation test (TCP, UDP, ICMP) OK - NAT plugin run deterministic mode OK - Deterministic NAT multiple users OK - Deterministic NAT maximum sessions per user limit SKIP - Deterministic NAT session timeouts SKIP - Set deterministic NAT timeouts OK - Deterministic NAT TCP session close from inside network OK - Deterministic NAT TCP session close from outside network OK - -NAT64 -````` -:: - - ============================================================================== - NAT64 Test Cases - ============================================================================== - NAT64 dynamic translation test OK - NAT64 translate fragments arriving in order OK - NAT64 translate fragments arriving out of order OK - NAT64 hairpinning OK - NAT64 translate packet with unknown protocol - hairpinning OK - NAT64 ICMP Error message translation OK - Enable/disable NAT64 feature on the interface OK - Acquire NAT64 pool addresses from interface OK - One armed NAT64 OK - Add/delete address to NAT64 pool OK - NAT64 Network-Specific Prefix OK - NAT64 fragments hairpinning OK - NAT64 session timeout SKIP - Set NAT64 timeouts OK - NAT64 static translation test OK - Add/delete static BIB entry OK - NAT64 translate packet with unknown protocol OK - -P2P Ethernet Subinterface -````````````````````````` -:: - - ============================================================================== - P2P Ethernet tests - ============================================================================== - delete/create p2p subif OK - create 1k of p2p subifs OK - - ============================================================================== - P2P Ethernet IPv4 tests - ============================================================================== - receive ipv4 packet via p2p subinterface OK - route rx packet not matching p2p subinterface OK - send ip4 packet via p2p subinterface OK - drop tx ip4 packet not matching p2p subinterface OK - - ============================================================================== - P2P Ethernet IPv6 tests - ============================================================================== - receive ipv6 packet via p2p subinterface OK - drop rx packet not matching p2p subinterface OK - route rx ip6 packet not matching p2p subinterface OK - send packet via p2p subinterface OK - drop tx ip6 packet not matching p2p subinterface OK - standard routing without p2p subinterfaces OK - -PPPoE Encapsulation -``````````````````` -:: - - ============================================================================== - PPPoE Test Case - ============================================================================== - PPPoE Add Same Session Twice Test OK - PPPoE Decap Test OK - PPPoE Decap Multiple Sessions Test OK - PPPoE Delete Same Session Twice Test OK - PPPoE Encap Test OK - PPPoE Encap Multiple Sessions Test OK - -SPAN Switch Port Analyzer -````````````````````````` -:: - - ============================================================================== - SPAN Test Case - ============================================================================== - SPAN device rx mirror OK - SPAN l2 broadcast mirror OK - SPAN l2 rx tx mirror OK - SPAN l2 tx mirror OK - SPAN l2 rx mirror OK - SPAN l2 rx mirror into 1ad subif+vtr OK - SPAN l2 rx mirror into 1q subif+vtr OK - SPAN l2 rx mirror into gre-subif+vtr OK - SPAN l2 rx mirror into vxlan OK - -SRv6 Routing -```````````` -:: - - ============================================================================== - SRv6 Test Case - ============================================================================== - Test SRv6 End (without PSP) behavior. OK - Test SRv6 End.DT4 behavior. OK - Test SRv6 End.DT6 behavior. OK - Test SRv6 End.DX2 behavior. OK - Test SRv6 End.DX4 behavior. OK - Test SRv6 End.DX6 behavior. OK - Test SRv6 End.X (without PSP) behavior. OK - Test SRv6 End.X with PSP behavior. OK - Test SRv6 End with PSP behavior. OK - Test SRv6 Transit.Encaps behavior for IPv6. OK - Test SRv6 Transit.Encaps behavior for IPv4. OK - Test SRv6 Transit.Encaps behavior for L2. SKIP - Test SRv6 Transit.Insert behavior (IPv6 only). OK - Test SRv6 Transit.Insert behavior (IPv6 only). OK - -TCP/IP Stack -```````````` -:: - - ============================================================================== - TCP Test Case - ============================================================================== - TCP builtin client/server transfer OK - TCP Unit Tests OK - -UDP Stack -````````` -:: - - ============================================================================== - UDP Encap Test Case - ============================================================================== - UDP Encap test OK - -VTR VLAN Tag Rewrites -````````````````````` -:: - - ============================================================================== - VTR Test Case - ============================================================================== - 1AD VTR pop 1 test OK - 1AD VTR pop 2 test OK - 1AD VTR push 1 1AD test OK - 1AD VTR push 1 1Q test OK - 1AD VTR push 2 1AD test OK - 1AD VTR push 2 1Q test OK - 1AD VTR translate 1 -> 1 1AD test OK - 1AD VTR translate 1 -> 1 1Q test OK - 1AD VTR translate 1 -> 2 1AD test OK - 1AD VTR translate 1 -> 2 1Q test OK - 1AD VTR translate 2 -> 1 1AD test OK - 1AD VTR translate 2 -> 1 1Q test OK - 1AD VTR translate 2 -> 2 1AD test OK - 1AD VTR translate 2 -> 2 1Q test OK - 1Q VTR pop 1 test OK - 1Q VTR push 1 test OK - 1Q VTR push 2 test OK - 1Q VTR translate 1 -> 1 test OK - 1Q VTR translate 1 -> 2 test OK - -VXLAN Tunnels -````````````` -:: - - ============================================================================== - VXLAN Test Case - ============================================================================== - Decapsulation test OK - Encapsulation test OK - Multicast flood test OK - Multicast receive test OK - Unicast flood test OK - -VXLAN-GPE Tunnels -````````````````` -:: - - ============================================================================== - VXLAN-GPE Test Case - ============================================================================== - Decapsulation test SKIP - Encapsulation test SKIP - Multicast flood test SKIP - Multicast receive test SKIP - Unicast flood test SKIP - -Other Tests -``````````` -:: - - ============================================================================== - Ping Test Case - ============================================================================== - basic ping test OK - burst ping test OK - - ============================================================================== - Session Test Case - ============================================================================== - Session Unit Tests OK - - ============================================================================== - Template verification, timer tests - ============================================================================== - timer less than template timeout OK - timer greater than template timeout OK - verify cflow packet fields OK - diff --git a/docs/report/dpdk_performance_tests/csit_release_notes.rst b/docs/report/dpdk_performance_tests/csit_release_notes.rst deleted file mode 100644 index 413c7c3cea..0000000000 --- a/docs/report/dpdk_performance_tests/csit_release_notes.rst +++ /dev/null @@ -1,23 +0,0 @@ -CSIT Release Notes -================== - -Changes in CSIT |release| -------------------------- - -No code changes apart from bug fixes. - -Known Issues ------------- - -Here is the list of known issues in CSIT |release| for Testpmd performance tests: - -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ -| # | Issue | Jira ID | Description | -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ -| 1 | Testpmd in 1t1c and 2t2c setups - large variation | CSIT-569 | Suspected NIC firmware or DPDK driver issue affecting NDR | -| | of discovered NDR throughput values across | | throughput. Applies to XL710 and X710 NICs, no issues observed | -| | multiple test runs with xl710 and x710 NICs. | | on x520 NICs. | -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ -| 2 | Lower than expected NDR throughput with xl710 | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR | -| | and x710 NICs, compared to x520 NICs. | | throughput. Applies to XL710 and X710 NICs. | -+---+---------------------------------------------------+------------+-----------------------------------------------------------------+ diff --git a/docs/report/dpdk_performance_tests/documentation.rst b/docs/report/dpdk_performance_tests/documentation.rst deleted file mode 100644 index 47ec7cbe2a..0000000000 --- a/docs/report/dpdk_performance_tests/documentation.rst +++ /dev/null @@ -1,5 +0,0 @@ -Documentation -============= - -`CSIT DPDK Performance Tests Documentation`_ contains detailed -functional description and input parameters for each test case. diff --git a/docs/report/dpdk_performance_tests/index.rst b/docs/report/dpdk_performance_tests/index.rst deleted file mode 100644 index 44d34d3d4d..0000000000 --- a/docs/report/dpdk_performance_tests/index.rst +++ /dev/null @@ -1,12 +0,0 @@ -DPDK Performance Tests -====================== - -.. toctree:: - - overview - csit_release_notes - packet_throughput_graphs/index - packet_latency_graphs/index - test_environment - documentation - diff --git a/docs/report/dpdk_performance_tests/overview.rst b/docs/report/dpdk_performance_tests/overview.rst deleted file mode 100644 index 02651d7e53..0000000000 --- a/docs/report/dpdk_performance_tests/overview.rst +++ /dev/null @@ -1,240 +0,0 @@ -Overview -======== - -Tested Physical Topologies --------------------------- - -CSIT DPDK performance tests are executed on physical baremetal servers hosted -by :abbr:`LF (Linux Foundation)` FD.io project. Testbed physical topology is -shown in the figure below.:: - - +------------------------+ +------------------------+ - | | | | - | +------------------+ | | +------------------+ | - | | | | | | | | - | | <-----------------> | | - | | DUT1 | | | | DUT2 | | - | +--^---------------+ | | +---------------^--+ | - | | | | | | - | | SUT1 | | SUT2 | | - +------------------------+ +------------------^-----+ - | | - | | - | +-----------+ | - | | | | - +------------------> TG <------------------+ - | | - +-----------+ - -SUT1 and SUT2 are two System Under Test servers (Cisco UCS C240, each with two -Intel XEON CPUs), TG is a Traffic Generator (TG, another Cisco UCS C240, with -two Intel XEON CPUs). SUTs run Testpmd/L3FWD SW SW application in Linux -user-mode as a Device Under Test (DUT). TG runs TRex SW application as a packet -Traffic Generator. Physical connectivity between SUTs and to TG is provided -using different NIC models that need to be tested for performance. Currently -installed and tested NIC models include: - -#. 2port10GE X520-DA2 Intel. -#. 2port10GE X710 Intel. -#. 2port10GE VIC1227 Cisco. -#. 2port40GE VIC1385 Cisco. -#. 2port40GE XL710 Intel. - -From SUT and DUT perspective, all performance tests involve forwarding packets -between two physical Ethernet ports (10GE or 40GE). Due to the number of -listed NIC models tested and available PCI slot capacity in SUT servers, in -all of the above cases both physical ports are located on the same NIC. In -some test cases this results in measured packet throughput being limited not -by VPP DUT but by either the physical interface or the NIC capacity. - -Going forward CSIT project will be looking to add more hardware into FD.io -performance labs to address larger scale multi-interface and multi-NIC -performance testing scenarios. - -Note that reported DUT (DPDK) performance results are specific to the SUTs -tested. Current :abbr:`LF (Linux Foundation)` FD.io SUTs are based on Intel -XEON E5-2699v3 2.3GHz CPUs. SUTs with other CPUs are likely to yield different -results. A good rule of thumb, that can be applied to estimate DPDK packet -thoughput for Phy-to-Phy (NIC-to-NIC, PCI-to-PCI) topology, is to expect -the forwarding performance to be proportional to CPU core frequency, -assuming CPU is the only limiting factor and all other SUT parameters -equivalent to FD.io CSIT environment. The same rule of thumb can be also -applied for Phy-to-VM/LXC-to-Phy (NIC-to-VM/LXC-to-NIC) topology, but due to -much higher dependency on intensive memory operations and sensitivity to Linux -kernel scheduler settings and behaviour, this estimation may not always yield -good enough accuracy. - -For detailed :abbr:`LF (Linux Foundation)` FD.io test bed specification and -physical topology please refer to `LF FD.io CSIT testbed wiki page -`_. - -Performance Tests Coverage --------------------------- - -Performance tests are split into two main categories: - -- Throughput discovery - discovery of packet forwarding rate using binary search - in accordance to :rfc:`2544`. - - - NDR - discovery of Non Drop Rate packet throughput, at zero packet loss; - followed by one-way packet latency measurements at 10%, 50% and 100% of - discovered NDR throughput. - - PDR - discovery of Partial Drop Rate, with specified non-zero packet loss - currently set to 0.5%; followed by one-way packet latency measurements at - 100% of discovered PDR throughput. - -- Throughput verification - verification of packet forwarding rate against - previously discovered throughput rate. These tests are currently done against - 0.9 of reference NDR, with reference rates updated periodically. - -CSIT |release| includes following performance test suites, listed per NIC type: - -- 2port10GE X520-DA2 Intel - - - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between - two Interfaces. - -- 2port40GE XL710 Intel - - - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between - two Interfaces. - -- 2port10GE X520-DA2 Intel - - - **IPv4 Routed Forwarding** - L3 IP forwarding of Ethernet frames between - two Interfaces. - -Execution of performance tests takes time, especially the throughput discovery -tests. Due to limited HW testbed resources available within FD.io labs hosted -by Linux Foundation, the number of tests for NICs other than X520 (a.k.a. -Niantic) has been limited to few baseline tests. Over time we expect the HW -testbed resources to grow, and will be adding complete set of performance -tests for all models of hardware to be executed regularly and(or) -continuously. - -Performance Tests Naming ------------------------- - -CSIT |release| follows a common structured naming convention for all performance -and system functional tests, introduced in CSIT |release-1|. - -The naming should be intuitive for majority of the tests. Complete description -of CSIT test naming convention is provided on `CSIT test naming wiki -`_. - -Methodology: Multi-Core and Multi-Threading -------------------------------------------- - -**Intel Hyper-Threading** - CSIT |release| performance tests are executed with -SUT servers' Intel XEON processors configured in Intel Hyper-Threading Disabled -mode (BIOS setting). This is the simplest configuration used to establish -baseline single-thread single-core application packet processing and forwarding -performance. Subsequent releases of CSIT will add performance tests with Intel -Hyper-Threading Enabled (requires BIOS settings change and hard reboot of -server). - -**Multi-core Tests** - CSIT |release| multi-core tests are executed in the -following VPP thread and core configurations: - -#. 1t1c - 1 pmd worker thread on 1 CPU physical core. -#. 2t2c - 2 pmd worker threads on 2 CPU physical cores. - -Note that in many tests running Testpmd/L3FWD reaches tested NIC I/O bandwidth -or packets-per-second limit. - -Methodology: Packet Throughput ------------------------------- - -Following values are measured and reported for packet throughput tests: - -- NDR binary search per :rfc:`2544`: - - - Packet rate: "RATE: pps - (2x )" - - Aggregate bandwidth: "BANDWIDTH: Gbps (untagged)" - -- PDR binary search per :rfc:`2544`: - - - Packet rate: "RATE: pps (2x - )" - - Aggregate bandwidth: "BANDWIDTH: Gbps (untagged)" - - Packet loss tolerance: "LOSS_ACCEPTANCE "" - -- NDR and PDR are measured for the following L2 frame sizes: - - - IPv4: 64B, 1518B, 9000B. - -All rates are reported from external Traffic Generator perspective. - - -Methodology: Packet Latency ---------------------------- - -TRex Traffic Generator (TG) is used for measuring latency of Testpmd DUTs. -Reported latency values are measured using following methodology: - -- Latency tests are performed at 10%, 50% of discovered NDR rate (non drop rate) - for each NDR throughput test and packet size (except IMIX). -- TG sends dedicated latency streams, one per direction, each at the rate of - 10kpps at the prescribed packet size; these are sent in addition to the main - load streams. -- TG reports min/avg/max latency values per stream direction, hence two sets - of latency values are reported per test case; future release of TRex is - expected to report latency percentiles. -- Reported latency values are aggregate across two SUTs due to three node - topology used for all performance tests; for per SUT latency, reported value - should be divided by two. -- 1usec is the measurement accuracy advertised by TRex TG for the setup used in - FD.io labs used by CSIT project. -- TRex setup introduces an always-on error of about 2*2usec per latency flow - - additonal Tx/Rx interface latency induced by TRex SW writing and reading - packet timestamps on CPU cores without HW acceleration on NICs closer to the - interface line. - -Methodology: TRex Traffic Generator Usage ------------------------------------------ - -The `TRex traffic generator `_ is used for all -CSIT performance tests. TRex stateless mode is used to measure NDR and PDR -throughputs using binary search (NDR and PDR discovery tests) and for quick -checks of DUT performance against the reference NDRs (NDR check tests) for -specific configuration. - -TRex is installed and run on the TG compute node. The typical procedure is: - -- If the TRex is not already installed on TG, it is installed in the - suite setup phase - see `TRex intallation`_. -- TRex configuration is set in its configuration file - :: - - /etc/trex_cfg.yaml - -- TRex is started in the background mode - :: - - $ sh -c 'cd /scripts/ && sudo nohup ./t-rex-64 -i -c 7 --iom 0 > /tmp/trex.log 2>&1 &' > /dev/null - -- There are traffic streams dynamically prepared for each test, based on traffic - profiles. The traffic is sent and the statistics obtained using - :command:`trex_stl_lib.api.STLClient`. - -**Measuring packet loss** - -- Create an instance of STLClient -- Connect to the client -- Add all streams -- Clear statistics -- Send the traffic for defined time -- Get the statistics - -If there is a warm-up phase required, the traffic is sent also before test and -the statistics are ignored. - -**Measuring latency** - -If measurement of latency is requested, two more packet streams are created (one -for each direction) with TRex flow_stats parameter set to STLFlowLatencyStats. In -that case, returned statistics will also include min/avg/max latency values. diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst deleted file mode 100644 index b99bc2b1f6..0000000000 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst +++ /dev/null @@ -1,28 +0,0 @@ -Packet Latency Graphs -===================== - -Plotted results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Median and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by -`FD.io test executor dpdk performance jobs`_ that created result output file -used as data source for the graph, *Y-axis labels* are measured packet Latency -[uSec] values, and the *Graph legend* lists the plotted test suites and their -indices. Latency is reported for concurrent symmetric bi-directional flows, -separately for each direction: i) West-to-East: -TGint1-to-SUT1-to-SUT2-to-TGint2, and ii) East-to-West: -TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by - `FD.io test executor dpdk performance jobs`_ with Robot Framework result - files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_. - -.. toctree:: - - l2 - ip4 diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/ip4.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/ip4.rst deleted file mode 100644 index 0bccc4b920..0000000000 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/ip4.rst +++ /dev/null @@ -1,73 +0,0 @@ -IPv4 Routed-Forwarding -====================== - -This section includes summary graphs of L3FWD Phy-to-Phy performance with packet -routed forwarding measured at 50% of discovered NDR throughput rate. Latency is -reported for L3FWD running in multiple configurations of L3FWD pmd thread(s), -a.k.a. L3FWD data plane thread(s), and their physical CPU core(s) placement. - -L3FWD packet latency - running in configuration of **one worker thread (1t) on one -physical core (1c)** - is presented in the figure below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-ipv4-ndrdisc-lat50} - \label{fig:64B-1t1c-ipv4-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 1. L3FWD 1thread 1core - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -E '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -E '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - -Testpmd packet latency - running in configuration of **two worker threads (2t) -on two physical cores (2c)** - is presented in the figure below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-ipv4-ndrdisc-lat50} - \label{fig:64B-2t2c-ipv4-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 2. L3FWD 2thread 2core - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -E '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -E '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/l2.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/l2.rst deleted file mode 100644 index bcb7c4468c..0000000000 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/l2.rst +++ /dev/null @@ -1,74 +0,0 @@ -L2 Ethernet Interface Loop -========================== - -This section includes summary graphs of Testpmd Phy-to-Phy packet -latency with L2 Ethernet Interface Loop measured at 50% of discovered -NDR throughput rate. Latency is reported for Testpmd running in multiple -configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane -thread(s), and their physical CPU core(s) placement. - -Testpmd packet latency - running in configuration of **one worker thread (1t) on one -physical core (1c)** - is presented in the figure below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-l2-ndrdisc-lat50} - \label{fig:64B-1t1c-l2-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 1. Testpmd 1thread 1core - packet latency for Phy-to-Phy L2 Ethernet Looping.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - -Testpmd packet latency - running in configuration of **two worker threads (2t) -on two physical cores (2c)** - is presented in the figure below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-l2-ndrdisc-lat50} - \label{fig:64B-2t2c-l2-ndrdisc-lat50-dpdk} - \end{figure} - -*Figure 2. Testpmd 2thread 2core - packet latency for Phy-to-Phy L2 Ethernet Looping.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst deleted file mode 100644 index c1fc3f1bbc..0000000000 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst +++ /dev/null @@ -1,39 +0,0 @@ -Packet Throughput Graphs -======================== - -Plotted results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. - -For each plotted test case, Box-and-Whisker plots show the quartiles -(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / -mean, 3rd quartile / 75th percentile, Max) across collected data set -(data set size stated in the note below). Outliers are plotted as -individual points. Min and max values are plotted as bottom and top -Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and -top edge of the box. If multiple samples match only two values, and all -samples fall between them, then no whiskers are plotted. If all samples -have the same value, only a horizontal line is plotted. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by -`FD.io test executor dpdk performance jobs`_ jobs that created result output -files used as data sources for the graph, *Y-axis labels* are measured Packets -Per Second [pps] values, and the *Graph legend* lists the plotted test suites -and their indices. - -.. note:: - - Test results have been generated by - `FD.io test executor dpdk performance jobs`_ with Robot Framework result - files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_. - Plotted data set size per test case is equal to the number of job executions - presented in this report version: **10**. - -.. toctree:: - - l2 - ip4 diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/ip4.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/ip4.rst deleted file mode 100644 index 1ad08992bc..0000000000 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/ip4.rst +++ /dev/null @@ -1,150 +0,0 @@ -IPv4 Routed-Forwarding -====================== - -Following sections include summary graphs ofL3FWD Phy-to-Phy performance with -packet routed forwarding, including NDR throughput (zero packet loss) -and PDR throughput (<0.5% packet loss). Performance is reported for L3FWD -running in multiple configurations of L3FWD pmd thread(s), a.k.a. L3FWD -data plane thread(s), and their physical CPU core(s) placement. - -NDR Throughput -~~~~~~~~~~~~~~ - -Testpmd NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-ipv4-ndrdisc} - \label{fig:64B-1t1c-ipv4-ndrdisc-dpdk} - \end{figure} - -*Figure 1. L3FWD 1thread 1core - NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-ethip4-ip4base-l3fwd-ndrdisc' * - -Testpmd NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-ipv4-ndrdisc} - \label{fig:64B-2t2c-ipv4-ndrdisc-dpdk} - \end{figure} - -*Figure 2. L3FWD 2threads 2cores - NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-ethip4-ip4base-l3fwd-ndrdisc' * - -PDR Throughput -~~~~~~~~~~~~~~ - -L3FWD PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-ipv4-pdrdisc} - \label{fig:64B-1t1c-ipv4-pdrdisc-dpdk} - \end{figure} - -*Figure 3. L3FWD 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-ethip4-ip4base-l3fwd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-ethip4-ip4base-l3fwd-pdrdisc' * - -L3FWD PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-ipv4-pdrdisc} - \label{fig:64B-2t2c-ipv4-pdrdisc-dpdk} - \end{figure} - -*Figure 4. L3FWD 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-ethip4-ip4base-l3fwd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-ethip4-ip4base-l3fwd-pdrdisc' * diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/l2.rst deleted file mode 100644 index d96bf9f37d..0000000000 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/l2.rst +++ /dev/null @@ -1,150 +0,0 @@ -L2 Ethernet Interface Loop -========================== - -Following sections include summary graphs of DPDK Testpmd Phy-to-Phy performance -with L2 Ethernet Interface Loop, including NDR throughput (zero packet loss) -and PDR throughput (<0.5% packet loss). Performance is reported for Testpmd -running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd -data plane thread(s), and their physical CPU core(s) placement. - -NDR Throughput -~~~~~~~~~~~~~~ - -Testpmd NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-l2-ndrdisc} - \label{fig:64B-1t1c-l2-ndrdisc-dpdk} - \end{figure} - -*Figure 1. Testpmd 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-eth-l2xcbase-testpmd-ndrdisc' * - -Testpmd NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-l2-ndrdisc} - \label{fig:64B-2t2c-l2-ndrdisc-dpdk} - \end{figure} - -*Figure 2. Testpmd 2threads 2cores - NDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-eth-l2xcbase-testpmd-ndrdisc' * - -PDR Throughput -~~~~~~~~~~~~~~ - -Testpmd PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-1t1c-l2-pdrdisc} - \label{fig:64B-1t1c-l2-pdrdisc-dpdk} - \end{figure} - -*Figure 3. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-1t1c-eth-l2xcbase-testpmd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-1t1c-eth-l2xcbase-testpmd-pdrdisc' * - -Testpmd PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented -in the graph below. PDR measured for 0.5% packet loss ratio. - -.. raw:: html - - - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/dpdk/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{64B-2t2c-l2-pdrdisc} - \label{fig:64B-2t2c-l2-pdrdisc-dpdk} - \end{figure} - -*Figure 4. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet -Looping.* - -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/dpdk/perf && grep -P '64B-2t2c-eth-l2xcbase-testpmd-pdrdisc' * - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/dpdk/perf - $ grep -P '64B-2t2c-eth-l2xcbase-testpmd-pdrdisc' * diff --git a/docs/report/dpdk_performance_tests/test_environment.rst b/docs/report/dpdk_performance_tests/test_environment.rst deleted file mode 100644 index eb6617b11c..0000000000 --- a/docs/report/dpdk_performance_tests/test_environment.rst +++ /dev/null @@ -1,68 +0,0 @@ -.. include:: ../vpp_performance_tests/test_environment_intro.rst - -.. include:: ../vpp_performance_tests/test_environment_sut_conf_1.rst - -.. include:: ../vpp_performance_tests/test_environment_sut_conf_3.rst - - -DUT Configuration - DPDK ------------------------- - -**DPDK Version** - -|dpdk-release| - -**DPDK Compile Parameters** - -.. code-block:: bash - - make install T=x86_64-native-linuxapp-gcc -j - -**Testpmd Startup Configuration** - -Testpmd startup configuration changes per test case with different settings for CPU -cores, rx-queues. Startup config is aligned with applied test case tag: - -Tagged by **1T1C** - -.. code-block:: bash - - testpmd -c 0x3 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=1 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start - -Tagged by **2T2C** - -.. code-block:: bash - - testpmd -c 0x403 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=2 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start - -Tagged by **4T4C** - -.. code-block:: bash - - testpmd -c 0xc07 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=4 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=2 --txq=2 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start - -**L3FWD Startup Configuration** - -L3FWD startup configuration changes per test case with different settings for CPU -cores, rx-queues. Startup config is aligned with applied test case tag: - -Tagged by **1T1C** - -.. code-block:: bash - - l3fwd -l 1 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype - -Tagged by **2T2C** - -.. code-block:: bash - - l3fwd -l 1,2 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype - -Tagged by **4T4C** - -.. code-block:: bash - - l3fwd -l 1,2,3,4 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype - - -.. include:: ../vpp_performance_tests/test_environment_tg.rst diff --git a/docs/report/honeycomb_performance_tests/csit_release_notes.rst b/docs/report/honeycomb_performance_tests/csit_release_notes.rst deleted file mode 100644 index d79907d4ca..0000000000 --- a/docs/report/honeycomb_performance_tests/csit_release_notes.rst +++ /dev/null @@ -1,20 +0,0 @@ -CSIT Release Notes -================== - -Changes in CSIT |release| -------------------------- - -#. No changes since previous release. - -Known Issues ------------- - -Here is the list of known issues in CSIT |release| for Honeycomb performance -tests in VIRL: - -+---+--------------------------------------------+------------+----------------------------------------------------------------------------+ -| # | Issue | Jira ID | Description | -+---+--------------------------------------------+------------+----------------------------------------------------------------------------+ -| 1 | Intermittent failures in Honeycomb startup | HC2VPP-199 | During test setup Honeycomb sometimes fails to start. Does not appear to | -| | | | affect measured results, only fails test execution ocassionally. | -+---+--------------------------------------------+------------+----------------------------------------------------------------------------+ diff --git a/docs/report/honeycomb_performance_tests/documentation.rst b/docs/report/honeycomb_performance_tests/documentation.rst deleted file mode 100644 index 6b15bde6ee..0000000000 --- a/docs/report/honeycomb_performance_tests/documentation.rst +++ /dev/null @@ -1,5 +0,0 @@ -Documentation -============= - -`CSIT Honeycomb Performance Tests Documentation`_ contains detailed -functional description and input parameters for each test case. diff --git a/docs/report/honeycomb_performance_tests/index.rst b/docs/report/honeycomb_performance_tests/index.rst deleted file mode 100644 index 3177494395..0000000000 --- a/docs/report/honeycomb_performance_tests/index.rst +++ /dev/null @@ -1,11 +0,0 @@ -Honeycomb Performance Tests -=========================== - -.. toctree:: - - overview - csit_release_notes - test_environment - documentation - test_result_data - diff --git a/docs/report/honeycomb_performance_tests/overview.rst b/docs/report/honeycomb_performance_tests/overview.rst deleted file mode 100644 index ee9788ed26..0000000000 --- a/docs/report/honeycomb_performance_tests/overview.rst +++ /dev/null @@ -1,122 +0,0 @@ -Overview -======== - -Tested Physical Topologies --------------------------- - -CSIT VPP performance tests are executed on physical baremetal servers hosted by -LF FD.io project. Testbed physical topology is shown in the figure below.:: - - +------------------------+ +------------------------+ - | | | | - | +------------------+ | | +------------------+ | - | | | | | | | | - | | <-----------------> | | - | | DUT1 | | | | DUT2 | | - | +--^---------------+ | | +---------------^--+ | - | | | | | | - | | SUT1 | | SUT2 | | - +------------------------+ +------------------^-----+ - | | - | | - | +-----------+ | - | | | | - +------------------> TG <------------------+ - | | - +-----------+ - -SUT1 runs VPP SW application in Linux user-mode as a -Device Under Test (DUT), and a python script to generate traffic. SUT2 and TG -are unused. -sical connectivity between SUTs and to TG is provided using -different NIC model. Currently installed NIC models include: - -Performance tests involve sending Netconf requests over localhost to the -Honeycomb listener port, and measuring response time. - -Note that reported performance results are specific to the SUTs tested. -Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with -other CPUs are likely to yield different results. - -For detailed LF FD.io test bed specification and physical topology please refer -to `LF FDio CSIT testbed wiki page -`_. - -Performance Tests Coverage --------------------------- - -As of right now, there is only a single Honeycomb performance test. Measuring -response time for a simple read operation, performed synchronously and using -single (not batch) requests. - -Currently the tests do not trigger automatically, but can be run on-demand from -the hc2vpp project. - -Performance Tests Naming ------------------------- - -CSIT |release| follows a common structured naming convention for all -performance and system functional tests, introduced in CSIT |release-1|. - -The naming should be intuitive for majority of the tests. Complete -description of CSIT test naming convention is provided on `CSIT test naming wiki -`_. - -Here few illustrative examples of the new naming usage for performance test -suites: - -#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** - - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching - with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE - on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline - switching with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel - x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery. - - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput - discovery. - -#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, - P2V2P, NIC2VMchain2NIC, P2V2V2P** - - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation- - VirtPortConfig-VMconfig-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports - of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from four vhost interfaces and two VMs, NDR throughput - discovery. - -Methodology: Multi-Core ------------------------ - -**Multi-core Test** - CSIT |release| multi-core tests are executed in the -following thread and core configurations: - -#. 1t - 1 Honeycomb Netconf thread on 1 CPU physical core. -#. 8t - 8 Honeycomb Netconf thread on 8 CPU physical core. -#. 16t - 16 Honeycomb Netconf thread on 16 CPU physical core. - -Traffic generator also uses multiple threads/cores, to simulate multiple -Netconf clients accessing the Honeycomb server. - -Methodology: Performance measurement ------------------------------------- - -The following values are measured and reported in tests: - -- Average request rate. Averaged over the entire test duration, over all client - threads. Negative replies (if any) are not counted and are reported separately. diff --git a/docs/report/honeycomb_performance_tests/test_environment.rst b/docs/report/honeycomb_performance_tests/test_environment.rst deleted file mode 100644 index db187b2c1b..0000000000 --- a/docs/report/honeycomb_performance_tests/test_environment.rst +++ /dev/null @@ -1,22 +0,0 @@ -Test Environment -================ - -To execute performance tests, there are three identical testbeds, each testbed -consists of two DUTs and one TG. - -Server HW Configuration ------------------------ - -See `Performance HW Configuration <../vpp_performance_tests/test_environment.html>`_ - -Additionally, configuration for the Honeycomb client: - - -**Honeycomb Startup Command** - -Use the server mode JIT compiler, increase the default memory size, -metaspace size, and enable NUMA optimizations for the JVM. - -:: - - $ java -server -Xms128m -Xmx512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=512m -XX:+UseNUMA -XX:+UseParallelGC diff --git a/docs/report/honeycomb_performance_tests/test_result_data.rst b/docs/report/honeycomb_performance_tests/test_result_data.rst deleted file mode 100644 index 9e791d6e88..0000000000 --- a/docs/report/honeycomb_performance_tests/test_result_data.rst +++ /dev/null @@ -1,19 +0,0 @@ -Test Result Data -================ - -This section includes summary of Netconf read operation performance. -Performance is reported for Honeycomb running in multiple configurations of -netconf thread(s) and their physical CPU core(s) placement, and for different -read operation targets. - -.. note:: - - Test results have been generated by - `FD.io test executor honeycomb performance jobs`_ with Robot Framework - result files csit-vpp-perf-\*.zip `archived here <../../_static/archive/>`_. - -Honeycomb + Netconf -=================== - -No new data has been generated in the course of this release. Results from -the last successful run are available in `17.07 release report `_. \ No newline at end of file diff --git a/docs/report/index.rst b/docs/report/index.rst index bbcd5cbab8..abd4178833 100644 --- a/docs/report/index.rst +++ b/docs/report/index.rst @@ -1,18 +1,15 @@ -CSIT 18.01 -========== +CSIT 18.01.2 +============ .. toctree:: :numbered: introduction/index vpp_performance_tests/index - dpdk_performance_tests/index - honeycomb_performance_tests/index vpp_functional_tests/index honeycomb_functional_tests/index - vpp_unit_tests/index nsh_sfc_functional_tests/index detailed_test_results/index test_configuration/index test_operational_data/index - csit_framework_documentation/index \ No newline at end of file + csit_framework_documentation/index diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst index 994a53e6d0..48c20ad132 100644 --- a/docs/report/introduction/general_notes.rst +++ b/docs/report/introduction/general_notes.rst @@ -40,20 +40,6 @@ is listed separately, as follows: orchestrated by Kubernetes, with `Ligato `_ for container networking. TRex is used as a traffic generator. -#. **DPDK Performance** - VPP is using DPDK code to control and drive - the NICs and physical interfaces. Tests are used as a baseline to - profile performance of the DPDK sub-system. DPDK tests are executed in - physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for - Phy-to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE - interfaces, range of multi-thread and multi-core configurations. - Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic - generator. - -#. **Honeycomb Performance** - Honeycomb performance tests are executed in - physical FD.io testbeds, focusing on the performance of Honeycomb management - and programming functionality of VPP. Tests cover a range of CRUD operations - executed against VPP. - #. **VPP Functional** - VPP functional tests are executed in virtual FD.io testbeds focusing on VPP packet processing functionality, including network data plane and in -line control plane. Tests cover vNIC-to-vNIC diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index 22bc29f071..4e290639ff 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -36,21 +36,6 @@ CSIT |release| report contains following main sections and sub-sections: test job executions; *Test Environment* - environment description; *Documentation* - CSIT source code documentation for VPP performance tests. -#. **DPDK Performance Tests** - DPDK performance tests executed in - physical FD.io testbeds; *Overview* - tested topologies, test coverage; - *CSIT Release Notes* - changes in CSIT |release|, any known CSIT issues; - *Packet Throughput Graphs* and *Packet Latency Graphs* - - plotted NDR, PDR throughput and latency results from multiple test job - executions; *Test Environment* - environment description; *Documentation* - - CSIT source code documentation for DPDK performance tests. - -#. **Honeycomb Performance Tests** - Honeycomb performance tests executed in - physical FD.io testbeds; *Overview* - tested topologies, test coverage - and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, - added tests, environment or methodology changes, known CSIT issues; - *Test Environment* - environment description; *Documentation* - source - code documentation for Honeycomb performance tests. - #. **VPP Functional Tests** - VPP functional tests executed in virtual FD.io testbeds; *Overview* - tested virtual topologies, test coverage and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, added @@ -65,12 +50,6 @@ CSIT |release| report contains following main sections and sub-sections: *Test Environment* - environment description ; *Documentation* - source code documentation for Honeycomb functional tests. -#. **VPP Unit Tests** - refers to VPP functional unit tests executed as - part of vpp make test verify option within the FD.io VPP project; listed in - this report to give a more complete view about executed VPP functional tests; - *Overview* - short overview of unit test framework and executed tests; - *Documentation* - source code documentation of VPP unit tests. - #. **NSH_SFC Functional Tests** - NSH_SFC functional tests executed in virtual FD.io testbeds; *Overview* - tested virtual topologies, test coverage and naming specifics; *CSIT Release Notes* - changes in CSIT @@ -80,7 +59,7 @@ CSIT |release| report contains following main sections and sub-sections: #. **Detailed Test Results** - auto-generated results from CSIT jobs executions using CSIT Robot Framework output files as source data; *VPP - Performance Results*, *DPDK Performance Results*, *VPP Functional + Performance Results*, *VPP Functional Results*, *Honeycomb Functional Results*, *VPPtest Functional Results*. #. **Test Configuration** - auto-generated DUT configuration data from CSIT jobs diff --git a/docs/report/test_configuration/index.rst b/docs/report/test_configuration/index.rst index 3963896b82..8c231a85cd 100644 --- a/docs/report/test_configuration/index.rst +++ b/docs/report/test_configuration/index.rst @@ -7,4 +7,6 @@ .. toctree:: vpp_performance_configuration/index + vpp_mrr_configuration/index vpp_functional_configuration/index + vpp_functional_configuration_centos/index diff --git a/docs/report/test_configuration/vpp_functional_configuration/index.rst b/docs/report/test_configuration/vpp_functional_configuration/index.rst index aab94bd31c..36f72eba04 100644 --- a/docs/report/test_configuration/vpp_functional_configuration/index.rst +++ b/docs/report/test_configuration/vpp_functional_configuration/index.rst @@ -1,5 +1,5 @@ -VPP Functional Test Configs -=========================== +VPP Functional Test Configs - Ubuntu +==================================== .. note:: diff --git a/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst new file mode 100644 index 0000000000..cf416ac48e --- /dev/null +++ b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst @@ -0,0 +1,12 @@ +VPP Functional Test Configs - CentOS +==================================== + +.. note:: + + Data sources for reported test results: i) `FD.io test executor vpp + functional jobs`_, ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + + vpp_functional_configuration_centos diff --git a/docs/report/test_configuration/vpp_mrr_configuration/index.rst b/docs/report/test_configuration/vpp_mrr_configuration/index.rst new file mode 100644 index 0000000000..72b6ec01cd --- /dev/null +++ b/docs/report/test_configuration/vpp_mrr_configuration/index.rst @@ -0,0 +1,12 @@ +VPP MRR Test Configs +==================== + +.. note:: + + Data sources for reported test results: i) `FD.io test executor vpp + performance jobs`_, ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + + vpp_mrr_configuration diff --git a/docs/report/vpp_performance_tests/impact_meltdown/index.rst b/docs/report/vpp_performance_tests/impact_meltdown/index.rst deleted file mode 100644 index 35ecb7453c..0000000000 --- a/docs/report/vpp_performance_tests/impact_meltdown/index.rst +++ /dev/null @@ -1,164 +0,0 @@ -Impact of Meltdown Patches -========================== - -Following sections list changes to VPP throughput performance after applying -patches addressing security vulnerabilities referred to as Meltdown -(Variant3: Rogue Data Cache Load). Incremental kernel patches are -applied for Ubuntu 16.04LTS as documented on -`Ubuntu SpectreAndMeltdown page `_. -Detailed listing of used software versions and patches is documented in -:ref:`test_environment`. - -NDR and PDR packet throughput results are compared for 1-core/1-thread, -2-cores/2-threads and 4-cores/4-threads VPP configurations, with -reference performance numbers coming from tests without the Meltdown -patches. Tables show test results grouped into Best 20 changes (minimal -performance impact), followed by Worst 20 changes (maximal performance -impact). All results are also provided in downloadable CSV and pretty -ASCII formats. - -NDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-ndr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-ndr-1t1c-top.csv} - } - -NDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-ndr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-ndr-1t1c-bottom.csv} - } - -.. only:: html - - NDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all NDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-impact-ndr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-impact-ndr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-impact-ndr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-impact-ndr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-impact-ndr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-impact-ndr-4t4c-full.txt>`_. - -PDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-pdr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-pdr-1t1c-top.csv} - } - -PDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-pdr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-pdr-1t1c-bottom.csv} - } - -.. only:: html - -PDR Throughput: All Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all PDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-impact-pdr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-impact-pdr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-impact-pdr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-impact-pdr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-impact-pdr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-impact-pdr-4t4c-full.txt>`_. diff --git a/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst b/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst deleted file mode 100644 index eec38e8698..0000000000 --- a/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst +++ /dev/null @@ -1,167 +0,0 @@ -Impact of SpectreAndMeltdown Patches -==================================== - -Following sections list changes to VPP throughput performance after -applying patches addressing security vulnerabilities referred to as: -Meltdown (Variant3: Rogue Data Cache Load) and Spectre (Variant1: Bounds -Check Bypass; Variant2: Branch Target Injection) security -vulnerabilities. Incremental kernel patches for Ubuntu 16.04 LTS as -documented on -`Ubuntu SpectreAndMeltdown page `_. -For Spectre additional Processor microcode and BIOS firmware changes are -applied. Detailed listing of used software versions and patches is -documented in :ref:`test_environment`. - -NDR and PDR packet throughput results are compared for 1-core/1-thread, -2-cores/2-threads and 4-cores/4-threads VPP configurations, with -reference performance numbers coming from tests without the Meltdown -patches. Tables show test results grouped into Best 20 changes (minimal -performance impact), followed by Worst 20 changes (maximal performance -impact). All results are also provided in downloadable CSV and pretty -ASCII formats. - -NDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-top.csv} - } - -NDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-bottom.csv} - } - -.. only:: html - - NDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all NDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-spectre-impact-ndr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-spectre-impact-ndr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-spectre-impact-ndr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-spectre-impact-ndr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-spectre-impact-ndr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-spectre-impact-ndr-4t4c-full.txt>`_. - -PDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-top.csv} - } - -PDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-bottom.csv} - } - -.. only:: html - - PDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all PDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-spectre-impact-pdr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-spectre-impact-pdr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-spectre-impact-pdr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-spectre-impact-pdr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-spectre-impact-pdr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-spectre-impact-pdr-4t4c-full.txt>`_. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst index 50074e1b6a..4507978b66 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst @@ -27,6 +27,7 @@ TGint2-to-SUT2-to-SUT1-to-TGint1. l2 ip4 ip6 + srv6 ip4_tunnels ip6_tunnels vm_vhost diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/srv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/srv6.rst new file mode 100644 index 0000000000..163d892cde --- /dev/null +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/srv6.rst @@ -0,0 +1,48 @@ +SRv6 +==== + +This section includes summary graphs of VPP Phy-to-Phy packet latency +with SRv6 measured at 50% of discovered NDR throughput +rate. Latency is reported for VPP running in multiple configurations of +VPP worker thread(s), a.k.a. VPP data plane thread(s), and their +physical CPU core(s) placement. + +VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-1t1c-ethip6-srv6-ndrdisc-lat50} + \label{fig:78B-1t1c-ethip6-srv6-ndrdisc-lat50} + \end{figure} + +*Figure 1. VPP 1thread 1core - packet latency for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +VPP packet latency in 2t2c setup (2thread, 2core) is presented in the graph below. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-2t2c-ethip6-srv6-ndrdisc-lat50} + \label{fig:78B-2t2c-ethip6-srv6-ndrdisc-lat50} + \end{figure} + +*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst index 6bfe4fe1af..ec3c9f9173 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst @@ -39,6 +39,7 @@ and their indices. l2 ip4 ip6 + srv6 ip4_tunnels ip6_tunnels vm_vhost diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/srv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/srv6.rst new file mode 100644 index 0000000000..0df40ac636 --- /dev/null +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/srv6.rst @@ -0,0 +1,99 @@ +SRv6 +==== + +Following sections include summary graphs of VPP Phy-to-Phy performance +with SRv6, including NDR throughput (zero packet loss) +and PDR throughput (<0.5% packet loss). Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +~~~~~~~~~~~~~~ + +VPP NDR 78B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-1t1c-ethip6-srv6-ndrdisc} + \label{fig:78B-1t1c-ethip6-srv6-ndrdisc} + \end{figure} + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +VPP NDR 78B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-2t2c-ethip6-srv6-ndrdisc} + \label{fig:78B-2t2c-ethip6-srv6-ndrdisc} + \end{figure} + +*Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +~~~~~~~~~~~~~~ + +VPP PDR 78B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-1t1c-ethip6-srv6-pdrdisc} + \label{fig:78B-1t1c-ethip6-srv6-pdrdisc} + \end{figure} + +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +VPP PDR 78B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{78B-2t2c-ethip6-srv6-pdrdisc} + \label{fig:78B-2t2c-ethip6-srv6-pdrdisc} + \end{figure} + +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 +Routed-Forwarding.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.rst new file mode 100644 index 0000000000..4fb8791dff --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_memif.rst @@ -0,0 +1,66 @@ +Container memif Connections +=========================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-memif-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-container-memif-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy L2 Ethernet Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-memif-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-container-memif-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy L2 Ethernet Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst new file mode 100644 index 0000000000..4e8ff4cd25 --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst @@ -0,0 +1,110 @@ +Container Orchestrated Topologies +================================= + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 and 10ge2p1x710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-orchestrated-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-container-orchestrated-tsa-ndrdisc} + \end{figure} + +*Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-container-orchestrated-tsa-ndrdisc} + \label{fig:10ge2p1x710-64B-container-orchestrated-tsa-ndrdisc} + \end{figure} + +*Figure 2. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 and 10ge2p1x710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-container-orchestrated-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-container-orchestrated-tsa-pdrdisc} + \end{figure} + +*Figure 3. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-container-orchestrated-tsa-pdrdisc} + \label{fig:10ge2p1x710-64B-container-orchestrated-tsa-pdrdisc} + \end{figure} + +*Figure 4. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet +Switching (base).* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst index ddce548b6c..dca77c5b7b 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst @@ -1,3 +1,5 @@ +.. _throughput_speedup_multi_core: + Throughput Speedup Multi-Core ============================= @@ -18,3 +20,10 @@ threaded VPP configurations relative to 1-core configurations. l2 ip4 ip6 + srv6 + ip4_tunnels + ip6_tunnels + vm_vhost + container_memif + container_orchestrated + ipsec diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst index af9bd255be..0b2b7ef791 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4.rst @@ -31,24 +31,12 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-ip4-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip4 && grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip4 - $ grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - *Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + NIC 40ge2p1xl710 ~~~~~~~~~~~~~~~~ @@ -65,24 +53,12 @@ NIC 40ge2p1xl710 \label{fig:40ge2p1xl710-64B-ip4-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip4 && grep -P '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip4 - $ grep -P '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - *Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + PDR Throughput -------------- @@ -106,20 +82,8 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-ip4-tsa-pdrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip4 && grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip4 - $ grep -E '64B-(1t1c|2t2c|4t4c)-ethip4-ip4(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - *Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4_tunnels.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4_tunnels.rst new file mode 100644 index 0000000000..001584b856 --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip4_tunnels.rst @@ -0,0 +1,66 @@ +IPv4 Overlay Tunnels +==================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-ethip4-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-ethip4-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-ethip4-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-ethip4-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst index 077b330652..e8c70d0c18 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6.rst @@ -32,24 +32,12 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-78B-ip6-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip6 && grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip6 - $ grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 10ge2p1x520* - *Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + NIC 40ge2p1xl710 ~~~~~~~~~~~~~~~~ @@ -66,24 +54,12 @@ NIC 40ge2p1xl710 \label{fig:40ge2p1xl710-78B-ip6-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip6 && grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip6 - $ grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-ndrdisc' 40ge2p1xl710* - *Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + PDR Throughput -------------- @@ -107,20 +83,8 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-78B-ip6-tsa-pdrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/ip6 && grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-pdrdisc' 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ cd tests/vpp/perf/ip6 - $ grep -E '78B-(1t1c|2t2c|4t4c)-ethip6-ip6(base|scale[a-z0-9]*)*-pdrdisc' 10ge2p1x520* - *Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized PDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6_tunnels.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6_tunnels.rst new file mode 100644 index 0000000000..3965242e8c --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ip6_tunnels.rst @@ -0,0 +1,66 @@ +IPv6 Overlay Tunnels +==================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-ethip6-tsa-ndrdisc} + \label{fig:10ge2p1x520-78B-ethip6-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-ethip6-tsa-pdrdisc} + \label{fig:10ge2p1x520-78B-ethip6-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ipsec.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ipsec.rst new file mode 100644 index 0000000000..a5a4d7d1bc --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/ipsec.rst @@ -0,0 +1,71 @@ +IPSec Crypto HW: IP4 Routed-Forwarding +====================================== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. +VPP IPSec encryption is accelerated using DPDK cryptodev +library driving Intel Quick Assist (QAT) crypto PCIe hardware cards. +Performance is reported for VPP running in multiple configurations of +VPP worker thread(s), a.k.a. VPP data plane thread(s), and their +physical CPU core(s) placement. + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 40ge2p1xl710 network interface card. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-ipsechw-tsa-ndrdisc} + \label{fig:40ge2p1xl710-64B-ipsechw-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy IPSEC HW.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 40ge2p1xl710 network interface card. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +VPP PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-ipsechw-tsa-pdrdisc} + \label{fig:40ge2p1xl710-64B-ipsechw-tsa-pdrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy IPSEC HW.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst index 2df3fb6229..5db8eba307 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/l2.rst @@ -31,23 +31,12 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-l2-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/l2 && grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" tests/vpp/perf/l2/10ge2p1x520* - *Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy L2 Ethernet Switching.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + NIC 40ge2p1xl710 ~~~~~~~~~~~~~~~~ @@ -64,23 +53,12 @@ NIC 40ge2p1xl710 \label{fig:40ge2p1xl710-64B-l2-tsa-ndrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/l2 && grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" 40ge2p1xl710* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*ndrdisc" tests/vpp/perf/l2/40ge2p1xl710* - *Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized NDR Throughput for Phy-to-Phy L2 Ethernet Switching.* +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + PDR Throughput -------------- @@ -104,19 +82,8 @@ NIC 10ge2p1x520 \label{fig:10ge2p1x520-64B-l2-tsa-pdrdisc} \end{figure} -CSIT source code for the test cases used for above plots can be found in CSIT -git repository: - -.. only:: html - - .. program-output:: cd ../../../../../ && set +x && cd tests/vpp/perf/l2 && grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*pdrdisc" 10ge2p1x520* - :shell: - -.. only:: latex - - .. code-block:: bash - - $ grep -E "64B-(1t1c|2t2c|4t4c)-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn|l2bdscale.*|l2dbscale.*)-(eth.*)*pdrdisc" tests/vpp/perf/l2/10ge2p1x520* - *Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/srv6.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/srv6.rst new file mode 100644 index 0000000000..4ece4d538e --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/srv6.rst @@ -0,0 +1,68 @@ +SRv6 +==== + +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. Input data +used for the graphs comes from Phy-to-Phy 78B performance tests with VPP +SRv6, including NDR throughput (zero packet loss) and +PDR throughput (<0.5% packet loss). + +NDR Throughput +-------------- + +VPP NDR 78B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. + + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-srv6-tsa-ndrdisc} + \label{fig:10ge2p1x520-78B-srv6-tsa-ndrdisc} + \end{figure} + +*Figure 1. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +-------------- + +VPP PDR 78B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520 network interface card. PDR +measured for 0.5% packet loss ratio. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-78B-srv6-tsa-pdrdisc} + \label{fig:10ge2p1x520-78B-srv6-tsa-pdrdisc} + \end{figure} + +*Figure 3. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy SRv6.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/vm_vhost.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/vm_vhost.rst new file mode 100644 index 0000000000..9bd49b42ab --- /dev/null +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/vm_vhost.rst @@ -0,0 +1,229 @@ +VM vhost Connections +==================== +Following sections include Throughput Speedup Analysis for VPP multi- +core multi-thread configurations with no Hyper-Threading, specifically +for tested 2t2c (2threads, 2cores) and 4t4c scenarios. 1t1c throughput +results are used as a reference for reported speedup ratio. Input data +used for the graphs comes from Phy-to-Phy 64B performance tests with +VM vhost-user, including NDR throughput (zero packet loss) and +PDR throughput (<0.5% packet loss). + +NDR Throughput +-------------- + +VPP NDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520, 10ge2p1x710 and 40ge2p1xl710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel1-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel1-tsa-ndrdisc} + \end{figure} + +*Figure 1a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel2-tsa-ndrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel2-tsa-ndrdisc} + \end{figure} + +*Figure 1b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-vhost-sel2-tsa-ndrdisc} + \label{fig:10ge2p1x710-64B-vhost-sel2-tsa-ndrdisc} + \end{figure} + +*Figure 2. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel1-tsa-ndrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel1-tsa-ndrdisc} + \end{figure} + +*Figure 3a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel2-tsa-ndrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel2-tsa-ndrdisc} + \end{figure} + +*Figure 3b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +NDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +PDR Throughput +-------------- + +VPP PDR 64B packet throughput speedup ratio is presented in the graphs +below for 10ge2p1x520, 10ge2p1x710 and 40ge2p1xl710 network interface cards. + +NIC 10ge2p1x520 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel1-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel1-tsa-pdrdisc} + \end{figure} + +*Figure 4a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x520-64B-vhost-sel2-tsa-pdrdisc} + \label{fig:10ge2p1x520-64B-vhost-sel2-tsa-pdrdisc} + \end{figure} + +*Figure 4b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +NIC 10ge2p1x710 +~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{10ge2p1x710-64B-vhost-sel2-tsa-pdrdisc} + \label{fig:10ge2p1x710-64B-vhost-sel2-tsa-pdrdisc} + \end{figure} + +*Figure 5. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +NIC 40ge2p1xl710 +~~~~~~~~~~~~~~~~ + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel1-tsa-pdrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel1-tsa-pdrdisc} + \end{figure} + +*Figure 6a. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. + +.. raw:: html + + + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/vpp/}} + \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{40ge2p1xl710-64B-vhost-sel2-tsa-pdrdisc} + \label{fig:40ge2p1xl710-64B-vhost-sel2-tsa-pdrdisc} + \end{figure} + +*Figure 6b. Throughput Speedup Analysis - Multi-Core Speedup Ratio - Normalized +PDR Throughput for Phy-to-Phy VM vhost-user selected TCs.* + +CSIT source code for the test cases used for above plots can be found in +`CSIT git repository `_. diff --git a/docs/report/vpp_unit_tests/documentation.rst b/docs/report/vpp_unit_tests/documentation.rst deleted file mode 100644 index 304db1121b..0000000000 --- a/docs/report/vpp_unit_tests/documentation.rst +++ /dev/null @@ -1,6 +0,0 @@ -Documentation -============= - -For complete description of the VPP Test Framework including anatomy of a test -case and detailed documentation of existing VPP unit test cases please refer -to the `VPP test framework documentation`_. diff --git a/docs/report/vpp_unit_tests/index.rst b/docs/report/vpp_unit_tests/index.rst deleted file mode 100644 index 525eb1230d..0000000000 --- a/docs/report/vpp_unit_tests/index.rst +++ /dev/null @@ -1,8 +0,0 @@ -VPP Unit Tests -============== - -.. toctree:: - - overview - documentation - diff --git a/docs/report/vpp_unit_tests/overview.rst b/docs/report/vpp_unit_tests/overview.rst deleted file mode 100644 index 9472680aa5..0000000000 --- a/docs/report/vpp_unit_tests/overview.rst +++ /dev/null @@ -1,87 +0,0 @@ -Overview -======== - -.. note:: - - This section includes an abbreviated version of the VPP Test Framework - overview maintained within the VPP project. Complete overview can be found - in `VPP test framework documentation`_. - -VPP Unit Test Framework ------------------------ - -VPP Test Framework is used to ease writing, running and debugging unit tests -for the VPP. It is based on python as a high level language to allow rapid -test development. scapy\_ is used as a tool for creating and dissecting -packets. - -VPP Test Framework does not send any packets to VPP directly. Traffic is -instead injected using VPP packet-generator interfaces. Packets are written -into a temporary .pcap file, which is then read by the VPP code with packets -getting injected into the VPP processing nodes. - -Similarly, VPP does not send any packets to VPP Test Framework directly. -Instead, VPP packet capture feature is used to capture and write packets to a -temporary .pcap file, which is then read and analyzed by the VPP Test -Framework. - -For complete description of the VPP Test Framework including anatomy of a test -case and detailed documentation of existing VPP unit test cases please refer -to the `VPP test framework documentation`_ - -Unit Tests Coverage -------------------- - -Following VPP functional test areas are covered in VPP unit test code included -in VPP rls1710 with results listed in this report: - -- ACL Security - stateful and stateless security-groups access-control-lists. -- APIs - VAPI, VOM, PAPI, JVPP. -- ARP - ARP, proxy ARP, static arp. -- BFD - API, Authentication, Authentication Change, CLI. -- BFD IPv4 - sessions operation. -- BFD IPv6 - sessions operation. -- BIER - Bit Indexed Explicit Replication. -- Classifier - classification with IP ACL, MAC ACL, IP PBR. -- Container Integration - IPv4, IPv6 local-spoof connectivity tests. -- CRUD Loopback - create, read, update, delete Loopback interfaces. -- DHCP - DHCPv4/v6 Client and Proxy. -- Distributed Virtual Router. -- DS-Lite Softwire - softwire termination. -- FIB - baseline and scale tests. -- Flowprobe. -- Geneve Tunnels. -- GRE Tunnels - GRE IPv4/IPv6 tunnel, L2, VRF tests. -- GTPU Tunnels - baseline GTPU tests. -- IP Multicast Routing - IPv4/IPv6 multicast replication, connected source check. -- IPSec - baseline IPSec sanity tests. -- IPv4 FIB CRUD - add/update/delete IPv4 routes. -- IPv4 Routing. -- IP4 VRF Multi-instance - create, read, update, delete and verify IPv4 VRFs. -- IPv6 Routing - baseline FIB operations, NS/RS exception handling. -- IP6 VRF Multi-instance - create, read, update, delete and verify IPv6 VRFs. -- IRB Integrated Routing-Bridging. -- Kube-proxy - data plane NAT tests. -- L2 FIB CRUD - add/update/delete L2 MAC entries. -- L2BD Multi-instance. -- L2BD Switching - L2 Bridge-Domain baseline tests incl. single- and dual-loop. -- L2XC Multi-instance - L2 cross-connect multi-instance tests. -- L2XC Switching - L2 cross-connect baseline tests incl. single- and dual-loop. -- LISP Tunnels - basic LISP tests. -- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6. -- MACIP Access Control - ingress access control for IPv4, IPv6 with L2BDP and IP routing. -- MAP Softwires - softwire termination. -- MFIB Multicast FIB. -- MPLS Switching - MPLS baseline, prefix independent convergence for MPLS PE. -- NAT44 - NAT44 tests, IPFIX logging, VRF awareness, deterministic CGNAT. -- NAT64 - NAT64 static and dynamic translation tests. -- P2P Ethernet Subinterface. -- PPPoE Encapsulation. -- SPAN Switch Port Analyzer - packet mirroring. -- SRv6 Routing - Segment Routing IPv6 tests. -- TCP/IP Stack - unit tests, builtin client/server transfers. -- UDP Stack - unit tests. -- VTR VLAN Tag Rewrites - VLAN tag rewrite tests. -- VXLAN Tunnels - baseline VXLAN tests including multicast. -- VXLAN-GPE Tunnels - baseline VXLAN-GPE tunneling including multicast. -- Other Tests - ping, session, template verification, timer tests. -- cgit 1.2.3-korg