aboutsummaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
authorVratko Polak <vrpolak@cisco.com>2021-11-10 13:36:27 +0100
committerVratko Polak <vrpolak@cisco.com>2021-11-10 13:36:27 +0100
commitf0f6265a54cc65e27daf04329df209c303518f04 (patch)
treee2b71c9223d04113044965549078a725fa76a3bb /docs
parent80e8f662317324156f9997e788f7ed2ae46401f8 (diff)
Report: Update known issues
+ Remove MLRsearch comment, it happened a release before. + Separated known issues to fixed, old and new. + RCA section present, but empty yet. Change-Id: Ic713319017d6c2eb7ba2750de7c27de5237ee2c7 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Diffstat (limited to 'docs')
-rw-r--r--docs/report/vpp_performance_tests/csit_release_notes.rst73
1 files changed, 47 insertions, 26 deletions
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 4f986ebd24..b057af187e 100644
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -50,47 +50,68 @@ Changes in |csit-release|
Known Issues
------------
-List of known issues in |csit-release| for VPP performance tests:
+Fixed
+_____
+
+Issues reported in previous releases which were fixed in this release:
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
| # | JiraID | Issue Description |
+====+=========================================+===========================================================================================================+
-| 1 | `CSIT-1782 | Multicore AVF tests are failing when trying to create interface. The issue persists after FVL FW upgrade. |
-| | <https://jira.fd.io/browse/CSIT-1782>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 2 | `CSIT-1763 | Adapt ramp-up phase of nat44 tests for different frame sizes. |
-| | <https://jira.fd.io/browse/CSIT-1763>`_ | Currently ramp-up phase rate and duration values are correctly set for tests with 64B frame size. |
+| 1 | `CSIT-1789 | AVF driver does not perform RSS in a deterministic way. |
+| | <https://jira.fd.io/browse/CSIT-1789>`_ | VPP now uses the same RSS key with AVF driver as with DPDK driver. |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 3 | `CSIT-1671 | All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much |
-| | <https://jira.fd.io/browse/CSIT-1671>`_ | longer to program VPP). This needs to be addressed on the PAPI side. |
-| +-----------------------------------------+ The usual PAPI library spends too much time parsing arguments, so even with async processing (hundreds of |
-| | `VPP-1763 | commands in flight over socket), the VPP configuration for large scale tests (millions of messages) takes |
-| | <https://jira.fd.io/browse/VPP-1763>`_ | too long. |
+| 2 | `CSIT-1786 | IP4 and IP6 scale tests failing with no traffic forwarded. |
+| | <https://jira.fd.io/browse/CSIT-1786>`_ | CSIT replaced the old single VAT command by file full of "exec" CLI commands executed by VAT. |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 4 | `CSIT-1789 | AVF driver does not perform RSS in a deterministic way. |
-| | <https://jira.fd.io/browse/CSIT-1789>`_ | This increases standard deviation of tests with small number of flows (mainly ipsec). |
+
+Old
+___
+
+Issues reported in previous releases which still affect the current results.
+
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 5 | `CSIT-1780 | IPSEC SW async scheduler MRR tests failing with no traffic forwarded. |
-| | <https://jira.fd.io/browse/CSIT-1780>`_ | |
+| # | JiraID | Issue Description |
++====+=========================================+===========================================================================================================+
+| 3 | `CSIT-1782 | Multicore AVF tests are failing when trying to create interface. |
+| | <https://jira.fd.io/browse/CSIT-1782>`_ | Frequency is reduced by s CSIT workaround, but occasional failures do still happen. |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 6 | `CSIT-1786 | IP4 and IP6 scale tests failing with no traffic forwarded. |
-| | <https://jira.fd.io/browse/CSIT-1786>`_ | Issue reported to VPP devs. |
+| 4 | `CSIT-1671 | All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much |
+| | <https://jira.fd.io/browse/CSIT-1671>`_ | longer to program VPP). This needs to be addressed on the PAPI side. |
+| +-----------------------------------------+ Currently, the time critical code uses VAT running large files with exec statements and CLI commands. |
+| | `VPP-1763 | Still, we needed to reduce the number of scale tests run to keep overall duration reasonable. |
+| | <https://jira.fd.io/browse/VPP-1763>`_ | More improvements needed to achieve sufficient configuration speed. |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 7 | `CSIT-1785 | NAT44ED tests failing to establish all TCP sessions. |
-| | <https://jira.fd.io/browse/CSIT-1785>`_ | |
+| 5 | `CSIT-1789 | IPSEC SW async scheduler tests show bad behavior. |
+| | <https://jira.fd.io/browse/CSIT-1789>`_ | VPP code is not behaving correctly when crypto workers are the bottleneck. |
+| +-----------------------------------------+ |
+| 5 | `VPP-1998 | |
+| | <https://jira.fd.io/browse/VPP-1998>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 8 | `CSIT-1791 | Performance regression in RDMA tests, due to CSIT environment changes. |
-| | <https://jira.fd.io/browse/CSIT-1791>`_ | |
+| 6 | `CSIT-1785 | NAT44ED tests failing to establish all TCP sessions. |
+| | <https://jira.fd.io/browse/CSIT-1785>`_ | At least for max scale, in allotted time (limited by session 500s timeout) due to worse |
+| +-----------------------------------------+ slow path performance than previously measured and calibrated for. |
+| | `VPP-1972 | CSIT removed the max scale NAT tests to avoid this issue. |
+| | <https://jira.fd.io/browse/VPP-1972>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+| 7 | `CSIT-1791 | Performance regression in RDMA tests, due to CSIT environment changes. |
+| | <https://jira.fd.io/browse/CSIT-1791>`_ | Two symptoms: 1. 10-20% regression across most tests. 2. DUT performance cap just below 38 Mpps. |
++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+
+New
+___
+
+No new issues appeared in |csit-release| for VPP performance tests,
+except for the performance changes listed below.
Root Cause Analysis for Performance Changes
-------------------------------------------
List of RCAs in |csit-release| for VPP performance changes:
-+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| # | JiraID | Issue Description |
-+====+=========================================+===========================================================================================================+
-| 1 | `VPP-1972 | One VPP change has decreased performance of NAT44ed processing, mostly slow path. |
-| | <https://jira.fd.io/browse/VPP-1972>`_ | |
-+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
++----+-----------------------------------------+------------------------------------------------------------+
+| # | JiraID | Issue Description |
++====+=========================================+============================================================+
+| 0 | | To be updated when descriptions in Jira tickets are ready. |
++----+-----------------------------------------+------------------------------------------------------------+
, flags=self.flags) / pkt ) @classmethod def ip_range(cls, s, e): """range of remote ip's""" tmp = cls.pg0.remote_ip6.rsplit(":", 1)[0] return ("%s:%x" % (tmp, i) for i in range(s, e)) def encap_mcast(self, pkt, src_ip, src_mac, vni): """ Encapsulate the original payload frame by adding VXLAN header with its UDP, IP and Ethernet fields """ return ( Ether(src=src_mac, dst=self.mcast_mac) / IPv6(src=src_ip, dst=self.mcast_ip6) / UDP(sport=self.dport, dport=self.dport, chksum=0) / VXLAN(vni=vni, flags=self.flags) / pkt ) def decapsulate(self, pkt): """ Decapsulate the original payload frame by removing VXLAN header """ # check if is set I flag self.assertEqual(pkt[VXLAN].flags, int("0x8", 16)) return pkt[VXLAN].payload # Method for checking VXLAN encapsulation. # def check_encapsulation(self, pkt, vni, local_only=False, mcast_pkt=False): # TODO: add error messages # Verify source MAC is VPP_MAC and destination MAC is MY_MAC resolved # by VPP using ARP. self.assertEqual(pkt[Ether].src, self.pg0.local_mac) if not local_only: if not mcast_pkt: self.assertEqual(pkt[Ether].dst, self.pg0.remote_mac) else: self.assertEqual(pkt[Ether].dst, type(self).mcast_mac) # Verify VXLAN tunnel source IP is VPP_IP and destination IP is MY_IP. self.assertEqual(pkt[IPv6].src, self.pg0.local_ip6) if not local_only: if not mcast_pkt: self.assertEqual(pkt[IPv6].dst, self.pg0.remote_ip6) else: self.assertEqual(pkt[IPv6].dst, type(self).mcast_ip6) # Verify UDP destination port is VXLAN 4789, source UDP port could be # arbitrary. self.assertEqual(pkt[UDP].dport, self.dport) # Verify UDP checksum self.assert_udp_checksum_valid(pkt, ignore_zero_checksum=False) # Verify VNI self.assertEqual(pkt[VXLAN].vni, vni) @classmethod def create_vxlan_flood_test_bd(cls, vni, n_ucast_tunnels, port): # Create 10 ucast vxlan tunnels under bd start = 10 end = start + n_ucast_tunnels for dest_ip6 in cls.ip_range(start, end): # add host route so dest ip will not be resolved rip = VppIpRoute( cls, dest_ip6, 128, [VppRoutePath(cls.pg0.remote_ip6, INVALID_INDEX)], register=False, ) rip.add_vpp_config() r = VppVxlanTunnel( cls, src=cls.pg0.local_ip6, src_port=port, dst_port=port, dst=dest_ip6, vni=vni, ) r.add_vpp_config() cls.vapi.sw_interface_set_l2_bridge(r.sw_if_index, bd_id=vni) @classmethod def add_mcast_tunnels_load(cls): cls.add_del_mcast_tunnels_load(is_add=1) @classmethod def del_mcast_tunnels_load(cls): cls.add_del_mcast_tunnels_load(is_add=0) # Class method to start the VXLAN test case. # Overrides setUpClass method in VppTestCase class. # Python try..except statement is used to ensure that the tear down of # the class will be executed even if exception is raised. # @param cls The class pointer. @classmethod def setUpClass(cls): super(TestVxlan6, cls).setUpClass() try: cls.flags = 0x8 # Create 2 pg interfaces. cls.create_pg_interfaces(range(4)) for pg in cls.pg_interfaces: pg.admin_up() # Configure IPv6 addresses on VPP pg0. cls.pg0.config_ip6() # Resolve MAC address for VPP's IP address on pg0. cls.pg0.resolve_ndp() # Our Multicast address cls.mcast_ip6 = "ff0e::1" cls.mcast_mac = util.mcast_ip_to_mac(cls.mcast_ip6) except Exception: super(TestVxlan6, cls).tearDownClass() raise @classmethod def tearDownClass(cls): super(TestVxlan6, cls).tearDownClass() def setUp(self): super(TestVxlan6, self).setUp() def createVxLANInterfaces(self, port=4789): # Create VXLAN VTEP on VPP pg0, and put vxlan_tunnel0 and pg1 # into BD. self.dport = port self.single_tunnel_vni = 0x12345 self.single_tunnel_bd = 1 r = VppVxlanTunnel( self, src=self.pg0.local_ip6, dst=self.pg0.remote_ip6, src_port=self.dport, dst_port=self.dport, vni=self.single_tunnel_vni, ) r.add_vpp_config() self.vapi.sw_interface_set_l2_bridge( rx_sw_if_index=r.sw_if_index, bd_id=self.single_tunnel_bd ) self.vapi.sw_interface_set_l2_bridge( rx_sw_if_index=self.pg1.sw_if_index, bd_id=self.single_tunnel_bd ) # Setup vni 2 to test multicast flooding self.n_ucast_tunnels = 10 self.mcast_flood_bd = 2 self.create_vxlan_flood_test_bd( self.mcast_flood_bd, self.n_ucast_tunnels, self.dport ) r = VppVxlanTunnel( self, src=self.pg0.local_ip6, dst=self.mcast_ip6, src_port=self.dport, dst_port=self.dport, mcast_sw_if_index=1, vni=self.mcast_flood_bd, ) r.add_vpp_config() self.vapi.sw_interface_set_l2_bridge( rx_sw_if_index=r.sw_if_index, bd_id=self.mcast_flood_bd ) self.vapi.sw_interface_set_l2_bridge( rx_sw_if_index=self.pg2.sw_if_index, bd_id=self.mcast_flood_bd ) # Setup vni 3 to test unicast flooding self.ucast_flood_bd = 3 self.create_vxlan_flood_test_bd( self.ucast_flood_bd, self.n_ucast_tunnels, self.dport ) self.vapi.sw_interface_set_l2_bridge( rx_sw_if_index=self.pg3.sw_if_index, bd_id=self.ucast_flood_bd ) # Set scapy listen custom port for VxLAN bind_layers(UDP, VXLAN, dport=self.dport) # Method to define VPP actions before tear down of the test case. # Overrides tearDown method in VppTestCase class. # @param self The object pointer. def tearDown(self): super(TestVxlan6, self).tearDown() def show_commands_at_teardown(self): self.logger.info(self.vapi.cli("show bridge-domain 1 detail")) self.logger.info(self.vapi.cli("show bridge-domain 2 detail")) self.logger.info(self.vapi.cli("show bridge-domain 3 detail")) self.logger.info(self.vapi.cli("show vxlan tunnel")) def encap_fragmented_packet(self): frame = ( Ether(src="00:00:00:00:00:02", dst="00:00:00:00:00:01") / IP(src="4.3.2.1", dst="1.2.3.4") / UDP(sport=20000, dport=10000) / Raw(b"\xa5" * 1000) ) frags = util.fragment_rfc791(frame, 400) self.pg1.add_stream(frags) self.pg0.enable_capture() self.pg_start() out = self.pg0.get_capture(3) payload = [] for pkt in out: payload.append(self.decapsulate(pkt)) self.check_encapsulation(pkt, self.single_tunnel_vni) reassembled = util.reassemble4(payload) self.assertEqual(Ether(raw(frame))[IP], reassembled[IP]) """ Tests with default port (4789) """ def test_decap(self): """Decapsulation test from BridgeDoman """ self.createVxLANInterfaces() super(TestVxlan6, self).test_decap() def test_encap(self): """Encapsulation test from BridgeDoman """ self.createVxLANInterfaces() super(TestVxlan6, self).test_encap() def test_encap_fragmented_packet(self): """Encapsulation test send fragments from pg1 Verify receipt of encapsulated frames on pg0 """ self.createVxLANInterfaces() self.encap_fragmented_packet() def test_ucast_flood(self): """Unicast flood test from BridgeDoman """ self.createVxLANInterfaces() super(TestVxlan6, self).test_ucast_flood() def test_mcast_flood(self): """Multicast flood test from BridgeDoman """ self.createVxLANInterfaces() super(TestVxlan6, self).test_mcast_flood() def test_mcast_rcv(self): """Multicast receive test from BridgeDoman """ self.createVxLANInterfaces() super(TestVxlan6, self).test_mcast_rcv() """ Tests with custom port """ def test_decap_custom_port(self): """Decapsulation test custom port from BridgeDoman """ self.createVxLANInterfaces(1111) super(TestVxlan6, self).test_decap() def test_encap_custom_port(self): """Encapsulation test custom port from BridgeDoman """ self.createVxLANInterfaces(1111) super(TestVxlan6, self).test_encap() def test_ucast_flood_custom_port(self): """Unicast flood test custom port from BridgeDoman """ self.createVxLANInterfaces(1111) super(TestVxlan6, self).test_ucast_flood() def test_mcast_flood_custom_port(self): """Multicast flood test custom port from BridgeDoman """ self.createVxLANInterfaces(1111) super(TestVxlan6, self).test_mcast_flood() def test_mcast_rcv_custom_port(self): """Multicast receive test custom port from BridgeDoman """ self.createVxLANInterfaces(1111) super(TestVxlan6, self).test_mcast_rcv() if __name__ == "__main__": unittest.main(testRunner=VppTestRunner)