diff options
| author |   Maciek Konstantynowicz <mkonstan@cisco.com> | 2017-11-01 14:33:12 +0000 |
|---|---|---|
| committer |   Peter Mikus <pmikus@cisco.com> | 2017-11-02 05:47:16 +0000 |
| commit | 1749fc8da247a5aacd772b7f7b7353b3ee0733db (patch) | |
| tree | 3334025a5bddf5dda00893e9fb4db958b792d864 /docs | |
| parent | fc5eab7361265b97e0170726042aece7133e672a (diff) | |
take-2: csit report static content updates for rls1710.
Change-Id: I097d62a36473303f9e0c6c06a38a68e077f86b14
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst | 554 | ||||
| -rw-r--r-- | docs/report/introduction/general_notes.rst | 53 | ||||
| -rw-r--r-- | docs/report/introduction/overview.rst | 7 | ||||
| -rw-r--r-- | docs/report/vpp_unit_tests/overview.rst | 62 |
4 files changed, 533 insertions, 143 deletions
diff --git a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst index 63bcb1ec2c..f20bb98055 100644 --- a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst +++ b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst @@ -8,14 +8,43 @@ CRUD Loopback test_crud (test_interface_crud.TestLoopbackInterfaceCRUD) OK
test_down (test_interface_crud.TestLoopbackInterfaceCRUD) OK
-Flow-per-packet plugin
-``````````````````````
+Flowprobe feature
+`````````````````
::
==============================================================================
- Flow-per-packet plugin: test both L2 and IP4 reporting
+ Re-enable Flowprobe feature
==============================================================================
- Flow per packet L3 test OK
+ disable flowprobe feature after first packets and re-enable SKIP
+ ==============================================================================
+ collect information on Ethernet, IP4 and IP6 datapath (no timers)
+ ==============================================================================
+ no timers, one CFLOW packet, 9 Flows inside OK
+ no timers, two CFLOW packets (mtu=256), 3 Flows in each OK
+ L2 data on IP4 datapath OK
+ L2 data on IP6 datapath OK
+ L2 data on L2 datapath OK
+ L3 data on IP4 datapath OK
+ L3 data on IP6 datapath OK
+ L3 data on L2 datapath OK
+ L4 data on IP4 datapath OK
+ L4 data on IP6 datapath OK
+ L4 data on L2 datapath OK
+ verify templates on IP6 datapath OK
+ verify templates on IP4 datapath OK
+ verify template on L2 datapath OK
+ ==============================================================================
+ Disable Flowprobe feature
+ ==============================================================================
+ disable flowprobe feature after first packets SKIP
+ ==============================================================================
+ Re-enable IPFIX
+ ==============================================================================
+ disable IPFIX after first packets and re-enable after few packets SKIP
+ ==============================================================================
+ Disable IPFIX
+ ==============================================================================
+ disable IPFIX after first packets SKIP
DHCP
````
@@ -25,6 +54,7 @@ DHCP DHCP Test Case
==============================================================================
DHCPv6 Proxy OK
+ DHCP Client OK
DHCPv4 Proxy OK
IPv4 VRF Multi-instance
@@ -70,8 +100,13 @@ BFD IPv6 echo function used SKIP
echo packets looped back SKIP
hold BFD session up SKIP
+ interface with bfd session deleted SKIP
bring BFD session up SKIP
bring BFD session up - first frame looked up by address pair SKIP
+ ==============================================================================
+ BFD-FIB interactions (IPv6)
+ ==============================================================================
+ BFD-FIB interactions SKIP
BFD authentication
``````````````````
@@ -175,6 +210,8 @@ BFD IPv4 stale echo packets don't keep a session up SKIP
no packets when zero remote required min rx interval SKIP
+
+
IPv6 Tests
``````````
::
@@ -184,29 +221,38 @@ IPv6 Tests ==============================================================================
IPv6 FIB test OK
IPv6 Neighbour Solicitation Exceptions OK
+ ND Duplicates OK
IPv6 Router Solicitation Exceptions OK
==============================================================================
IPv6 disabled
==============================================================================
IP Disabled OK
- IPv6 ND Proxy
==============================================================================
IPv6 ND ProxyTest Case
==============================================================================
IPv6 Proxy ND OK
==============================================================================
+ IPv6 Load-Balancing
+ ==============================================================================
+ IPv6 Load-Balancing OK
+ ==============================================================================
IPv6 routes via NULL
==============================================================================
IP NULL route OK
-SPAN Test
-`````````
+SPAN Tests
+``````````
::
==============================================================================
SPAN Test Case
==============================================================================
- SPAN test OK
+ SPAN device rx mirror test OK
+ SPAN l2 tx mirror test OK
+ SPAN l2 rx mirror test OK
+ SPAN l2 rx mirror into gre-subif+vtr OK
+ SPAN l2 rx mirror into subif+vtr OK
+ SPAN l2 rx mirror into vxlan test OK
GRE Tests
`````````
@@ -215,7 +261,8 @@ GRE Tests ==============================================================================
GRE Test Case
==============================================================================
- GRE tunnel Tests OK
+ GRE IPv4 tunnel Tests OK
+ GRE IPv6 tunnel Tests OK
GRE tunnel L2 Tests OK
GRE tunnel VRF Tests OK
@@ -250,40 +297,79 @@ LISP tests ==============================================================================
Test case for basic encapsulation OK
-NAT Test Cases
-``````````````
+NAT44 Test Cases
+````````````````
::
==============================================================================
- SNAT Test Cases
+ NAT44 Test Cases
==============================================================================
- SNAT dynamic translation test OK
- SNAT handling of client packets with TTL=1 OK
- SNAT handling of error responses to client packets with TTL=2 OK
- SNAT handling of server packets with TTL=1 OK
- SNAT handling of error responses to server packets with TTL=2 OK
- SNAT hairpinning OK
- SNAT multiple inside interfaces with overlapping address space OK
- Acquire SNAT addresses from interface OK
+ NAT44 dynamic translation test OK
+ NAT44 handling of client packets with TTL=1 OK
+ NAT44 handling of error responses to client packets with TTL=2 OK
+ NAT44 handling of server packets with TTL=1 OK
+ NAT44 handling of error responses to server packets with TTL=2 OK
+ NAT44 interfaces without configured IP address OK
+ NAT44 hairpinning - 1:1 NAPT OK
+ NAT44 hairpinning - 1:1 NAT OK
+ 1:1 NAT translate packet with unknown protocol - hairpinning OK
+ NAT44 translate packet with unknown protocol - hairpinning OK
+ NAT44 multiple inside interfaces with overlapping address space OK
+ Acquire NAT44 addresses from interface OK
Static mapping with addresses from interface OK
- S-NAT IPFIX logging NAT addresses exhausted OK
- S-NAT IPFIX logging NAT44 session created/delted OK
+ IPFIX logging NAT addresses exhausted OK
+ IPFIX logging NAT44 session created/delted OK
MAX translations per user - recycle the least recently used OK
- SNAT multiple inside interfaces (non-overlapping address space) OK
+ NAT44 multiple non-overlapping address space inside interfaces OK
+ NAT44 interface output feature (in2out postrouting) OK
+ NAT44 interface output feature hairpinning (in2out postrouting) OK
+ NAT44 interface output feature VRF aware (in2out postrouting) OK
Ping internal host from outside network OK
- Ping SNAT out interface from outside network OK
- S-NAT add pool addresses to FIB OK
- SNAT 1:1 NAT initialized from inside network OK
- SNAT 1:1 NAT initialized from outside network OK
- SNAT 1:1 NAT VRF awareness OK
- SNAT 1:1 NAT with port initialized from inside network OK
- SNAT 1:1 NAT with port initialized from outside network OK
- S-NAT tenant VRF independent address pool mode OK
- S-NAT tenant VRF aware address pool mode OK
+ Ping NAT44 out interface from outside network OK
+ NAT44 add pool addresses to FIB OK
+ 1:1 NAT initialized from inside network OK
+ NAT44 interfaces without configured IP address - 1:1 NAT OK
+ NAT44 local service load balancing OK
+ 1:1 NAT initialized from outside network OK
+ 1:1 NAT translate packet with unknown protocol OK
+ 1:1 NAT VRF awareness OK
+ 1:1 NAPT initialized from inside network OK
+ NAT44 interfaces without configured IP address - 1:1 NAPT OK
+ 1:1 NAPT initialized from outside network OK
+ NAT44 translate packet with unknown protocol OK
+ NAT44 tenant VRF independent address pool mode OK
+ NAT44 tenant VRF aware address pool mode OK
==============================================================================
Deterministic NAT Test Cases
==============================================================================
- S-NAT run deterministic mode OK
+ Deterministic NAT translation test (TCP, UDP, ICMP) OK
+ NAT plugin run deterministic mode OK
+ Deterministic NAT multiple users OK
+ Deterministic NAT maximum sessions per user limit SKIP
+ Deterministic NAT session timeouts SKIP
+ Set deterministic NAT timeouts OK
+ Deterministic NAT TCP session close from inside network OK
+ Deterministic NAT TCP session close from outside network OK
+
+NAT64 Test Cases
+````````````````
+::
+
+ ==============================================================================
+ NAT64 Test Cases
+ ==============================================================================
+ NAT64 dynamic translation test OK
+ NAT64 hairpinning OK
+ NAT64 translate packet with unknown protocol - hairpinning OK
+ NAT64 ICMP Error message translation OK
+ Enable/disable NAT64 feature on the interface OK
+ Add/delete address to NAT64 pool OK
+ NAT64 Network-Specific Prefix OK
+ NAT64 session timeout SKIP
+ Set NAT64 timeouts OK
+ NAT64 static translation test OK
+ Add/delete static BIB entry OK
+ NAT64 translate packet with unknown protocol OK
L2XC Multi-instance
```````````````````
@@ -313,6 +399,18 @@ IPv4 Tests IPv4 disabled
==============================================================================
IP Disabled OK
+ ==============================================================================
+ IPv4 Subnets
+ ==============================================================================
+ IP Sub Nets OK
+ ==============================================================================
+ IPv4 VLAN-0
+ ==============================================================================
+ IP VLAN-0 OK
+ ==============================================================================
+ IPv4 Load-Balancing
+ ==============================================================================
+ IP Load-Balancing OK
IPv4 FIB CRUD
`````````````
@@ -333,8 +431,39 @@ IRB Tests ==============================================================================
IRB Test Case
==============================================================================
- IPv4 IRB test 1 OK
- IPv4 IRB test 2 OK
+ ACL plugin prepare OK
+ ACL IPv6 routed -> bridged, L2 ACL deny OK
+ ACL IPv6 routed -> bridged, L3 ACL deny OK
+ ACL IPv4 routed -> bridged, L2 ACL deny OK
+ ACL IPv4 routed -> bridged, L3 ACL deny OK
+ ACL IPv6 bridged -> routed, L2 ACL deny OK
+ ACL IPv6 bridged -> routed, L3 ACL deny OK
+ ACL IPv4 bridged -> routed, L2 ACL deny OK
+ ACL IPv4 bridged -> routed, L3 ACL deny OK
+ ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK
+ ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK
+ ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK
+ ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK
+ ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK
+ ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK
+ ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK
+ ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK
+ ACL IPv6+EH routed -> bridged, L2 ACL deny OK
+ ACL IPv6+EH routed -> bridged, L3 ACL deny OK
+ ACL IPv6+EH bridged -> routed, L2 ACL deny OK
+ ACL IPv6+EH bridged -> routed, L3 ACL deny OK
+ ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK
+ ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK
+ ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK
+ ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK
+ ACL IPv4+MF routed -> bridged, L2 ACL deny OK
+ ACL IPv4+MF routed -> bridged, L3 ACL deny OK
+ ACL IPv4+MF bridged -> routed, L2 ACL deny OK
+ ACL IPv4+MF bridged -> routed, L3 ACL deny OK
+ ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK
+ ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK
+ ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK
+ ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK
ACL Security Groups
```````````````````
@@ -344,7 +473,7 @@ ACL Security Groups ACL plugin Test Case
==============================================================================
ACL plugin version check; learn MACs OK
- ACL create test OK
+ ACL create/delete test OK
permit ACL apply test OK
deny ACL apply test OK
VPP_624 permit ICMPv4 OK
@@ -365,22 +494,75 @@ ACL Security Groups deny single TCPv4/v6 OK
deny single UDPv4/v6 OK
deny single UDPv4/v6, permit ip any, verify non-initial fragment blocked OK
+ VPP-687 zero length udp ipv4 packet OK
+ VPP-687 zero length udp ipv6 packet OK
+ permit TCPv4 + non-match range OK
+ permit TCPv6 + non-match range OK
+ permit UDPv4 + non-match range OK
+ permit UDPv6 + non-match range OK
+ deny TCPv4/v6 + non-match range OK
+ deny UDPv4/v6 + non-match range OK
+ ==============================================================================
+ ACL plugin connection-oriented extended testcases
+ ==============================================================================
+ Prepare the settings SKIP
+ IPv4: Basic conn timeout test reflect on ingress SKIP
+ IPv4: Basic conn timeout test reflect on egress SKIP
+ IPv4: reflect egress, clear conn SKIP
+ IPv4: reflect ingress, clear conn SKIP
+ IPv4: Idle conn behind active conn, reflect on ingress SKIP
+ IPv4: Idle conn behind active conn, reflect on egress SKIP
+ IPv6: Basic conn timeout test reflect on ingress SKIP
+ IPv6: Basic conn timeout test reflect on egress SKIP
+ IPv6: reflect egress, clear conn SKIP
+ IPv6: reflect ingress, clear conn SKIP
+ IPv6: Idle conn behind active conn, reflect on ingress SKIP
+ IPv6: Idle conn behind active conn, reflect on egress SKIP
+ Prepare for TCP session tests SKIP
+ IPv4: transient TCP session (incomplete 3WHS), ref. on ingress SKIP
+ IPv4: transient TCP session (incomplete 3WHS), ref. on egress SKIP
+ IPv4: established TCP session (complete 3WHS), ref. on ingress SKIP
+ IPv4: established TCP session (complete 3WHS), ref. on egress SKIP
+ IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP
+ IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP
+ IPv6: transient TCP session (incomplete 3WHS), ref. on ingress SKIP
+ IPv6: transient TCP session (incomplete 3WHS), ref. on egress SKIP
+ IPv6: established TCP session (complete 3WHS), ref. on ingress SKIP
+ IPv6: established TCP session (complete 3WHS), ref. on egress SKIP
+ IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP
+ IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP
MPLS Tests
``````````
::
==============================================================================
+ MPLS-L2
+ ==============================================================================
+ Virtual Private LAN Service OK
+ Virtual Private Wire Service OK
+ ==============================================================================
MPLS Test Case
==============================================================================
MPLS Local Label Binding test OK
MPLS Deagg OK
MPLS label imposition test OK
+ MPLS Interface Receive OK
+ MPLS Multicast Head-end OK
+ MPLS IPv4 Multicast Tail OK
+ MPLS IPv6 Multicast Tail OK
+ MPLS Multicast Mid Point OK
MPLS label swap tests OK
MPLS Tunnel Tests OK
MPLS V4 Explicit NULL test OK
MPLS V6 Explicit NULL test OK
==============================================================================
+ MPLS PIC edge convergence
+ ==============================================================================
+ MPLS eBGP PIC edge convergence OK
+ MPLS iBGP PIC edge convergence OK
+ MPLSv6 eBGP PIC edge convergence OK
+ ==============================================================================
MPLS disabled
==============================================================================
MPLS Disabled OK
@@ -424,14 +606,16 @@ IP Multicast Tests IP Multicast Test Case
==============================================================================
IPv6 Multicast Replication OK
+ IPv6 Multicast Replication in non-default table OK
IP Multicast Replication OK
IP Multicast Connected Source check OK
IP Multicast Signal OK
+ IP Multicast Replication in non-default table OK
Classifier
``````````
::
-
+#######
==============================================================================
Classifier Test Case
==============================================================================
@@ -439,56 +623,6 @@ Classifier MAC ACL test OK
IP PBR test OK
-IRB Tests
-`````````
-::
-
- ==============================================================================
- IRB Test Case
- ==============================================================================
- ACL plugin prepare OK
- ACL IPv6 routed -> bridged, L2 ACL deny OK
- ACL IPv6 routed -> bridged, L3 ACL deny OK
- ACL IPv4 routed -> bridged, L2 ACL deny OK
- ACL IPv4 routed -> bridged, L3 ACL deny OK
- ACL IPv6 bridged -> routed, L2 ACL deny OK
- ACL IPv6 bridged -> routed, L3 ACL deny OK
- ACL IPv4 bridged -> routed, L2 ACL deny OK
- ACL IPv4 bridged -> routed, L3 ACL deny OK
- ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK
- ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK
- ACL IPv6+EH routed -> bridged, L2 ACL deny OK
- ACL IPv6+EH routed -> bridged, L3 ACL deny OK
- ACL IPv6+EH bridged -> routed, L2 ACL deny OK
- ACL IPv6+EH bridged -> routed, L3 ACL deny OK
- ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK
- ACL IPv4+MF routed -> bridged, L2 ACL deny OK
- ACL IPv4+MF routed -> bridged, L3 ACL deny OK
- ACL IPv4+MF bridged -> routed, L2 ACL deny OK
- ACL IPv4+MF bridged -> routed, L3 ACL deny OK
- ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK
- ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK
- ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK
- ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK
- ACL plugin set old L2 datapath OK
- ACL IPv6 routed -> bridged, L2 ACL deny OK
- ACL IPv6 routed -> bridged, L3 ACL deny OK
- ACL IPv4 routed -> bridged, L2 ACL deny OK
- ACL IPv4 routed -> bridged, L3 ACL deny OK
- ACL IPv6 bridged -> routed, L2 ACL deny OK
- ACL IPv6 bridged -> routed, L3 ACL deny OK
- ACL IPv4 bridged -> routed, L2 ACL deny OK
- ACL IPv4 bridged -> routed, L3 ACL deny OK
-
VXLAN Tests
```````````
::
@@ -502,6 +636,19 @@ VXLAN Tests Multicast receive test OK
Unicast flood test OK
+VXLAN-GPE Tests
+```````````
+::
+
+ ==============================================================================
+ VXLAN-GPE Test Case
+ ==============================================================================
+ Decapsulation test SKIP
+ Encapsulation test SKIP
+ Multicast flood test SKIP
+ Multicast receive test SKIP
+ Unicast flood test SKIP
+
L2 FIB CRUD
```````````
::
@@ -513,6 +660,11 @@ L2 FIB CRUD L2 FIB test 2 - delete 12 MAC entries OK
L2 FIB test 3 - program new 100 MAC addresses OK
L2 FIB test 4 - delete 160 MAC entries OK
+ L2 FIB test 5 - Program 10 new MAC entries, learn 10 OK
+ L2 FIB test 6 - flush first interface OK
+ L2 FIB test 7 - flush bd_id OK
+ L2 FIB test 8 - flush all OK
+ L2 FIB test 9 - mac learning events OK
ARP Tests
`````````
@@ -534,3 +686,231 @@ L2BD Tests ==============================================================================
L2BD MAC learning dual-loop test OK
L2BD MAC learning single-loop test OK
+
+MACIP Access Control Tests
+``````````
+::
+
+ ==============================================================================
+ MACIP Test Case
+ ==============================================================================
+ IP4 MACIP wildcardMAC|exactIP ACL bridged traffic OK
+ MACIP 10 ACLs each with 100+ entries OK
+ MACIP 10 ACLs each with 100+ entries with IP4 traffic OK
+ MACIP 10 ACLs each with 100+ entries with IP6 traffic OK
+ MACIP ACL with 10 entries OK
+ MACIP ACL with 100 entries OK
+ MACIP ACL with 2 entries OK
+ MACIP ACL with 20 entries OK
+ MACIP ACL with 5 entries OK
+ MACIP ACL with 50 entries OK
+ MACIP 2 ACLs each with 100+ entries OK
+ IP4 MACIP exactMAC|exactIP ACL bridged traffic OK
+ IP4 MACIP exactMAC|subnetIP ACL bridged traffic OK
+ IP4 MACIP exactMAC|wildIP ACL bridged traffic OK
+ IP4 MACIP ouiMAC|exactIP ACL bridged traffic OK
+ IP4 MACIP ouiMAC|subnetIP ACL bridged traffic OK
+ IP4 MACIP ouiMAC|wildIP ACL bridged traffic OK
+ IP4 MACIP wildcardMAC|subnetIP ACL bridged traffic OK
+ IP4 MACIP wildcardMAC|wildIP ACL bridged traffic OK
+ IP6 MACIP exactMAC|exactIP ACL bridged traffic OK
+ IP6 MACIP exactMAC|subnetIP ACL bridged traffic OK
+ IP6 MACIP exactMAC|wildIP ACL bridged traffic OK
+ IP6 MACIP oui_MAC|exactIP ACL bridged traffic OK
+ IP6 MACIP ouiMAC|subnetIP ACL bridged traffic OK
+ IP6 MACIP ouiMAC|wildIP ACL bridged traffic OK
+ IP6 MACIP wildcardMAC|exactIP ACL bridged traffic OK
+ IP6 MACIP wildcardMAC|subnetIP ACL bridged traffic OK
+ IP6 MACIP wildcardMAC|wildIP ACL bridged traffic OK
+ MACIP replace ACL OK
+ MACIP replace ACL with IP4 traffic OK
+ MACIP replace ACL with IP6 traffic OK
+ IP4 MACIP exactMAC|exactIP ACL routed traffic OK
+ IP4 MACIP exactMAC|subnetIP ACL routed traffic OK
+ IP4 MACIP exactMAC|wildIP ACL routed traffic OK
+ IP4 MACIP ouiMAC|exactIP ACL routed traffic OK
+ IP4 MACIP ouiMAC|subnetIP ACL routed traffic OK
+ IP4 MACIP ouiMAC|wildIP ACL routed traffic OK
+ IP4 MACIP wildcardMAC|exactIP ACL routed traffic OK
+ IP4 MACIP wildcardMAC|subnetIP ACL routed traffic OK
+ IP4 MACIP wildcardMAC|wildIP ACL OK
+ IP6 MACIP exactMAC|exactIP ACL routed traffic OK
+ IP6 MACIP exactMAC|subnetIP ACL routed traffic OK
+ IP6 MACIP exactMAC|wildIP ACL routed traffic OK
+ IP6 MACIP ouiMAC|exactIP ACL routed traffic OK
+ IP6 MACIP ouiMAC|subnetIP ACL routed traffic OK
+ IP6 MACIP ouiMAC|wildIP ACL routed traffic OK
+ IP6 MACIP wildcardMAC|exactIP ACL routed traffic OK
+ IP6 MACIP wildcardMAC|subnetIP ACL routed traffic OK
+ IP6 MACIP wildcardMAC|wildIP ACL OK
+ MACIP ACL delete intf with acl OK
+
+SRv6 Tests
+``````````
+::
+
+ ==============================================================================
+ SRv6 Test Case
+ ==============================================================================
+ Test SRv6 End (without PSP) behavior. OK
+ Test SRv6 End.DT4 behavior. OK
+ Test SRv6 End.DT6 behavior. OK
+ Test SRv6 End.DX2 behavior. OK
+ Test SRv6 End.DX4 behavior. OK
+ Test SRv6 End.DX6 behavior. OK
+ Test SRv6 End.X (without PSP) behavior. OK
+ Test SRv6 End.X with PSP behavior. OK
+ Test SRv6 End with PSP behavior. OK
+ Test SRv6 Transit.Encaps behavior for IPv6. OK
+ Test SRv6 Transit.Encaps behavior for IPv4. OK
+ Test SRv6 Transit.Encaps behavior for L2. SKIP
+ Test SRv6 Transit.Insert behavior (IPv6 only). OK
+
+PPPoE Tests
+```````````
+::
+
+ ==============================================================================
+ PPPoE Test Case
+ ==============================================================================
+ PPPoE Add Same Session Twice Test OK
+ PPPoE Decap Test OK
+ PPPoE Decap Multiple Sessions Test OK
+ PPPoE Delete Same Session Twice Test OK
+ PPPoE Encap Test OK
+ PPPoE Encap Multiple Sessions Test OK
+
+VTR VLAN Tag Rewrite Tests
+``````````````````````````
+::
+
+ ==============================================================================
+ VTR Test Case
+ ==============================================================================
+ 1AD VTR pop 1 test OK
+ 1AD VTR pop 2 test OK
+ 1AD VTR push 1 1AD test OK
+ 1AD VTR push 1 1Q test OK
+ 1AD VTR push 2 1AD test OK
+ 1AD VTR push 2 1Q test OK
+ 1AD VTR translate 1 -> 1 1AD test OK
+ 1AD VTR translate 1 -> 1 1Q test OK
+ 1AD VTR translate 1 -> 2 1AD test OK
+ 1AD VTR translate 1 -> 2 1Q test OK
+ 1AD VTR translate 2 -> 1 1AD test OK
+ 1AD VTR translate 2 -> 1 1Q test OK
+ 1AD VTR translate 2 -> 2 1AD test OK
+ 1AD VTR translate 2 -> 2 1Q test OK
+ 1Q VTR pop 1 test OK
+ 1Q VTR push 1 test OK
+ 1Q VTR push 2 test OK
+ 1Q VTR translate 1 -> 1 test OK
+ 1Q VTR translate 1 -> 2 test OK
+
+ARP Tests
+`````````
+::
+
+ ==============================================================================
+ ARP Test Case
+ ==============================================================================
+ ARP OK
+ ARP Duplicates OK
+ ARP Static OK
+ ARP reply with VRRP virtual src hw addr OK
+ MPLS OK
+ Proxy ARP OK
+ Interface Mirror Proxy ARP OK
+ ==============================================================================
+ L2BD arp termination Test Case
+ ==============================================================================
+ L2BD arp term - add 5 hosts, verify arp responses OK
+ L2BD arp term - delete 3 hosts, verify arp responses OK
+ L2BD arp term - recreate BD1, readd 3 hosts, verify arp responses OK
+ L2BD arp term - 2 IP4 addrs per host OK
+ L2BD arp term - create and update 10 IP4-mac pairs OK
+ L2BD arp/ND term - hosts with both ip4/ip6 OK
+ L2BD ND term - Add and Del hosts, verify ND replies OK
+ L2BD ND term - Add and update IP+mac, verify ND replies OK
+ L2BD arp term - send garps, verify arp event reports OK
+ L2BD arp term - send duplicate garps, verify suppression OK
+ L2BD arp term - disable ip4 arp events,send garps, verify no events OK
+ L2BD ND term - send NS packets verify reports OK
+ L2BD ND term - send duplicate ns, verify suppression OK
+ L2BD ND term - disable ip4 arp events,send ns, verify no events OK
+
+P2P Ethernet Subinterface ests
+``````````````````````````````
+::
+
+ ==============================================================================
+ P2P Ethernet tests
+ ==============================================================================
+ delete/create p2p subif OK
+ create 100k of p2p subifs SKIP
+ create 1k of p2p subifs OK
+ ==============================================================================
+ P2P Ethernet IPv4 tests
+ ==============================================================================
+ receive ipv4 packet via p2p subinterface OK
+ route rx packet not matching p2p subinterface OK
+ send ip4 packet via p2p subinterface OK
+ drop tx ip4 packet not matching p2p subinterface OK
+ ==============================================================================
+ P2P Ethernet IPv6 tests
+ ==============================================================================
+ receive ipv6 packet via p2p subinterface OK
+ drop rx packet not matching p2p subinterface OK
+ route rx ip6 packet not matching p2p subinterface OK
+ send packet via p2p subinterface OK
+ drop tx ip6 packet not matching p2p subinterface OK
+ standard routing without p2p subinterfaces OK
+
+GTPU Test Case
+``````````````
+::
+
+ ==============================================================================
+ GTPU Test Case
+ ==============================================================================
+ Decapsulation test OK
+ Encapsulation test OK
+ Multicast flood test OK
+ Multicast receive test OK
+ Unicast flood test OK
+
+Other Tests
+```````````
+::
+
+ ==============================================================================
+ VAPI test
+ ==============================================================================
+ run C VAPI tests SKIP
+ run C++ VAPI tests SKIP
+ ==============================================================================
+ Ping Test Case
+ ==============================================================================
+ basic ping test OK
+ burst ping test OK
+ ==============================================================================
+ Template verification, timer tests
+ ==============================================================================
+ timer less than template timeout OK
+ timer greater than template timeout OK
+ verify cflow packet fields OK
+ ==============================================================================
+ JVPP Core Test Case
+ ==============================================================================
+ JVPP Acl Callback Api Test Case OK
+ JVPP Acl Future Api Test Case OK
+ JVPP Core Callback Api Test Case OK
+ JVPP Core Future Api Test Case OK
+ JVPP Ioamexport Callback Api Test Case OK
+ JVPP Ioamexport Future Api Test Case OK
+ JVPP Ioampot Callback Api Test Case OK
+ JVPP Ioampot Future Api Test Case OK
+ JVPP Ioamtrace Callback Api Test Case OK
+ JVPP Ioamtrace Future Api Test Case OK
+ JVPP Snat Callback Api Test Case OK
+ JVPP Snat Future Api Test Case OK
\ No newline at end of file diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst index 0ddeb6a569..461e186f6d 100644 --- a/docs/report/introduction/general_notes.rst +++ b/docs/report/introduction/general_notes.rst @@ -15,32 +15,33 @@ FD.io CSIT project currently covers multiple FD.io system and sub-system testing areas and this is reflected in this report, where each testing area is listed separately, as follows: -#. **VPP Performance Tests** - VPP performance tests are executed in physical +#. **VPP - Performance** - VPP benchmarking tests are executed in physical FD.io testbeds, focusing on VPP network data plane performance at this stage, both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC) forwarding topologies. Tested across a range of NICs, 10GE and 40GE interfaces, range of multi-thread and multi-core configurations. VPP application runs in host user-mode. TRex is used as a traffic generator. -#. **Container memif connections** - VPP memif virtual interface (shared memory - interface) tests to interconnect VPP instances. VPP vswitch instance runs in - bare-metal user-mode handling Intel x520 NIC 10GbE interfaces and connecting - over memif (Master side) virtual interfaces to more instances of VPP running - in LXC or in Docker Containers, both with memif virtual interfaces (Slave - side). Tested across a range of NICs, 10GE and 40GE interfaces, range of - multi-thread and multi-core configurations. VPP application runs in host - user-mode. TRex is used as a traffic generator. +#. **LXC and Docker Containers VPP memif - Performance** - VPP memif + virtual interface tests interconnect multiple VPP instances running in + containers. VPP vswitch instance runs in bare-metal user-mode + handling Intel x520 NIC 10GbE interfaces and connecting over memif + (Master side) virtual interfaces to more instances of VPP running in + LXC or in Docker Containers, both with memif virtual interfaces (Slave + side). Tested across a range of multi-thread and multi-core + configurations. TRex is used as a traffic generator. -#. **Container Orchestrated Performance Tests** - CSIT |release| introduced new - tests of Container topologies connected over the memif virtual interface - (shared memory interface). For these tests VPP vswitch instance runs in a - Docker Container handling Intel x520 NIC 10GbE interfaces and connecting over - memif (Master side) virtual interfaces to more instances of VPP running in - Docker Containers with memif virtual interfaces (Slave side). Tested across a - range of multi-thread and multi-core configurations. VPP application runs in - host user-mode. TRex is used as a traffic generator. +#. **Container Topologies Orchestrated by K8s - Performance** - CSIT + |release| introduced new tests of Container topologies connected over + the memif virtual interface (shared memory interface). For these tests + VPP vswitch instance runs in a Docker Container handling Intel x520 NIC + 10GbE interfaces and connecting over memif (Master side) virtual + interfaces to more instances of VPP running in Docker Containers with + memif virtual interfaces (Slave side). All containers are orchestrated + by Kubernetes, with Ligato for container networking. TRex is used as a + traffic generator. -#. **DPDK Performance Tests** - VPP is using DPDK code to control and drive +#. **DPDK Performance** - VPP is using DPDK code to control and drive the NICs and physical interfaces. Testpmd tests are used as a baseline to profile the DPDK sub-system of VPP. DPDK performance tests executed in physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for @@ -49,23 +50,23 @@ is listed separately, as follows: Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic generator. -#. **VPP Functional Tests** - VPP functional tests are executed in virtual +#. **Honeycomb Performance** - Honeycomb performance tests are executed in + physical FD.io testbeds, focusing on the performance of Honeycomb management + and programming functionality of VPP. Tests cover a range of CRUD operations + executed against VPP. + +#. **VPP Functional** - VPP functional tests are executed in virtual FD.io testbeds focusing on VPP packet processing functionality, including network data plane and in -line control plane. Tests cover vNIC-to-vNIC vNIC-to-VM-to-vNIC forwarding topologies. Scapy is used as a traffic generator. -#. **Honeycomb Functional Tests** - Honeycomb functional tests are executed in +#. **Honeycomb Functional** - Honeycomb functional tests are executed in virtual FD.io testbeds, focusing on Honeycomb management and programming functionality of VPP. Tests cover a range of CRUD operations executed against VPP. -#. **Honeycomb Performance Tests** - Honeycomb performance tests are executed in - physical FD.io testbeds, focusing on the performance of Honeycomb management - and programming functionality of VPP. Tests cover a range of CRUD operations - executed against VPP. - -#. **NSH_SFC Functional Tests** - NSH_SFC functional tests are executed in +#. **NSH_SFC Functional** - NSH_SFC functional tests are executed in virtual FD.io testbeds focusing on NSH_SFC of VPP. Tests cover a range of CRUD operations executed against VPP. diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index 536d5d3cf1..22bc29f071 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -44,6 +44,13 @@ CSIT |release| report contains following main sections and sub-sections: executions; *Test Environment* - environment description; *Documentation* - CSIT source code documentation for DPDK performance tests. +#. **Honeycomb Performance Tests** - Honeycomb performance tests executed in + physical FD.io testbeds; *Overview* - tested topologies, test coverage + and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, + added tests, environment or methodology changes, known CSIT issues; + *Test Environment* - environment description; *Documentation* - source + code documentation for Honeycomb performance tests. + #. **VPP Functional Tests** - VPP functional tests executed in virtual FD.io testbeds; *Overview* - tested virtual topologies, test coverage and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, added diff --git a/docs/report/vpp_unit_tests/overview.rst b/docs/report/vpp_unit_tests/overview.rst index cc32ee4f3d..90c582c943 100644 --- a/docs/report/vpp_unit_tests/overview.rst +++ b/docs/report/vpp_unit_tests/overview.rst @@ -33,40 +33,42 @@ Unit Tests Coverage -------------------
Following VPP functional test areas are covered in VPP unit test code included
-in VPP rls1704 with results listed in this report:
+in VPP rls1710 with results listed in this report:
+- ARP - ARP, Proxy ARP.
+- ACL plugin - stateful and stateless security-groups access-control-lists.
+- BFD IPv4 - Bidirectional Forwarding Detection - baseline, APIs, authorization, authentication.
+- BFD IPv6 - Bidirectional Forwarding Detection - baseline, APIs, authorization, authentication.
+- Classifier - classification with IP ACL, MAC ACL, IP PBR.
- CRUD Loopback - create, read, update, delete Loopback interfaces.
-- Flow-per-packet plugin - collect and report L2 and IP4 flow statistics.
+- Deterministic NAT - Carrier Grade NAT tests.
- DHCP - DHCPv4 and DHCPv6 Proxy.
+- FIB - baseline and scale tests.
+- Flow-per-packet plugin - collect and report L2 and IP4 flow statistics.
+- Flowprobe tests.
+- GRE - GRE IPv4/IPv6 tunnel, L2, VRF tests.
+- GTPU - baseline GTPU tests.
+- IP Multicast - IPv4/IPv6 multicast replication, connected source check.
- IP4 VRF Multi-instance - create, read, update, delete and verify IPv4 VRFs.
-- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6.
-- FIB Unit Tests.
-- Bidirectional Forwarding Detection (BFD) IPv4 - baseline BFD session operation.
-- Bidirectional Forwarding Detection (BFD) IPv6 - baseline BFD session operation.
-- Bidirectional Forwarding Detection (BFD) - API tests.
-- Bidirectional Forwarding Detection (BFD) - changing authorization.
-- Bidirectional Forwarding Detection (BFD) - SHA1 authentication.
-- IPv6 Tests - baseline FIB operations, NS/RS exception handling.
-- SPAN Test - Switched Port Analyzer packet mirroring.
-- GRE Tests - GRE tunnel tests.
-- SNAT Test Cases - SNAT44 tests.
-- Deterministic NAT Test Cases - Carrier Grade NAT tests.
-- NAT64 Test Cases - NAT64 static and dynamic translation tests.
-- L2XC Multi-instance - L2 cross-connect multi-instance tests.
-- IPv4 Tests - baseline FIB tests.
+- IP6 VRF Multi-instance - create, read, update, delete and verify IPv6 VRFs.
+- IPv4 - baseline FIB tests.
- IPv4 FIB CRUD - add/update/delete IPv4 routes.
-- IRB Tests - Integrated Routing and Bridging tests.
-- ACL plugin - stateful and stateless security-groups access-control-lists.
-- MPLS Tests - MPLS baseline tests.
+- IPv6 - baseline FIB operations, NS/RS exception handling.
+- IRB - Integrated Routing and Bridging tests.
+- L2 FIB CRUD - add/update/delete L2 MAC entries.
+- L2BD - L2 Bridge-Domain baseline tests incl. single- and dual-loop.
+- L2XC - L2 cross-connect baseline tests incl. single- and dual-loop.
+- L2XC Multi-instance - L2 cross-connect multi-instance tests.
+- LISP - basic LISP tests.
+- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6.
+- MACIP - ingress access control for IPv4, IPv6 with L2BDP and IP routing.
+- MFIB Unit.
+- MPLS - MPLS baseline tests.
- MPLS PIC edge convergence - prefix independent convergence tests for MPLS PE.
-- L2XC Tests - L2 cross-connect baseline tests incl. single- and dual-loop.
-- MFIB Unit Tests.
-- IP Multicast Tests - IPv4/IPv6 multicast replication, connected source check.
-- Classifier - classification with IP ACL, MAC ACL, IP PBR.
-- VXLAN Tests - baseline VXLAN tunneling.
-- VXLAN-GPE Test Cases - baseline VXLAN-GPE tunneling tests including multicast.
+- NAT44 - NAT44 tests, IPFIX logging, VRF awareness.
+- NAT64 - NAT64 static and dynamic translation tests.
+- SPAN - Switched Port Analyzer packet mirroring.
+- SRv6 - Segment Routing IPv6 tests.
- VTR Test Case - VLAN tag manipulation tests.
-- L2 FIB CRUD - add/update/delete L2 MAC entries.
-- L2BD Tests - L2 Bridge-Domain baseline tests incl. single- and dual-loop.
-- Flowprobe tests.
-- GTPU Test Cases - baseline GTPU tests.
+- VXLAN - baseline VXLAN tunneling.
+- VXLAN-GPE - baseline VXLAN-GPE tunneling tests including multicast.
|