aboutsummaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
authorpmikus <pmikus@cisco.com>2016-11-16 16:45:05 +0100
committerPeter Mikus <pmikus@cisco.com>2017-01-10 11:42:01 +0000
commite3ed9257cd7ef50b44eca72ef7d9ae6c7c792647 (patch)
treeb5234556896a1860c172c2792a55c7d402daa36a /docs
parent2671e2616aab7a0bfec7a3501d53ccd72e12d668 (diff)
CSIT-469 Rename performance suites
Rename performance suites and add more TAGS Change-Id: I6a5251bb6e380fe2a47a6ec1c1aeb4a9f70c5ce3 Signed-off-by: pmikus <pmikus@cisco.com>
Diffstat (limited to 'docs')
-rw-r--r--docs/tag_documentation.rst352
1 files changed, 276 insertions, 76 deletions
diff --git a/docs/tag_documentation.rst b/docs/tag_documentation.rst
index ab5e2d1345..e0be374ee2 100644
--- a/docs/tag_documentation.rst
+++ b/docs/tag_documentation.rst
@@ -1,131 +1,331 @@
-# Copyright (c) 2016 Cisco and/or its affiliates.
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at:
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+.. |copy| unicode:: 0xA9 .. copyright sign
+
+| Copyright |copy| 2016 Cisco and/or its affiliates.
+| Licensed under the Apache License, Version 2.0 (the "License");
+| you may not use this file except in compliance with the License.
+| You may obtain a copy of the License at:
+|
+| http://www.apache.org/licenses/LICENSE-2.0
+|
+| Unless required by applicable law or agreed to in writing, software
+| distributed under the License is distributed on an "AS IS" BASIS,
+| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+| See the License for the specific language governing permissions and
+| limitations under the License.
+
+
+.. contents:: Table of Contents
+ :depth: 2
+
+
+TAGs and their descriptions
+===========================
Documentation for tags used to select and identify test cases.
-List of TAGs and their descriptions
-===================================
Topology TAGs
-------------
-3_NODE_DOUBLE_LINK_TOPO
+.. topic:: 3_NODE_DOUBLE_LINK_TOPO
+
3 nodes connected in a circular topology with two links interconnecting
the devices.
-3_NODE_SINGLE_LINK_TOPO
+.. topic:: 3_NODE_SINGLE_LINK_TOPO
+
3 nodes connected in a circular topology with at least one link
interconnecting devices.
Objective TAGs
--------------
-SKIP_PATCH
+.. topic:: SKIP_PATCH
+
Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch)
and csit-vpp-verify jobs (i.e. CSIT patch).
+
Environment TAGs
----------------
-HW_ENV
+.. topic:: HW_ENV
+
DUTs and TGs are running on bare metal.
-VM_ENV
+.. topic:: VM_ENV
+
DUTs and TGs are running in virtual environment.
-VPP_VM_ENV
+.. topic:: VPP_VM_ENV
+
DUTs with VPP and capable of running Virtual Machine.
-DUT Setup TAGs
---------------
-1_THREAD_NOHTT_RSS_1
- 1 worker thread pinned to dedicated core without use of Hyper-threading
- technology with 1 thread per interface. Main thread pinned to core 0.
+Scaling TAGs
+------------
-2_THREAD_NOHTT_RSS_1
- 2 worker threads each pinned to dedicated core without use of Hyper-threading
- technology with 1 thread per interface. Main thread pinned to core 0.
+.. topic:: FIB_20K
-4_THREAD_NOHTT_RSS_2
- 4 worker threads each pinned to dedicated core without use of Hyper-threading
- technology with 2 threads per interface. Main thread pinned to core 0.
+ 2x10,000 entries in single fib table
-6_THREAD_NOHTT_RSS_3
- 6 worker threads each pinned to dedicated core without use of Hyper-threading
- technology with 3 threads per interface. Main thread pinned to core 0.
+.. topic:: FIB_200K
-8_THREAD_HTT_RSS_4
- 8 worker threads each pinned to dedicated core without use of Hyper-threading
- technology with 4 threads per interface. Main thread pinned to core 0.
+ 2x100,000 entries in single fib table
-SINGLE_THREAD
- All single threaded test cases.
+.. topic:: FIB_2M
-MULTI_THREAD
- All test cases with more then one thread.
+ 2x1,000,000 entries in single fib table
-Performance testing TAGs
-------------------------
-PERFTEST
+Tags marking the kinds of tags
+------------------------------
+
+.. topic:: FUNCTEST
+
+ All functional test cases.
+
+.. topic:: PERFTEST
+
All performance test cases.
-PERFTEST_VHOST
- All performance test cases that include testing of VHOST with VM.
-PERFTEST_SHORT
- Performance of DUT should pass specific value. Each test case run is
- executed for 10 seconds.
+Performance testing tags
+------------------------
+
+.. topic:: PDRDISC
-PERFTEST_LONG
- Find performance of DUT based on RFC2544 with linear/binary/combined
- search. Each test case run is executed for 60 seconds.
+ Partial Drop Rate evaluation of single run result, with non-zero packet
+ loss tolerance (LT) expressed in percentage of packets transmitted.
-PDR
- Partial Drop Rate evaluation of single run result. Loss acceptance of
- dropped packets from number of sent packet is set as variable in frames or
- percentage.
+.. topic:: NDRDISC
-NDR
Non Drop Rate evaluation of results. Loss acceptance of dropped packets is
set to zero lost packets.
-Scaling TAGs
+.. topic:: NDRCHK
+
+ Performance tests where TG verifies DUTs' throughput at ref-NDR (reference
+ Non Drop Rate) with zero packet loss tolerance.
+
+.. topic:: NDRPDRDISC
+
+ Find performance of DUT based on RFC2544 with linear / binary / combined
+ search. (Previous LONG tests.)
+
+
+Test type tags
+--------------
+
+.. topic:: BASE
+
+ Baseline test cases, no encapsulation, no feature(s) configured in tests.
+
+.. topic:: SCALE
+
+ Scale test cases.
+
+.. topic:: ENCAP
+
+ Test cases where encapsulation is used. Use also encapsulation tag(s).
+
+.. topic:: FEATURE
+
+ At least one feature is configured in test cases. Use also feature tag(s).
+
+
+Forwarding mode tags
+--------------------
+
+.. topic:: L2BDMACSTAT
+
+ VPP L2 bridge-domain, L2 MAC static.
+
+.. topic:: L2BDMACLRN
+
+ VPP L2 bridge-domain, L2 MAC learning.
+
+.. topic:: L2XCFWD
+
+ VPP L2 point-to-point cross-connect.
+
+.. topic:: IP4FWD
+
+ VPP IPv4 routed forwarding.
+
+.. topic:: IP6FWD
+
+ VPP IPv6 routed forwarding.
+
+
+Underlay tags
+----------
+
+.. topic:: IP4UNRLAY
+
+ IPv4 underlay.
+
+.. topic:: IP6UNRLAY
+
+ IPv6 underlay.
+
+.. topic:: MPLSUNRLAY
+
+ MPLS underlay.
+
+
+Overlay tags
+----------
+
+.. topic:: L2OVRLAY
+
+ L2 overlay.
+
+.. topic:: IP4OVRLAY
+
+ IPv4 overlay (IPv4 payload).
+
+.. topic:: IP6OVRLAY
+
+ IPv6 overlay (IPv6 payload).
+
+
+Encapsulation tags
+------------------
+
+.. topic:: ETH
+
+ All test cases with base Ethernet (no encapsulation).
+
+.. topic:: DOT1Q
+
+ All test cases with dot1q.
+
+.. topic:: DOT1AD
+
+ All test cases with dot1ad.
+
+.. topic:: LISP
+
+ All test cases with LISP.
+
+.. topic:: LISPGPE
+
+ All test cases with LISP-GPE.
+
+.. topic:: VXLAN
+
+ All test cases with Vxlan.
+
+.. topic:: VXLANGPE
+
+ All test cases with VXLAN-GPE.
+
+.. topic:: GRE
+
+ All test cases with GRE.
+
+
+Interface tags
+--------------
+
+.. topic:: PHY
+
+ All test cases which use physical interface(s).
+
+.. topic:: VHOST
+
+ All test cases which uses VHOST.
+
+.. topic:: TUNTAP
+
+ All test cases which uses TUN and TAP.
+
+.. topic:: AFPKT
+
+ All test cases which uses AFPKT.
+
+.. topic:: NETMAP
+
+ All test cases which uses Netmap.
+
+
+Feature tags
------------
-FIB_20K
- 2x10,000 entries in single fib table
+.. topic:: IACLDST
-FIB_200K
- 2x100,000 entries in single fib table
+ iACL destination.
-FIB_2M
- 2x1,000,000 entries in single fib table
+.. topic:: COPWHLIST
+
+ COP whitelist.
+
+
+Client-workload tags
+--------------------
+
+.. topic:: VM
+
+ All test cases which use at least one virtual machine.
+
+.. topic:: LXC
+
+ All test cases which use Linux container.
+
+.. topic:: APP
+
+ All test cases with specific APP use.
+
+
+Multi-threading tags
+--------------------
+
+.. topic:: STHREAD
+
+ All test cases using single poll mode thread.
+
+.. topic:: MTHREAD
+
+ All test cases using more then one poll mode driver thread.
+
+.. topic:: 1NUMA
+
+ All test cases with packet processing on single socket.
+
+.. topic:: 2NUMA
+
+ All test cases with packet processing on two sockets.
+
+.. topic:: SMT
+
+ All test cases with symmetric Multi-Threading (HyperThreading) enabled.
+
+.. topic:: NOSMT
+
+ All test cases with symmetric Multi-Threading (HyperThreading) disabled.
+
+.. topic:: 1T1C
+
+ 1 worker thread pinned to 1 dedicated physical core. 1 receive queue per
+ interface. Main thread pinned to core 0.
+
+.. topic:: 2T2C
+
+ 2 worker threads pinned to 2 dedicated physical cores. 1 receive queue per
+ interface. Main thread pinned to core 0.
+
+.. topic:: 4T4C
-LISP TAGs
----------
-LISP
- All Lisp test cases.
+ 4 worker threads pinned to 4 dedicated physical cores. 2 receive queues per
+ interface. Main thread pinned to core 0.
-LISP_IPv4
- Test Lisp in IPv4 topology.
+.. topic:: 6T6C
-LISP_IPv6
- Test Lisp in IPv6 topology.
+ 6 worker threads pinned to 6 dedicated physical cores. 3 receive queues per
+ interface. Main thread pinned to core 0.
-LISP_IPv4oIPv6
- Test IPv4 over IPv6 topology.
+.. topic:: 8T8C
-LISP_IPv6oIPv4
- Test IPv6 over IPv4 topology.
+ 8 worker threads pinned to 8 dedicated physical cores. 4 receive queues per
+ interface. Main thread pinned to core 0.