Methodology: Added packet flow ordering

Change-Id: If0f0397c2ace9112490e017ec8d18cfbf1dffbc5
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>

3 files changed, 92 insertions, 38 deletions

diff --git a/docs/report/introduction/methodology.rst b/docs/report/introduction/methodology.rst
index 6044063f36..01059478ed 100644
--- a/docs/report/introduction/methodology.rst
+++ b/docs/report/introduction/methodology.rst
@@ -12,10 +12,11 @@ Test Methodology
     methodology_trex_traffic_generator
     methodology_dut_state
     methodology_nat44
+    methodology_packet_latency
+    methodology_packet_flow_ordering
     methodology_tunnel_encapsulations
     methodology_ipsec
     methodology_acls
-    methodology_packet_latency
     methodology_multi_core_speedup
     methodology_hoststack_testing/index
     methodology_reconf
diff --git a/docs/report/introduction/methodology_packet_flow_ordering.rst b/docs/report/introduction/methodology_packet_flow_ordering.rst
new file mode 100644
index 0000000000..3796b21796
--- /dev/null
+++ b/docs/report/introduction/methodology_packet_flow_ordering.rst
@@ -0,0 +1,42 @@
+.. _packet_flow_ordering:
+
+Packet Flow Ordering
+^^^^^^^^^^^^^^^^^^^^
+
+TRex Traffic Generator (TG) supports two main ways how to cover
+address space (on allowed ranges) in scale tests.
+
+In most cases only one field value (e.g. IPv4 destination address) is
+altered, in some cases two fields (e.g. IPv4 destination address and UDP
+destination port) are altered.
+
+Incremental Ordering
+--------------------
+
+This case is simpler to implement and offers greater control.
+
+When changing two fields, they can be incremented synchronously, or one
+after another. In the latter case we can specify which one is
+incremented each iteration and which is incremented by "carrying over"
+only when the other "wraps around". This way also visits all
+combinations once before the "carry" field also wraps around.
+
+It is possible to use increments other than 1.
+
+Randomized Ordering
+-------------------
+
+This case chooses each field value at random (from the allowed range).
+In case of two fields, they are treated independently.
+TRex allows to set random seed to get deterministic numbers.
+We use a different seed for each field and traffic direction.
+The seed has to be a non-zero number, we use 1, 2, 3, and so on.
+
+The seeded random mode in TRex requires a "limit" value,
+which acts as a cycle length limit (after this many iterations,
+the seed resets to its initial value).
+We use the maximal allowed limit value (computed as 2^24 - 1).
+
+Randomized profiles do not avoid duplicated values,
+and do not guarantee each possible value is visited,
+so it is not very useful for stateful tests.
diff --git a/docs/report/introduction/methodology_vpp_forwarding_modes.rst b/docs/report/introduction/methodology_vpp_forwarding_modes.rst
index 1af3a46556..1fd902a22d 100644
--- a/docs/report/introduction/methodology_vpp_forwarding_modes.rst
+++ b/docs/report/introduction/methodology_vpp_forwarding_modes.rst
@@ -1,9 +1,9 @@
 VPP Forwarding Modes
 --------------------
 
-VPP is tested in a number of L2 and IP packet lookup and forwarding
-modes. Within each mode baseline and scale tests are executed, the
-latter with varying number of lookup entries.
+VPP is tested in a number of L2, IPv4 and IPv6 packet lookup and
+forwarding modes. Within each mode baseline and scale tests are
+executed, the latter with varying number of FIB entries.
 
 L2 Ethernet Switching
 ~~~~~~~~~~~~~~~~~~~~~
@@ -21,17 +21,23 @@ VPP is tested in three L2 forwarding modes:
 
 l2bd tests are executed in baseline and scale configurations:
 
-- *l2bdbase*: low number of L2 flows (254 per direction) is switched by
-  VPP. They drive the content of MAC FIB size (508 total MAC entries).
-  Both source and destination MAC addresses are incremented on a packet
-  by packet basis.
-
-- *l2bdscale*: high number of L2 flows is switched by VPP. Tested MAC
-  FIB sizes include: i) 10k (5k unique flows per direction), ii) 100k
-  (2x 50k flows) and iii) 1M (2x 500k). Both source and destination MAC
-  addresses are incremented on a packet by packet basis, ensuring new
-  entries are learn refreshed and looked up at every packet, making it
-  the worst case scenario.
+- *l2bdbase*: Two MAC FIB entries are learned by VPP to enable packet
+  switching between two interfaces in two directions. VPP L2 switching
+  is tested with 254 IPv4 unique flows per direction, varying IPv4
+  source address per flow in order to invoke RSS based packet
+  distribution across VPP workers. The same source and destination MAC
+  address is used for all flows per direction. IPv4 source address is
+  incremented for every packet.
+
+- *l2bdscale*: A high number of MAC FIB entries are learned by VPP to
+  enable packet switching between two interfaces in two directions.
+  Tested MAC FIB sizes include: i) 10k with 5k unique flows per
+  direction, ii) 100k with 2 x 50k flows and iii) 1M with 2 x 500k
+  flows. Unique flows are created by using distinct source and
+  destination MAC addresses that are changed for every packet using
+  incremental ordering, making VPP learn (or refresh) distinct src MAC
+  entries and look up distinct dst MAC entries for every packet. For
+  details, see :ref:`packet_flow_ordering`.
 
 Ethernet wire encapsulations tested include: untagged, dot1q, dot1ad.
 
@@ -40,34 +46,39 @@ IPv4 Routing
 
 IPv4 routing tests are executed in baseline and scale configurations:
 
-- *ip4base*: low number of IPv4 flows (253 or 254 per direction) is routed by
-  VPP. They drive the content of IPv4 FIB size (506 or 508 total /32 prefixes).
-  Destination IPv4 addresses are incremented on a packet by packet
-  basis.
+- *ip4base*: Two /32 IPv4 FIB entries are configured in VPP to enable
+  packet routing between two interfaces in two directions. VPP routing
+  is tested with 253 IPv4 unique flows per direction, varying IPv4
+  source address per flow in order to invoke RSS based packet
+  distribution across VPP workers. IPv4 source address is incremented
+  for every packet.
 
-- *ip4scale*: high number of IPv4 flows is routed by VPP. Tested IPv4
-  FIB sizes of /32 prefixes include: i) 20k (10k unique flows per
-  direction), ii) 200k (2x 100k flows) and iii) 2M (2x 1M). Destination
-  IPv4 addresses are incremented on a packet by packet basis, ensuring
-  new FIB entries are looked up at every packet, making it the worst
-  case scenario.
+- *ip4scale*: A high number of /32 IPv4 FIB entries are configured in
+  VPP. Tested IPv4 FIB sizes include: i) 20k with 10k unique flows per
+  direction, ii) 200k with 2 * 100k flows and iii) 2M with 2 * 1M
+  flows. Unique flows are created by using distinct IPv4 destination
+  addresses that are changed for every packet, using incremental or
+  random ordering. For details, see :ref:`packet_flow_ordering`.
 
 IPv6 Routing
 ~~~~~~~~~~~~
 
-IPv6 routing tests are executed in baseline and scale configurations:
-
-- *ip6base*: low number of IPv6 flows (253 or 254 per direction) is routed by
-  VPP. They drive the content of IPv6 FIB size (506 or 508 total /128 prefixes).
-  Destination IPv6 addresses are incremented on a packet by packet
-  basis.
-
-- *ip6scale*: high number of IPv6 flows is routed by VPP. Tested IPv6
-  FIB sizes of /128 prefixes include: i) 20k (10k unique flows per
-  direction), ii) 200k (2x 100k flows) and iii) 2M (2x 1M). Destination
-  IPv6 addresses are incremented on a packet by packet basis, ensuring
-  new FIB entries are looked up at every packet, making it the worst
-  case scenario.
+Similarly to IPv4, IPv6 routing tests are executed in baseline and scale
+configurations:
+
+- *ip6base*: Two /128 IPv4 FIB entries are configured in VPP to enable
+  packet routing between two interfaces in two directions. VPP routing
+  is tested with 253 IPv6 unique flows per direction, varying IPv6
+  source address per flow in order to invoke RSS based packet
+  distribution across VPP workers. IPv6 source address is incremented
+  for every packet.
+
+- *ip4scale*: A high number of /128 IPv6 FIB entries are configured in
+  VPP. Tested IPv6 FIB sizes include: i) 20k with 10k unique flows per
+  direction, ii) 200k with 2 * 100k flows and iii) 2M with 2 * 1M
+  flows. Unique flows are created by using distinct IPv6 destination
+  addresses that are changed for every packet, using incremental or
+  random ordering. For details, see :ref:`packet_flow_ordering`.
 
 SRv6 Routing
 ~~~~~~~~~~~~