report: updates to methodology section including nat44, acl, ipsec

Change-Id: I13464728d903cba14feedd3cfb78226d50f3d4a1
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>

6 files changed, 225 insertions, 30 deletions

diff --git a/docs/report/introduction/methodology.rst b/docs/report/introduction/methodology.rst
index 61752a4560..f62d415a7e 100644
--- a/docs/report/introduction/methodology.rst
+++ b/docs/report/introduction/methodology.rst
@@ -9,7 +9,9 @@ Test Methodology
     methodology_terminology
     methodology_vpp_forwarding_modes
     methodology_tunnel_encapsulations
-    methodology_vpp_features
+    methodology_ipsec
+    methodology_nat44
+    methodology_acls
     methodology_data_plane_throughput/index
     methodology_packet_latency
     methodology_multi_core_speedup
@@ -20,5 +22,4 @@ Test Methodology
     methodology_lxc_drc_container_memif
     methodology_nfv_service_density
     methodology_vpp_device_functional
-    methodology_ipsec_on_intel_qat
     methodology_trex_traffic_generator
diff --git a/docs/report/introduction/methodology_vpp_features.rst b/docs/report/introduction/methodology_acls.rst
index a19caa4428..512532fead 100644
--- a/docs/report/introduction/methodology_vpp_features.rst
+++ b/docs/report/introduction/methodology_acls.rst
@@ -1,5 +1,5 @@
-VPP Features
-------------
+Access Control Lists
+--------------------
 
 VPP is tested in a number of data plane feature configurations across
 different forwarding modes. Following sections list features tested.
@@ -66,15 +66,3 @@ entries and number of flows:
 - {F} - number of UDP flows with different tuple (dst-ip4, dst-mac),
   {F} = [100, 10k, 100k]
 - All {E}x{F} combinations are tested per ACL type, total of 9.
-
-NAT44
-~~~~~
-
-NAT44 is tested in baseline and scale configurations with IPv4 routing:
-
-- *ip4base-nat44*: baseline test with single NAT entry (addr, port),
-  single UDP flow.
-- *ip4base-udpsrcscale{U}-nat44*: baseline test with {U} NAT entries
-  (addr, {U}ports), {U}=15.
-- *ip4scale{R}-udpsrcscale{U}-nat44*: scale tests with {R}*{U} NAT
-  entries ({R}addr, {U}ports), {R}=[100, 1k, 2k, 4k], {U}=15.
diff --git a/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst b/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst
index 3df54b3963..6389353a65 100644
--- a/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst
+++ b/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst
@@ -1,3 +1,5 @@
+.. _data_plane_throughput:
+
 Data Plane Throughput Tests
 ---------------------------
 
diff --git a/docs/report/introduction/methodology_ipsec.rst b/docs/report/introduction/methodology_ipsec.rst
new file mode 100644
index 0000000000..ee0572ffc5
--- /dev/null
+++ b/docs/report/introduction/methodology_ipsec.rst
@@ -0,0 +1,52 @@
+Internet Protocol Security (IPsec)
+----------------------------------
+
+VPP IPsec performance tests are executed for the following crypto
+plugins:
+
+- `crypto_native`, used for software based crypto leveraging CPU
+  platform optimizations e.g. Intel's AES-NI instruction set.
+- `crypto_ipsecmb`, used for hardware based crypto with Intel QAT PCIe
+  cards.
+
+IPsec with VPP Native SW Crypto
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Currently |csit-release| implements following IPsec test cases relying
+on VPP native crypto (`crypto_native` plugin):
+
++-------------------+------------------+----------------+------------------+
+| VPP Crypto Engine | ESP Encryption   | ESP Integrity  | Scale Tested     |
++===================+===================+===============+==================+
+| crypto_native     | AES[128|256]-GCM | GCM            | 1 to 60k tunnels |
++-------------------+------------------+----------------+------------------+
+| crypto_native     | AES128-CBC       | SHA[256|512]   | 1 to 60k tunnels |
++-------------------+------------------+----------------+------------------+
+
+VPP IPsec with SW crypto are executed in both tunnel and policy modes,
+with tests running on 3-node testbeds: 3n-hsw and 3n-skx.
+
+IPsec with Intel QAT HW
+^^^^^^^^^^^^^^^^^^^^^^^
+
+Currently |csit-release| implements following IPsec test cases relying
+on ipsecmb library (`crypto_ipsecmb` plugin) and Intel QAT 8950 (50G HW
+crypto card):
+
+dpdk_cryptodev
+
++-------------------+---------------------+------------------+----------------+------------------+
+| VPP Crypto Engine | VPP Crypto Workers  | ESP Encryption   | ESP Integrity  | Scale Tested     |
++===================+=====================+==================+================+==================+
+| crypto_ipsecmb    | sync/all workers    | AES[128|256]-GCM | GCM            | 1, 1k tunnels    |
++-------------------+---------------------+------------------+----------------+------------------+
+| crypto_ipsecmb    | sync/all workers    | AES[128]-CBC     | SHA[256|512]   | 1, 1k tunnels    |
++-------------------+---------------------+------------------+----------------+------------------+
+| crypto_ipsecmb    | async/crypto worker | AES[128|256]-GCM | GCM            | 1, 4, 1k tunnels |
++-------------------+---------------------+------------------+----------------+------------------+
+| crypto_ipsecmb    | async/crypto worker | AES[128]-CBC     | SHA[256|512]   | 1, 4, 1k tunnels |
++-------------------+---------------------+------------------+----------------+------------------+
+
+VPP IPsec with HW crypto are executed in both tunnel and policy modes,
+with tests running on 3-node Haswell testbeds (3n-hsw), as these are the
+only testbeds equipped with Intel QAT cards.
diff --git a/docs/report/introduction/methodology_ipsec_on_intel_qat.rst b/docs/report/introduction/methodology_ipsec_on_intel_qat.rst
deleted file mode 100644
index 54fb1b0ef2..0000000000
--- a/docs/report/introduction/methodology_ipsec_on_intel_qat.rst
+++ /dev/null
@@ -1,14 +0,0 @@
-IPSec on Intel QAT
-------------------
-
-VPP IPSec performance tests are using DPDK cryptodev device driver in
-combination with HW cryptodev devices - Intel QAT 8950 50G - present in
-LF FD.io physical testbeds. DPDK cryptodev can be used for all IPSec
-data plane functions supported by VPP.
-
-Currently |csit-release| implements following IPSec test cases:
-
-- AES-GCM, CBC-SHA1 ciphers, in combination with IPv4 routed-forwarding
-  with Intel xl710 NIC.
-- CBC-SHA1 ciphers, in combination with LISP-GPE overlay tunneling for
-  IPv4-over-IPv4 with Intel xl710 NIC.
diff --git a/docs/report/introduction/methodology_nat44.rst b/docs/report/introduction/methodology_nat44.rst
new file mode 100644
index 0000000000..5dc558f029
--- /dev/null
+++ b/docs/report/introduction/methodology_nat44.rst
@@ -0,0 +1,166 @@
+Network Address Translation IPv4 to IPv4
+----------------------------------------
+
+NAT44 Prefix Bindings
+^^^^^^^^^^^^^^^^^^^^^
+
+NAT44 prefix bindings should be representative to target applications,
+where a number of private IPv4 addresses from the range defined by
+:rfc:`1918` is mapped to a smaller set of public IPv4 addresses from the
+public range.
+
+Following quantities are used to describe inside to outside IP address
+and port bindings scenarios:
+
+- inside-addresses, ports-per-inside-address, number of inside source
+  addresses (representing inside hosts) and number of TCP/UDP source
+  ports per inside source address.
+- outside-addresses, number of outside (public) source addresses
+  allocated to NAT44. The maximal number of ports-per-outside-address
+  usable for NAT is 64 512 (in non-reserved port range 1024-65535,
+  :rfc:`4787`).
+- sharing-ratio, equal to inside-addresses / outside-addresses.
+
+CSIT NAT44 tests are designed to take into account the maximum number of
+ports (sessions) required per inside host (inside-address) and at the
+same time to maximize the use of outside-address range by using all
+available outside ports. With this in mind, the following scheme of
+NAT44 sharing ratios has been devised for use in CSIT:
+
++--------------------------+---------------+
+| ports-per-inside-address | sharing-ratio |
++==========================+===============+
+| 63                       | 1024          |
++--------------------------+---------------+
+| 126                      | 512           |
++--------------------------+---------------+
+| 252                      | 256           |
++--------------------------+---------------+
+| 504                      | 128           |
++--------------------------+---------------+
+
+Initial CSIT NAT44 tests, including associated TG/TRex traffic profiles,
+are based on ports-per-inside-address set to 63 and the sharing ratio of
+1024. This approach is currently used for all NAT44 tests including
+NAT44det (NAT44 deterministic used for Carrier Grade NAT applications)
+and NAT44ed.
+
+..
+    .. TODO::
+
+    Note that in the latter case, due to overloading of (ouside-address,
+    outside-port) tuple for different endpoint destinations the actual
+    sharing ratio is likely to different, as it will depend on the
+    destination addresses used by NAT'ed flows.
+
+Private address ranges to be used in tests:
+
+- 192.168.0.0 - 192.168.255.255 (192.168/16 prefix)
+
+  - Total of 2^16 (65 536) of usable IPv4 addresses.
+  - Used in tests for up to 65 536 inside addresses (inside hosts).
+
+- 172.16.0.0 - 172.31.255.255  (172.16/12 prefix)
+
+  - Total of 2^20 (1 048 576) of usable IPv4 addresses.
+  - Used in tests for up to 1 048 576 inside addresses (inside hosts).
+
+NAT44 Session Scale
+~~~~~~~~~~~~~~~~~~~
+
+NAT44 session scale tested is govern by the following logic:
+
+- Number of inside addresses/hosts H[i] = (H[i-1] x 2^2) with H(0)=1 024, i = 1,2,3, ...
+
+  - H[i] = 1 024, 4 096, 16 384, 65 536, 262 144, 1 048 576, ...
+
+- Number of sessions S[i](ports-per-host) = H[i] * ports-per-inside-address
+
+  - ports-per-host = 63
+
++---+---------+------------+
+| i |   hosts |   sessions |
++===+=========+============+
+| 0 |   1 024 |     64 512 |
++---+---------+------------+
+| 1 |   4 096 |    258 048 |
++---+---------+------------+
+| 2 |  16 384 |  1 032 192 |
++---+---------+------------+
+| 3 |  65 536 |  4 128 768 |
++---+---------+------------+
+| 4 | 262 144 | 16 515 072 |
++---+---------+------------+
+
+NAT44 Deterministic
+^^^^^^^^^^^^^^^^^^^
+
+NAT44det throughput tests are using TRex STL (Stateless) API and traffic
+profiles, similar to all other stateless packet forwarding tests like
+ip4, ip6 and l2, sending UDP packets in both directions
+inside-to-outside and outside-to-inside. See
+:ref:`data_plane_throughput` for more detail.
+
+NAT44det translation entries are created during the ramp-up phase
+preceding the throughput test, followed by verification that all entries
+are present, before proceeding to the throughput test. This ensures
+session setup does not impact the forwarding performance test.
+
+Associated CSIT test cases use the following naming scheme to indicate
+NAT44det scenario tested:
+
+- ethip4udp-nat44det-h{H}-p{P}-s{S}-[mrr|ndrpdr|soak]
+
+  - {H}, number of inside hosts, H = 1024, 4096, 16384, 65536, 262144.
+  - {P}, number of ports per inside host, P = 63.
+  - {S}, number of sessions, S = 64512, 258048, 1032192, 4128768,
+    16515072.
+  - [mrr|ndrpdr|soak], MRR, NDRPDR or SOAK test.
+
+NAT44 Endpoint-Dependent
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+NAT44ed is benchmarked using following methodologies:
+
+- Uni-directional throughput using *stateless* traffic profile.
+- Connections-per-second using *stateful* traffic profile.
+- Bi-directional throughput using *stateful* traffic profile.
+
+Uni-directional NAT44ed throughput tests are using TRex STL (Stateless)
+APIs and traffic profiles, but with packets sent only in
+inside-to-outside direction. Due to indeterministic bindings of outside
+to inside (src_addr,src_port) that are created dynamically at flow start
+bidirectional testing is not possible with stateless traffic profiles.
+See :ref:`data_plane_throughput` for more detail.
+
+Similarly to NAT44det, NAT44ed uni-directional throughput tests include
+a ramp-up phase to establish and verify the presence of required NAT44ed
+binding entries. NAT44ed CPS (connections-per-second) and throughput /
+PPS stateful tests do not have a ramp-up phase.
+
+Stateful NAT44ed tests are using TRex ASTF (Advanced Stateful) APIs and
+traffic profiles, with packets sent in both directions. Tests are run
+with both UDP and TCP/IP sessions.
+
+Associated CSIT test cases use the following naming scheme to indicate
+NAT44DET case tested:
+
+- Stateless: ethip4udp-nat44ed-h{H}-p{P}-s{S}-udir-[mrr|ndrpdr|soak]
+
+  - {H}, number of inside hosts, H = 1024, 4096, 16384, 65536, 262144.
+  - {P}, number of ports per inside host, P = 63.
+  - {S}, number of sessions, S = 64512, 258048, 1032192, 4128768,
+    16515072.
+  - udir-[mrr|ndrpdr|soak], unidirectional stateless tests MRR, NDRPDR
+    or SOAK.
+
+- Stateful: ethip4[udp|tcp]-nat44ed-h{H}-p{P}-s{S}-[cps|pps]-[mrr|ndrpdr]
+
+  - [udp|tcp], UDP or TCP/IP sessions
+  - {H}, number of inside hosts, H = 1024, 4096, 16384, 65536, 262144.
+  - {P}, number of ports per inside host, P = 63.
+  - {S}, number of sessions, S = 64512, 258048, 1032192, 4128768,
+    16515072.
+  - [cps|pps], connections-per-second session establishment rate or
+    packets-per-second throughput rate.
+  - [mrr|ndrpdr], bidirectional stateful tests MRR, NDRPDR.