Support existing test types with ASTF

+ Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites.
 + Update existing cps traffic profiles.
 + Add missing traffic profiles.
 + UDP:
  + Single burst of 32 packets was confirmed as safe enough for TRex.
  + Maybe 64 could work, but not enough testing for that.
  + Multiple bursts have lead to reduced TRex performance,
    as overlaping bursts (from different client instances)
    tend to fill up the buffers.
 + TCP:
  + Data size set to 11111 bytes, completely arbitrarily.
  + Results look reasonable, so I have kept that.
  - MSS not set at all
 - No tested support for frame size other than 64B.
  - Frame size does not even factor into TCP profiles.
 + So other frame sizes are skipped in autogen.
+ Update tags in related suites.
 - HOSTS_{n} and SRC_USER_{n} should be unified.
 - Questionable clarification on difference between IP4BASE and SCALE.
+ Add NAT state resetters to tests that need them.
 + Resetter is called (if set) before each measurement.
 + If ramp-up is detected, resetter is not set.
+ Rename "mult" argument to "multiplier".
+ Abstracted from packets to transactions.
 + Transaction corresponds to profile.
 + TRex multiplier argument sets target rate in transactions per second.
 + The familiar STL traffic:
  + Bidirectional is considered to be 2 packets per transaction.
  + Unidirectional is considered to be 1 packet per transaction.
 + The newer ASTF traffic:
  + 4 subtypes, each has different number of packets per transaction.
 + For max rate computation:
  + Packets in the more numerous direction are considered.
+ Rely on TRex reported traffic duration for ASTF:
 + Use the server side value.
  - Client side value is higher by an overhead.
  - TRex is not sending traffic during that time.
 + Remove delays from traffic profiles.
  - Those delays would increase the reprted traffic time.
+ Support for scale lmited trials.
 + Only for ASTF profiles, each ASTF profile has limited scale.
 + Scale defined in suite variables.
 + For TRex to send all transactions provided duration value is ignored.
  + The appropriate value is computed in TrafficGenerator.
  + An ad-hoc time constant is added to match the TRex client side time overhead.
  + The profile driver receives the computed duration.
  + Measurement for PLRsearch add a sleep if the computed duration is smaller.
 + Alternative argument for search algos if scale is limited.
  + Both need higher timeout to accomodate big scales.
  + MLRsearch can afford fewer phases.
 + Added a parameter to optionally shorten the duration.
  + Use short duration for runtime stats trial and failure stats trial.
+ Use very large keepalive values in udp profiles to avoid ka packets.
+ No polling in ASTF profile driver.
 - Polling could eliminate the time overhead value.
 + But polling proved to introduce some loss, affecting the results.
+ Handle duration stretching in ASTF by stopping traffic.
 + The stop has several steps so that:
  + The traffic is really stopped entirely.
  + Late packets do not count (maybe as errors).
  + Stats are preserved to read for results (and cleared afterwards).
 + Several quantities added to ReceiveRateMeasurement:
  + Original target duration is preserved (algos need that).
  + Input estimate (tps) for early search iterations.
  + Output estimate (maybe pps) for MRR output.
  + Strict result (unsent counts as loss) for NDR.
+ Use L2 counters (opackets, ipackets) where possible.
 - TRex has trouble processing packets for the L7 ones at high loads.
+ Remove warmup from profile drivers and keywords.
 + Suites should call "Send ramp-up traffic" explicitly if needed.
+ Added parsing for few more counters.
 + Both to use in formulas or just for debug purposes.
- Only 64B cases in autogen, framesize support to be added later.
+ Latency streams during search can be enabled via PERF_USE_LATENCY env var.
+ MLRsearch improvments:
 + Rename argument names to min_rate and max_rate.
 + Use relative receive rate in initial phase.
+ PLRsearch improvements:
 + Careful computation when output (pps) does not match input (tps).
 + Use geometric distribution (instead of Poisson).
  + Helps agains math errors.
  + This should improve estimate stability.
  - But in practice big losses still lead to significant jumps.
+ Traffic generator improvements:
 + send_traffic_on_tg now calls the full set_rate_provider_defaults.
 + _send_traffic_on_tg_internal for the logic without provider defaults.
  + As the internal function is re-used by measure() without affecting defaults.
 + Move _parse_traffic_results just before get_measurement_result.
  + As the latter uses fields set bu the former, it is now easier to read.
 + Multiple sources for approximate duration.
  + Tried from more precise to more available.
 + Includes logic for _pps tests (added in later change).
 + Move explicit type conversions to earlier occurences.
+ Profile driver output field uses semicolons to simplify parsing.
+ Performance Robot lib file split to several smaller ones.
 + performance_actions.robot:
  + Hosts Additional Statistics Action For * keywords.
 + performance_display.robot:
  + Hosts keyword for displaying and verifying results.
  + Change test message to use the correct unit (pps or cps).
 + performance_limits.robot renamed to performance_vars.robot
  + Added many keywords, mostly for accessing test variables.
   + Moved variables for Policer into a new keyword there.
  + Some keywords need sophisticated logic.
  - Other are basically Get Variable Value.
   + But in future more logic can be added, without editing callers.
  + Documentation for the new keywords acts as a documentation for test variables.
 + performance_utils.robot has the rest.
  + Eliminated arguments if the value is in test variable.
  + Small improvements to documentation.
  - Still not enough cleanup with respect to arguments and test variables.
 + Keywords are sorted alphabetically now in each one.
+ Suites:
 + Unified variables table:
  + No colons in comments.
  + ${n_hosts}, ${n_ports} and use them instead hardcoded numbers.
 + Add -cps to existing cps suite names.
  + Remove "trial data overwrite".
  + Compute max rate as in STL suites.
 + Each NAT suite has ip4base suite to compare results to.
  - Those act as indirect TRex calibration.
  - VPP does not lose packets in those.
 + Latency in ASTF suites is disabled hard.
  - As we do not support latency in ASTF profiles yet.
 + Unidirectional tests governed by suite variable, not an argument.
+ Write long argument lists vertically.
 + Prefer to use argument names.
 + In Python, also the last argument is followed by comma.
 + It makes renaming and reordering easier.
 + Similarly applies to prints with long lists of values.
+ A TODO to update api crc file comments.

Change-Id: I84729355edbec051298a9de1162107f88ff5737d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

1 files changed, 117 insertions, 0 deletions

diff --git a/GPL/traffic_profiles/trex/trex-astf-ethip4tcp-262144h-pps.py b/GPL/traffic_profiles/trex/trex-astf-ethip4tcp-262144h-pps.py
new file mode 100644
index 0000000000..4d7215b77d
--- /dev/null
+++ b/GPL/traffic_profiles/trex/trex-astf-ethip4tcp-262144h-pps.py
@@ -0,0 +1,117 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Traffic profile for T-rex advanced stateful (astf) traffic generator.
+
+Traffic profile:
+ - Two streams sent in directions 0 --> 1 (client -> server, requests) and
+   1 --> 0 (server -> client, responses) at the same time.
+ - Packet: ETH / IP / TCP
+ - Direction 0 --> 1:
+   - Source IP address range:      172.16.0.0 - 172.19.255.255
+   - Destination IP address range: 20.16.0.0 - 20.19.255.255
+ - Direction 1 --> 0:
+   - Source IP address range:      destination IP address from packet received
+     on port 1
+   - Destination IP address range: source IP address from packet received
+     on port 1
+
+This is a profile for PPS tests, it combines TCP connect and data transfer.
+No delays, server response waits for full request.
+"""
+
+from trex.astf.api import *
+from profile_trex_astf_base_class import TrafficProfileBaseClass
+
+
+class TrafficProfile(TrafficProfileBaseClass):
+    """Traffic profile."""
+
+    def __init__(self):
+        """Initialization and setting of profile parameters."""
+
+        super(TrafficProfileBaseClass, self).__init__()
+
+        # IPs used in packet headers.
+        self.p1_src_start_ip = u"172.16.0.0"
+        self.p1_src_end_ip = u"172.19.255.255"
+        self.p1_dst_start_ip = u"20.16.0.0"
+        self.p1_dst_end_ip = u"20.19.255.255"
+
+        # Headers length; not used in this profile, just for the record of
+        # header length for TCP packet with 0B payload
+        self.headers_size = 58  # 14B l2 + 20B ipv4 + 24B tcp incl. 4B options
+        self.data_size = 11111
+
+    def define_profile(self):
+        """Define profile to be used by advanced stateful traffic generator.
+
+        This method MUST return:
+
+            return ip_gen, templates, None
+
+        :returns: IP generator and profile templates for ASTFProfile().
+        :rtype: tuple
+        """
+        # client commands
+        prog_c = ASTFProgram()
+        prog_c.connect()
+        prog_c.send(u"1" * self.data_size)
+        prog_c.recv(self.data_size)
+
+        # server commands
+        prog_s = ASTFProgram()
+        prog_s.accept()
+        prog_c.recv(self.data_size)
+        prog_c.send(u"1" * self.data_size)
+
+        # ip generators
+        ip_gen_c = ASTFIPGenDist(
+            ip_range=[self.p1_src_start_ip, self.p1_src_end_ip],
+            distribution=u"seq"
+        )
+        ip_gen_s = ASTFIPGenDist(
+            ip_range=[self.p1_dst_start_ip, self.p1_dst_end_ip],
+            distribution=u"seq"
+        )
+        ip_gen = ASTFIPGen(
+            glob=ASTFIPGenGlobal(ip_offset=u"0.0.0.1"),
+            dist_client=ip_gen_c,
+            dist_server=ip_gen_s
+        )
+
+        # server association
+        s_assoc = ASTFAssociation(rules=ASTFAssociationRule(port=8080))
+
+        # template
+        temp_c = ASTFTCPClientTemplate(
+            program=prog_c,
+            ip_gen=ip_gen,
+            limit=16515072,  # TODO: set via input parameter
+            port=8080
+        )
+        temp_s = ASTFTCPServerTemplate(program=prog_s, assoc=s_assoc)
+        template = ASTFTemplate(client_template=temp_c, server_template=temp_s)
+
+        return ip_gen, template, None
+
+
+def register():
+    """Register this traffic profile to T-Rex.
+
+    Do not change this function.
+
+    :return: Traffic Profiles.
+    :rtype: Object
+    """
+    return TrafficProfile()