aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/robot/ip
AgeCommit message (Collapse)AuthorFilesLines
2020-10-26Support existing test types with ASTFVratko Polak1-2/+22
Manual cherry-pick from master [1], reverting impact of [2] and [3]. [1] https://gerrit.fd.io/r/c/csit/+/28208/176 [2] https://gerrit.fd.io/r/c/csit/+/29077 [3] https://gerrit.fd.io/r/c/csit/+/29529 The heap multipliers are left in suites, as that simplifies cherry-picking between branches. Original [0] commit message: Support existing test types with ASTF + Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites. + Update existing cps traffic profiles. + Add missing traffic profiles. + UDP: + Single burst of 32 packets was confirmed as safe enough for TRex. + Maybe 64 could work, but not enough testing for that. + Multiple bursts have lead to reduced TRex performance, as overlaping bursts (from different client instances) tend to fill up the buffers. + TCP: + Data size set to 11111 bytes, completely arbitrarily. + Results look reasonable, so I have kept that. - MSS not set at all - No tested support for frame size other than 64B. - Frame size does not even factor into TCP profiles. + So other frame sizes are skipped in autogen. + Update tags in related suites. - HOSTS_{n} and SRC_USER_{n} should be unified. - Questionable clarification on difference between IP4BASE and SCALE. + Add NAT state resetters to tests that need them. + Resetter is called (if set) before each measurement. + If ramp-up is detected, resetter is not set. + Rename "mult" argument to "multiplier". + Abstracted from packets to transactions. + Transaction corresponds to profile. + TRex multiplier argument sets target rate in transactions per second. + The familiar STL traffic: + Bidirectional is considered to be 2 packets per transaction. + Unidirectional is considered to be 1 packet per transaction. + The newer ASTF traffic: + 4 subtypes, each has different number of packets per transaction. + For max rate computation: + Packets in the more numerous direction are considered. + Rely on TRex reported traffic duration for ASTF: + Use the server side value. - Client side value is higher by an overhead. - TRex is not sending traffic during that time. + Remove delays from traffic profiles. - Those delays would increase the reprted traffic time. + Support for scale lmited trials. + Only for ASTF profiles, each ASTF profile has limited scale. + Scale defined in suite variables. + For TRex to send all transactions provided duration value is ignored. + The appropriate value is computed in TrafficGenerator. + An ad-hoc time constant is added to match the TRex client side time overhead. + The profile driver receives the computed duration. + Measurement for PLRsearch add a sleep if the computed duration is smaller. + Alternative argument for search algos if scale is limited. + Both need higher timeout to accomodate big scales. + MLRsearch can afford fewer phases. + Added a parameter to optionally shorten the duration. + Use short duration for runtime stats trial and failure stats trial. + Use very large keepalive values in udp profiles to avoid ka packets. + No polling in ASTF profile driver. - Polling could eliminate the time overhead value. + But polling proved to introduce some loss, affecting the results. + Handle duration stretching in ASTF by stopping traffic. + The stop has several steps so that: + The traffic is really stopped entirely. + Late packets do not count (maybe as errors). + Stats are preserved to read for results (and cleared afterwards). + Several quantities added to ReceiveRateMeasurement: + Original target duration is preserved (algos need that). + Input estimate (tps) for early search iterations. + Output estimate (maybe pps) for MRR output. + Strict result (unsent counts as loss) for NDR. + Use L2 counters (opackets, ipackets) where possible. - TRex has trouble processing packets for the L7 ones at high loads. + Remove warmup from profile drivers and keywords. + Suites should call "Send ramp-up traffic" explicitly if needed. + Added parsing for few more counters. + Both to use in formulas or just for debug purposes. - Only 64B cases in autogen, framesize support to be added later. + Latency streams during search can be enabled via PERF_USE_LATENCY env var. + MLRsearch improvments: + Rename argument names to min_rate and max_rate. + Use relative receive rate in initial phase. + PLRsearch improvements: + Careful computation when output (pps) does not match input (tps). + Use geometric distribution (instead of Poisson). + Helps agains math errors. + This should improve estimate stability. - But in practice big losses still lead to significant jumps. + Traffic generator improvements: + send_traffic_on_tg now calls the full set_rate_provider_defaults. + _send_traffic_on_tg_internal for the logic without provider defaults. + As the internal function is re-used by measure() without affecting defaults. + Move _parse_traffic_results just before get_measurement_result. + As the latter uses fields set bu the former, it is now easier to read. + Multiple sources for approximate duration. + Tried from more precise to more available. + Includes logic for _pps tests (added in later change). + Move explicit type conversions to earlier occurences. + Profile driver output field uses semicolons to simplify parsing. + Performance Robot lib file split to several smaller ones. + performance_actions.robot: + Hosts Additional Statistics Action For * keywords. + performance_display.robot: + Hosts keyword for displaying and verifying results. + Change test message to use the correct unit (pps or cps). + performance_limits.robot renamed to performance_vars.robot + Added many keywords, mostly for accessing test variables. + Moved variables for Policer into a new keyword there. + Some keywords need sophisticated logic. - Other are basically Get Variable Value. + But in future more logic can be added, without editing callers. + Documentation for the new keywords acts as a documentation for test variables. + performance_utils.robot has the rest. + Eliminated arguments if the value is in test variable. + Small improvements to documentation. - Still not enough cleanup with respect to arguments and test variables. + Keywords are sorted alphabetically now in each one. + Suites: + Unified variables table: + No colons in comments. + ${n_hosts}, ${n_ports} and use them instead hardcoded numbers. + Add -cps to existing cps suite names. + Remove "trial data overwrite". + Compute max rate as in STL suites. + Each NAT suite has ip4base suite to compare results to. - Those act as indirect TRex calibration. - VPP does not lose packets in those. + Latency in ASTF suites is disabled hard. - As we do not support latency in ASTF profiles yet. + Unidirectional tests governed by suite variable, not an argument. + Write long argument lists vertically. + Prefer to use argument names. + In Python, also the last argument is followed by comma. + It makes renaming and reordering easier. + Similarly applies to prints with long lists of values. + A TODO to update api crc file comments. Change-Id: I84729355edbec051298a9de1162107f88ff5737d Signed-off-by: Vratko Polak <vrpolak@cisco.com> Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-12Test: Add ramp-up phase to nat44ed-udir testsJan Gelety1-1/+33
Change-Id: I3aa50ec1ef9b0445014daa31e767323060f4a03f Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit d68be735d882bafcb672ebb27a66efbcabbeb02d) Change-Id: Iad67c8445e18b22eccbea25d75b91827b398775f Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-12Framework: Add possibility to send ramp-up trafficJan Gelety1-0/+6
Change-Id: Ie24184ca4ac2d6c7abc32f0f103e10bc402ad93b Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit 61044d391d6e8d6b47d0d4f156071bd61cd278df) Change-Id: I50e2a674784688e6eeea566fc2bc4d45a8ecfb8b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-11test: nat44det - add session number checkJan Gelety1-0/+17
- some tests need to reduce rate for ramp-up phase - some tests need to extend trail duration in ramp-up phase - removed 2n1l-10ge2p1x710-ethip4udp-nat44det-h1-p63-s63 suite as nat out ports are randomly selected from available port range so T-Rex stateless is not able to provide required out2in traffic Change-Id: I1145496610d202f81d911e68aa819844d7600918 Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit 3b408b7ea702dd3817442186035121fe862cbf7f) Change-Id: I53da8c086373d06e0842e5563964d9287c0fa403 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-07VPP-DEV API COV: Add NAT44-ED testsJan Gelety1-1/+1
Jira: CSIT-1755 Change-Id: I34baa22a49f44da3fa80d91fa2f4132c982fe610 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-08-08Fix: NAT44 deterministic moved to separate det44 pluginJan Gelety1-43/+83
- align CSIT code with VPP code changes for NAT44 deterministic (DET44) feature - align test names according to snat44ed tests - remove obsolete 3-node nat tests - remove 2n1l-10ge2p1x710-ethip4udp-snat44det-h1048576-p63-s66060288 tests (not enough memory for such high number of sessions) Change-Id: I9a22b99b4cfa56d18e9c7ef9c58296e202567d42 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-08-20Fix: Use new-style interface variablesVratko Polak1-2/+2
The old-style variables contain None in some places, mainly in SRv6 proxy tests. Change-Id: If3887a7dba051454c504b345a6a316d5d69d0139 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-08-07Perf: NAT44 endpoint-dependent mode - udp, part IJan Gelety2-71/+69
- continuation of https://gerrit.fd.io/r/c/csit/+/26898 as there was reached limit of changes (1000) Jira: CSIT-1711 - udp synthetic profiles w/o data packets - udp cps perf tests, phase I (no special "search cps" KW) Part I means that we are using MRR tests to collect traffic data until there is ready new CPS test type with corresponding algorithm. Change-Id: I0d30feb9ecf1d0bff937152656f8eb422f831378 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-07-23T-Rex: Add advanced stateful modeJan Gelety1-4/+4
- provide base routines to run T-Rex in advanced stateful mode Change-Id: Ib0dc5f2919c370753335f6446860683dc4b12d93 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-06-10NAT44 EI testsMaros Mullner1-18/+46
Signed-off-by: Maros Mullner <maros.mullner@pantheon.tech> Change-Id: Ib5f58f60a1409ed139e2846793bf52fdc02a6571
2020-06-03FIX: IP4 IP6 3n-onlypmikus2-3/+3
ARP not resolved as ARP plugin is not in use. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I655d39794f99e32a2f3905154d78e83853356ccf
2020-05-06FIX: dot1q-ip6pmikus1-1/+1
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ied48646bd9de41e5a55e0006440d057c5fddf694
2020-04-27FIX: NAT44 3-node testsPeter Mikus1-1/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I3d9074175e1aaf5afba4e351790644df4ad33914
2020-04-23Fix 3n ip4 scale initVratko Polak1-1/+1
Last argument to Vpp Add Ip Neighbor should be a MAC address. Change-Id: Ia93ddf68aa2b8647c66b6a72210db3a8a01b81fb Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-03-31FIX: ip4Jan Gelety1-1/+1
Change-Id: I57fcb06c72caf517231f09478d9d6089d2f46af4 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-04-06Improve pf layerPeter Mikus3-135/+181
+ Merge single/double link + Introduce _pf{n}[0] variables so we can access physical function same way as virtual function + Cleanup code by moving complex logic to python + Prepare code for multiple vf functions Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ic2e74a38bfa146441357de8f0916aeb638941c49
2019-11-28Python3: resources and librariesJan Gelety3-117/+122
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-09-05FIX: Use vhost macs when testpmd_mac forwarding used in VMJan Gelety1-8/+52
Change-Id: I291c3dc852e7c1bde6fb6102a9168cfec9992682 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-12Clean: Various librariesPeter Mikus2-1/+143
- Remove translate TC as they have no perf opposite - Remove unused keywords - Merge similar keywords / rename - Move keywords from perf library to categories Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6433006136e2497ad8fb17193aa20c578d836999
2019-08-09Clean: L2 bridge domainPeter Mikus2-4/+4
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I3721900b91a9c8e9db8766bdcb9771b6bcf95bca
2019-08-09Clean: Interface KeywordsPeter Mikus1-312/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ideddcea56120cf770aec78ca24813baf7d05ba11
2019-08-07Clean: IPv6 keywordsPeter Mikus1-6/+200
+ Move all base ipv6 related KW to library + Remove useless KW to save space in output.xml + Remove cross dependency in ip6.robot Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I8c5f72e9852c1017356ab6034c4d271cf7b53265
2019-08-07Clean: IPv4 keywordsPeter Mikus1-17/+511
+ Move all base ipv4 related KW to library + Remove useless KW to save space in output.xml + Remove cross dependency in ip4.robot Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: If1e3a6370d36caf4ce7411fb956a0b7d0f939b6b
2019-07-29Remove obsolete L2 KWPeter Mikus2-467/+0
Change-Id: I9a8aa6f5ccf95d632f4e677c8bf14f50b982d930 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-19VAT-to-PAPI: NATUtilTibor Frank1-78/+0
Change-Id: Id64bbf899e78c7c55410a8ce608393c22fa596ab Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-18CSIT-1459: Migrate IP libraries from VAT to PAPIJan Gelety2-313/+38
Change-Id: Id84cf9c3afd53260b5bbf658a6a3870aa0c91333 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-05-31CSIT-1468: InterfaceUtil migration from VAT to PAPIJan Gelety2-0/+28
Change-Id: I0062710d58996be767a852dc00545fedd60a5c72 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-05-09Remove old VPP Restart sequencePeter Mikus1-1/+3
Change-Id: I7f71a9709ad55ce03d73e23cc8a9f5064947ed51 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-01-24FIX: correct call of vpp_add_routeJan Gelety2-26/+38
Change-Id: I63a2b5f484ac627c020e30c9f01a868c57aacf42 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-12-19FIX: Avoid duplicated KW names in perf and func testsJan Gelety2-28/+28
Change-Id: I66c68edf6e26fa7baa7bdc5f5d60d4d4aa3af6b9 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-12-18FIX: do not use dump stats in IPv6 base testsJan Gelety1-5/+0
Change-Id: I9983b4aa1d6575f017bdfd69b6ab3b26206d9605 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-12-07VPP_Device - add baseline tests - part IJan Gelety2-64/+478
CSIT-1370 Add following baseline tests to VPP_Device: - ip4 eth2p-ethip4-ip4base-dev - ip6 eth2p-ethip6-ip6base-dev - l2bd eth2p-eth-l2bdbasemaclrn-dev Change-Id: Ic20000eb38c5511c27d020f3aa76b2620d7f7ef9 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-10-10Add VXLAN scale perf testsJan Gelety1-0/+7
Jira: CSIT-1273 Change-Id: Ic2a41661c158384a5aaa7d4e73f30ffca13ddf82 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2017-09-08CSIT-665: Re-write L1 robot keywords in pythonDasa Simkova1-12/+14
Change-Id: Iddd823375ddd17b9abd5f02ee3aea3bfcb625149 Signed-off-by: Dasa Simkova <dspropagacie@gmail.com>
2017-08-28FIX: SNAT -> NAT renaming of APIPeter Mikus1-33/+32
- Based on https://gerrit.fd.io/r/#/c/8146/ Change-Id: I774483c7e2b419a9b9b6be78812ecf53a5c91eab Signed-off-by: Peter Mikus <pmikus@cisco.com>
2017-06-29CSIT-687: Directory structure reorganizationTibor Frank4-0/+817
Change-Id: I772c9e214be2461adf58124998d272e7d795220f Signed-off-by: Tibor Frank <tifrank@cisco.com> Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>