Age | Commit message (Collapse) | Author | Files | Lines |
|
+ With ramp-up, without reset, with session verification.
+ Uses the same profile as pps tests.
+ Ramp up duration is not specified, as duration is computed.
+ Timeout tracking with automated ramp-up.
+ Correct computation of next duration.
+ Checking both early and late sessions.
+ No loss measurement also acts as a ramp-up.
+ Return ReceiveRateMeasurement from send_traffic_on_tg_internal,
as that is needed for detecting whether trial is ok as ramp up.
- Udp needs quite low ramp-up rate after recent regression.
- Max scale has higher rate (so failing) to avoid session timeouts.
+ Bump copyright year.
Change-Id: I50c928659cd5b985b490a2e5fb69c5cd790600b0
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit b26fe6de7965f6621b54a80a938f82afe95aa585)
|
|
+ Replace pair of traffic profiles (2n and 3n) with single nodeless one.
+ Compared to incremental suites, randomized ones add IP6_RND tag.
Change-Id: I2f0dfc9e04bbcd0f88e95b92edf2da2c73faaab6
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit ad98ea70cf435d104b6a4f0090575e137660da77)
|
|
Results if NDR search is in TPS (so unidir) since
https://gerrit.fd.io/r/c/csit/+/28208
The bug was causing all reconf tests report zero loss,
as they used half of NDR load during reconfiguration.
+ Small improvements to descriptions of other traffic keywords.
Change-Id: Ib9af5861cc09e698eb762feb3f0a019571d17962
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit 293a88b56179094f655178b838b84844a672390d)
|
|
Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
Change-Id: I70127d8dd2241b70511f31842eb8a9e3cbafdc4b
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit 0f3baca2366f969de6b5d3b889c1b05f72995ec2)
|
|
+ Rename traffic profiles to avoid mentioning number of nodes.
+ Improve 2n rnd suite documentation slightly.
This is a manual cherry-pick of https://gerrit.fd.io/r/c/csit/+/27679
Change-Id: I82d6fb6a99133163a58d56f2acf8a7b9568ee77c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Jira: CSIT-1768
Change-Id: I888ae1a5754fa07297d4cdf65c2be0e3e49d89a5
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit 6962831a1f1905e50a3d663c0cf3ce8c2873ecc9)
Change-Id: I6496ae773b176b2abfedab843110d6f870c62888
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Modify initialize L2BD ACL keywords to be usable for both 2-node and
3-node topologies and update testsuites accordingly.
Add the missing macip T-rex profile.
Add classifier tests to 2n-tx2 job specs.
Change-Id: I17b84b8fc18ef9a6f275ae0238a0665ac2017f01
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
(cherry picked from commit 9d510d595f73d26e3f7c6646fa6530e8f252f790)
|
|
Disconnect+connect cycle is expensive and slow.
This change tracks connected client instances
so later "connect" to the same target uses it.
Explicit disconnects are allowed (and executed before VPP ends),
but once again disconnected instances are cached and reused,
as creating a new instance is more expensive than just connect.
+ Add missing checks on interfaces being up to appropriate keyword.
+ Use appropriate keywords.
+ Add a comment explaining why a simpler keyword is not appropriate.
+ Improve VPP checks in containers.
+ Fix the vppctl check to actually work.
+ Add PAPI check to ensure VPP is really ready.
+ Delay/reorder checks to make them faster with multiple containers.
+ Leave some TODOs to improve various lifecycles later.
+ As we do not stop VPP in test/suite teardown:
+ One final disconnect is needed, added to __init__.robot teardowns.
- Import of the final disconnect keyword is ugly, but it works.
- We could use a hashable class for distinguishing node+socket pairs.
- Are we connecting to VPP inside VMs?
Change-Id: I49cd726740c3e8cae1591c7c84b85a447241228f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit a33b52ae0f255021d89307ebc694f6e907906151)
|
|
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech>
Change-Id: Ifb97088a26f3c2ab2ecee28ed43e6e957a844c64
(cherry picked from commit 9446b9bf9c15999feec3c8a48fec428bca094500)
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Ibafaaca4438587284d1e255c764f0701002941e8
(cherry picked from commit 44dcb3113c8ade2e44543746abca861a89362c9b)
|
|
Jira: CSIT-1597
+ add ipsec_sa tear down action
Change-Id: I4d1e6f26c14e61e8ddc6f29fbf5528a034c84eb0
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Jira: CSIT-1769
Change-Id: I98510cd8e627d7347f77d0032b7bac28f2e36c61
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
This reverts commit a9f54ca5080aeef17686f300a6807bf9b46b7c90.
Reason for revert: DO NOT MERGE BROKEN TESTS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-device-master-ubuntu1804-1n-skx/9918/archives/log.html.gz
Change-Id: I2ce2970a43e5dd7487eeb54d1ccbb149e06cf8fa
Signed-off-by: pmikus <pmikus@cisco.com>
|
|
Jira: CSIT-1769
Change-Id: I0ab906966f760b809b8431530c010a8f0835d62c
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Ticket: CSIT-1771 VPP-1207 VPP-1675
Change-Id: I8ba2d62054361e72b833943327434ea071e7e568
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites.
+ Update existing cps traffic profiles.
+ Add missing traffic profiles.
+ UDP:
+ Single burst of 32 packets was confirmed as safe enough for TRex.
+ Maybe 64 could work, but not enough testing for that.
+ Multiple bursts have lead to reduced TRex performance,
as overlaping bursts (from different client instances)
tend to fill up the buffers.
+ TCP:
+ Data size set to 11111 bytes, completely arbitrarily.
+ Results look reasonable, so I have kept that.
- MSS not set at all
- No tested support for frame size other than 64B.
- Frame size does not even factor into TCP profiles.
+ So other frame sizes are skipped in autogen.
+ Update tags in related suites.
- HOSTS_{n} and SRC_USER_{n} should be unified.
- Questionable clarification on difference between IP4BASE and SCALE.
+ Add NAT state resetters to tests that need them.
+ Resetter is called (if set) before each measurement.
+ If ramp-up is detected, resetter is not set.
+ Rename "mult" argument to "multiplier".
+ Abstracted from packets to transactions.
+ Transaction corresponds to profile.
+ TRex multiplier argument sets target rate in transactions per second.
+ The familiar STL traffic:
+ Bidirectional is considered to be 2 packets per transaction.
+ Unidirectional is considered to be 1 packet per transaction.
+ The newer ASTF traffic:
+ 4 subtypes, each has different number of packets per transaction.
+ For max rate computation:
+ Packets in the more numerous direction are considered.
+ Rely on TRex reported traffic duration for ASTF:
+ Use the server side value.
- Client side value is higher by an overhead.
- TRex is not sending traffic during that time.
+ Remove delays from traffic profiles.
- Those delays would increase the reprted traffic time.
+ Support for scale lmited trials.
+ Only for ASTF profiles, each ASTF profile has limited scale.
+ Scale defined in suite variables.
+ For TRex to send all transactions provided duration value is ignored.
+ The appropriate value is computed in TrafficGenerator.
+ An ad-hoc time constant is added to match the TRex client side time overhead.
+ The profile driver receives the computed duration.
+ Measurement for PLRsearch add a sleep if the computed duration is smaller.
+ Alternative argument for search algos if scale is limited.
+ Both need higher timeout to accomodate big scales.
+ MLRsearch can afford fewer phases.
+ Added a parameter to optionally shorten the duration.
+ Use short duration for runtime stats trial and failure stats trial.
+ Use very large keepalive values in udp profiles to avoid ka packets.
+ No polling in ASTF profile driver.
- Polling could eliminate the time overhead value.
+ But polling proved to introduce some loss, affecting the results.
+ Handle duration stretching in ASTF by stopping traffic.
+ The stop has several steps so that:
+ The traffic is really stopped entirely.
+ Late packets do not count (maybe as errors).
+ Stats are preserved to read for results (and cleared afterwards).
+ Several quantities added to ReceiveRateMeasurement:
+ Original target duration is preserved (algos need that).
+ Input estimate (tps) for early search iterations.
+ Output estimate (maybe pps) for MRR output.
+ Strict result (unsent counts as loss) for NDR.
+ Use L2 counters (opackets, ipackets) where possible.
- TRex has trouble processing packets for the L7 ones at high loads.
+ Remove warmup from profile drivers and keywords.
+ Suites should call "Send ramp-up traffic" explicitly if needed.
+ Added parsing for few more counters.
+ Both to use in formulas or just for debug purposes.
- Only 64B cases in autogen, framesize support to be added later.
+ Latency streams during search can be enabled via PERF_USE_LATENCY env var.
+ MLRsearch improvments:
+ Rename argument names to min_rate and max_rate.
+ Use relative receive rate in initial phase.
+ PLRsearch improvements:
+ Careful computation when output (pps) does not match input (tps).
+ Use geometric distribution (instead of Poisson).
+ Helps agains math errors.
+ This should improve estimate stability.
- But in practice big losses still lead to significant jumps.
+ Traffic generator improvements:
+ send_traffic_on_tg now calls the full set_rate_provider_defaults.
+ _send_traffic_on_tg_internal for the logic without provider defaults.
+ As the internal function is re-used by measure() without affecting defaults.
+ Move _parse_traffic_results just before get_measurement_result.
+ As the latter uses fields set bu the former, it is now easier to read.
+ Multiple sources for approximate duration.
+ Tried from more precise to more available.
+ Includes logic for _pps tests (added in later change).
+ Move explicit type conversions to earlier occurences.
+ Profile driver output field uses semicolons to simplify parsing.
+ Performance Robot lib file split to several smaller ones.
+ performance_actions.robot:
+ Hosts Additional Statistics Action For * keywords.
+ performance_display.robot:
+ Hosts keyword for displaying and verifying results.
+ Change test message to use the correct unit (pps or cps).
+ performance_limits.robot renamed to performance_vars.robot
+ Added many keywords, mostly for accessing test variables.
+ Moved variables for Policer into a new keyword there.
+ Some keywords need sophisticated logic.
- Other are basically Get Variable Value.
+ But in future more logic can be added, without editing callers.
+ Documentation for the new keywords acts as a documentation for test variables.
+ performance_utils.robot has the rest.
+ Eliminated arguments if the value is in test variable.
+ Small improvements to documentation.
- Still not enough cleanup with respect to arguments and test variables.
+ Keywords are sorted alphabetically now in each one.
+ Suites:
+ Unified variables table:
+ No colons in comments.
+ ${n_hosts}, ${n_ports} and use them instead hardcoded numbers.
+ Add -cps to existing cps suite names.
+ Remove "trial data overwrite".
+ Compute max rate as in STL suites.
+ Each NAT suite has ip4base suite to compare results to.
- Those act as indirect TRex calibration.
- VPP does not lose packets in those.
+ Latency in ASTF suites is disabled hard.
- As we do not support latency in ASTF profiles yet.
+ Unidirectional tests governed by suite variable, not an argument.
+ Write long argument lists vertically.
+ Prefer to use argument names.
+ In Python, also the last argument is followed by comma.
+ It makes renaming and reordering easier.
+ Similarly applies to prints with long lists of values.
+ A TODO to update api crc file comments.
Change-Id: I84729355edbec051298a9de1162107f88ff5737d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/29463
- update vpp stable to version 21.01-rc0~283-g5f4f2081c
Change-Id: I079c10e4537448c3b078f22c3fe4ed266a5e2e2c
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I87f95c14034d89056f71ffaeccafc1d3b71ff933
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
There were occasional Router Advertisement packets received in a Traffic
Generator in VM testcases. Fix by adding config that disables sending
IPv6 RA messages on interfaces in VM VPP config before enabling them.
Also rename related keywords and change descriptions based on
https://docs.fd.io/vpp/21.01/db/d4c/clicmd_src_vnet_ip6-nd.html#clicmd_ip6_nd
Change-Id: I1c2a33337ac91f6039d287b4c2aac388e3a52383
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
- Specs
- Tests
- Library
- TODO: PAL
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I8890940d56c7a9924d4a5f9365b6bb881610db14
|
|
- after merge of commit vppinfra: use heap to store bihash data |
https://gerrit.fd.io/r/c/vpp/+/29099
it is necessary to allocate 14GB of memory for main heap in case
of nat44ed test with 16.5M sessions
Change-Id: I65dccdd4f50382b1b40cdffd2d9031ffb7d3bfa1
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I3aa50ec1ef9b0445014daa31e767323060f4a03f
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: Ie24184ca4ac2d6c7abc32f0f103e10bc402ad93b
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I1c638aef886bf37a9feb4a29e4949c7c8f19b717
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- default value for main heap size set to 2G
- allow allocation of n x 2G in tests that need it
Change-Id: I02dafe6bc61649d2823486a3e983f66f5a213a88
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- some tests need to reduce rate for ramp-up phase
- some tests need to extend trail duration in ramp-up phase
- removed 2n1l-10ge2p1x710-ethip4udp-nat44det-h1-p63-s63 suite
as nat out ports are randomly selected from available port range
so T-Rex stateless is not able to provide required out2in traffic
Change-Id: I1145496610d202f81d911e68aa819844d7600918
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: If141566f76d5b2f309953233e693c34fef8ee12f
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- lisp is moving to plugin: https://gerrit.fd.io/r/c/vpp/+/28972
Jira: CSIT-1597
Change-Id: I362f1931b5fc441da9c8a6dda933a66bfed8c4b8
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Iee488e2244a4e253471310bc7fb9640c69c6b0cb
|
|
In VPP 20.05, vpp added async crypto engine that support to use QAT hardware
to do encryption and decryption, vnet/ipsec enabled async mode to use async
crypto engine.
Current async crypto engine also use dpdk_cryptodev as async handlers, in the
future it may add other native QAT driver as async handlers.
Note that async crypto engine is to support vnet/ipsec, it is different
with current existing dpdk backend which itself has ESP implementation
in plugins/dpdk/ipsec.
Change-Id: I4e6eaa7ca1eddb8b1c45212de0684fb26907119b
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
|
|
Jira: CSIT-1756
Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
Change-Id: Ia6fdc6df91f36ffcabd428326195bb9d8f967e52
|
|
Jira: CSIT-1755
Change-Id: I34baa22a49f44da3fa80d91fa2f4132c982fe610
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
+ DPDK 20.08
+ Migrate make -> meson
+ Fix all trending issues
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I31dcb22627c0f8d17ec63c5b138a2da958b006f4
|
|
Change-Id: Idb7b415995f0a3474e379b0c023f64a96e886a55
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- align CSIT code with VPP code changes for NAT44 deterministic
(DET44) feature
- align test names according to snat44ed tests
- remove obsolete 3-node nat tests
- remove 2n1l-10ge2p1x710-ethip4udp-snat44det-h1048576-p63-s66060288 tests
(not enough memory for such high number of sessions)
Change-Id: I9a22b99b4cfa56d18e9c7ef9c58296e202567d42
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I74641cc89d2f25d50b67d51bf2567082b420aabb
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Jira: CSIT-1736
- tcp synthetic profiles w/o data packets
- tcp cps perf tests, phase I (no special "search cps" KW)
Change-Id: I52be34b0fdd51d7a33c8c5de9b46d7064c48f7fa
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- continuation of https://gerrit.fd.io/r/c/csit/+/26898 as there was
reached limit of changes (1000)
Jira: CSIT-1711
- udp synthetic profiles w/o data packets
- udp cps perf tests, phase I (no special "search cps" KW)
Part I means that we are using MRR tests to collect traffic data
until there is ready new CPS test type with corresponding algorithm.
Change-Id: I0d30feb9ecf1d0bff937152656f8eb422f831378
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- provide base routines to run T-Rex in advanced stateful mode
Change-Id: Ib0dc5f2919c370753335f6446860683dc4b12d93
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I3bbe1fe0073ddeead5219993675f24955e8c3dfd
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- test/suite/global
- binary logic is not working
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Ia3d81cbf2c5f04d1093a0a408c84a9ffc6f3eef0
|
|
The following VPP change renames and moves
COP functionality into a brand new plugin:
https://gerrit.fd.io/r/c/vpp/+/27916
In a subsequent change we should rename tests
and start using the new API messages.
This change is just a minimal edit to avoid test failures.
Change-Id: I1c4b81060dd533ef447f3cc44443b71c5e524d59
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Signed-off-by: Maros Mullner <maros.mullner@pantheon.tech>
Change-Id: Ib5f58f60a1409ed139e2846793bf52fdc02a6571
|
|
The test names are unique without it and the information doesn't add
anything extra.
Change-Id: Idc7d6d1d21c8c05691e1757227a0a3787406d370
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
Change-Id: Ie144c22575a7976da0c77e787e450355b73b0006
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I20e01dfe83a961dc8202d33783a678d38e71cff2
|
|
- and 'setup suite topology interfaces no tg' to
use a common keyword to create suite variables
using the required topology information.
Change-Id: I46894948bc86eb7ce72d036e5b84f09c5c1385db
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
|
|
- The VSAP project will be adding hoststack
connections-per-second and requests-per-second
tests of the hoststack using the Apache 'ab'
test tools.
- WRK infra in /resources/libraries will be
cleaned up separately as a part of the
hoststack + LDP + nginx test suite.
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: Ic1d2f4299e8b6ae6be84283f22f6e28dd05bd80f
|
|
Jira: CSIT-1698
Change-Id: I6d9154284990df8877850e4014716510016e485b
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/24663
- update vpp stable to version 20.05-rc0~637
- remove unused L1 and L2 lisp KWs
Change-Id: I2672b6a375ad70c82f331dcc991c145e868108b9
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|