Age | Commit message (Collapse) | Author | Files | Lines |
|
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com>
Change-Id: I1590d438216e4c886aae03f83364837153623dbc
|
|
+ Model version 1.0.0.
- Only some result types are exported.
+ MRR, NDRPDR and SOAK.
- Other result types to be added later.
+ In contrast, all test types are detected.
+ Convert custom classes to JSON-serializable equivalents.
+ Sort dict keys before converting to JSON.
+ Override the order for some known keys.
+ Export sets as sorted arrays.
+ Convert to info content from serialized raw content.
+ Also export outputs for suite setups and teardowns.
+ Info files for setup/teardown exist only temporarily.
+ The data is merged into suite.info.json file.
+ This simplifies presentation of total suite duration.
+ Define model via JSON schema:
- Just test case, suite setup/teardown/suite to be added later.
- Just info, raw to be added later.
+ Proper descriptions.
+ Json is generated from yaml.
+ This is a convenience for maintainers.
+ The officially used schema is the .json one.
+ TODOs written into a separate .txt file.
+ Validate exported instance against the schema.
+ Include format checking.
+ Update CSIT requirements for validation dependencies.
+ This needs python-dateutil==2.8.2, only a patch bump.
+ Compute bandwidth also for soak tests.
+ This unifies with NDRPDR to simplify schema definition.
- PAL may need an update for parsing soak test message.
+ Include SSH log items, raw output only.
+ Generate all outputs in a single filesystem tree.
+ Move raw outputs into test_output_raw.tar.xz.
+ Rename existing tar with suites to generated_robot_files.tar.xz.
Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Signed-off-by: Viliam Luc <vluc@cisco.com>
Change-Id: Ib0a853f7bd8cdbb0dee747e4f601bc4e54d006e9
|
|
When building documentation using sphinx we see ~1200 similar warnings [0]
[0] - https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-verify-tox-master-ubuntu2004-x86_64/3289/doc_verify.log.gz
These warning are harmless and can be fixed later
Signed-off-by: Viliam Luc <vluc@cisco.com>
Change-Id: I1ac1099d38935971d47491dde905715345d3935c
|
|
Some testbeds occasionally miss one session with the previous value.
+ Increase ramp-up duration proportionally.
Change-Id: I5300127d7e8fbc670c37ae392ab514b077b3c9b1
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Signed-off-by: Maros Mullner <maros.mullner@pantheon.tech>
Change-Id: I83b00d56a50d98c36547621cb2f41cb3b2f35741
|
|
Remaining ipsec reconf test suites for 40 tunnels include:
- int-aes128cbc-hmac512sha
- int-aes256gcm
Change-Id: I9c5f549368595bb0cbd0d7e9918b7695184d31c4
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: If98f575503a59106fe8b94d5c683881412ea4f1c
|
|
Remove following test suites:
- ipsec tests
- ethip4ipsectptlispgpe.
- policy-aes128gcm.
- policy-aes128cbc-hmac256sha.
- policy-aes128cbc-hmac512sha.
- int-aes128cbc-hmac256sha.
- scale of
- 400tnlsw.
- 5000tnlsw.
- 20000tnlsw.
- 60000tnlsw.
Change-Id: Ib4915213f6c440c70f9b17cbaac834245e51bd85
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
|
|
Add 1, 10, 100 and 1000 SPD entries using IPv4 outbound flow cache
optimization.
Change-Id: I7abb65a82454c17ef754cb11386186610f0c27e8
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
Change-Id: I75e836ce728fd333764f17d69813093cb947ef4f
Signed-off-by: Viliam Luc <vluc@cisco.com>
|
|
Change-Id: I8679c819b7c6ecd9e680bdc6edeafd2124def0b5
Signed-off-by: Viliam Luc <vluc@cisco.com>
|
|
The old one got deduplicated in https://gerrit.fd.io/r/c/csit/+/32687
Change-Id: Ie82ba36a0805154f17e9fcde0efafb93a686c686
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
New IPUtil.Vpp_Route_Add implementation detects wrong values,
as it refuses start addresses with host bits set.
Fixed by copying in_mask values from nearby ASTF suites.
Change-Id: I768c023bd004ce97b74199d70e5768c5e948d429
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Add testcases with plain ipv4 forwarding with 1, 10, 100 and 1000 SPD
entries on outbound traffic in both directions both directions. Only
match the last SPD entry and process others before the matching entry.
Add testcases only without flow cache optimization.
Refactor the Python functions that add SPD entries:
- Unify the args in functions that add one and multiple entries.
- For multiple entries, add the ability to pass an object that will
handle how values in each iteration (i.e. for each entry) are modified.
Change-Id: I061922eec6acc75a4e115202c07e72d89bf1f4d3
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I770bb2114d92bda4ab2408be9b075263c1cd2223
|
|
The list of actions stat_pre_trial is executed
after the list stat_runtime, which already contains vpp-runtime.
Change-Id: If9b403f38b2f811d77f36d9cae973130f30015da
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I6826add7b3032041632c3952c45a3c64409400b0
|
|
+ Set overhead in those suites to numeric values.
+ Change the library to tolerate string representations anyway.
Change-Id: Ic6215840f7797801c994a38db5637999eb85a034
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I909942dbb920df7f0fe15c0c92cda92c3cd8d8ad
|
|
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com>
Reviewed-by: Yulong Pei <yulong.pei@intel.com>
Change-Id: Ibe4420529192559a3cdbb84e0a67859750a9e0d5
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I2f019a083916aec9f7816266f6ad5b92dcc31fa0
|
|
Change-Id: I0c973ceffc13f4f3c5f959d56cb0890b87091a82
Signed-off-by: Viliam Luc <viliam.luc@pantheon.tech>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I710af90aa5474381a82095c8a11ecf6c01c2483b
|
|
Change-Id: Ie2e735971934d8aeac2d436505c120f09ce67ac7
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
|
|
1. Suite steup add download nginx
2. Add nginx-1.14.2/1.15.0 ldp test suite
3. Add NginxUtils,NginxConfigGenerator method
4. Taskset the PID of nginx to the unused cores in VPP and these cores are under NIC's NUMA ID
5. cleanup add Kill Processes - nohup
Signed-off-by: xizhanx <xix.zhang@intel.com>
Change-Id: Idbf0e4ec3bf63e88281a8e3e34f52e00a6801c85
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
|
|
Add more granularity test policy tests. Mirror the number of tunnels in
other IPsec tests under 1000 tunnels.
Change-Id: I9bde7447a5d809bab05db132bf6cb524e97e19b3
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
Add 2n1l udir 1 and 1000 tunnel sw policy IPsec tests to mirror the
existing 3n tests.
Add static ip neighbor mac entries which can't be retrieved in 2node
setups.
Change-Id: I13dd557cbeed7f907fa9b4c21e4e245d48916513
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
Suite name part is correct: swasync.
This fixes the tag, from IPSECHW to IPSECSW.
Change-Id: I13a5201dade7edf180d315416c82c15d69c1bfce
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Add unidirectional 2n crypto tests. Only one direction can be tested on
a 2 node topology, since we can't use the same interface for both
encrypted and unencrypted traffic. Add the following tests:
* {n}tnlsw-ip4base-int ndrpdr tests
* {n}tnlsw-1atnl-ip4base-int reconf tests
* {n}tnlswasync-scheduler-ip4base-int ndrpdr tests
Where n is the number of tunnels:
1, 4, 40, 400, 1000, 5000, 10000, 20000, 40000, 60000 for the first two
1, 2, 4, 8 for the async scheduler tests
All of these with the following ecryption-auth algorithms:
aes128gcm
aes256gcm
aes128cbc-hmac256sha
aes128cbc-hmac512sha
Also add the corresponding trex profiles:
trex-stl-2n-ethip4-ip4dst{n}-udir.py
Where n is the number of tunnels listed above. The profiles are shared
among the tests.
Change-Id: I22bb46e6ad59801581a78aa19310bee8a5293e56
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
This patch is to add test suites for vpp plugin crypto_sw_scheduler,
IPsec sync mode is to do crypto and packet forward work in same worker cores,
crypto_sw_scheduler can schedule crypto work to other async crypto cores to
improve whole crypto processing capability.
This test suites configure fixed 1 rx queues per port, then measure IPsec
performance with 1, 2, 3 crypto cores.
This patchset include 1, 2, 4, 8 ipsec tunnels test cases.
+Vratko help to change to count total physical cores instead of previous only
count crypto cores in test cases.
Change-Id: I0e67182e3d13273890a23703d838101900e25126
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Signed-off-by: pmikus <pmikus@cisco.com>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Idcee623abbc5568fc6ca49113bc2ff8add179046
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I35b2f786105103f19907d74c4f6cc77d7cd3a5bb
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I874568aebbe50c38e634438eb1201ca737bd18ad
|
|
+ With ramp-up, without reset, with session verification.
+ Uses the same profile as pps tests.
+ Ramp up duration is not specified, as duration is computed.
+ Timeout tracking with automated ramp-up.
+ Correct computation of next duration.
+ Checking both early and late sessions.
+ No loss measurement also acts as a ramp-up.
+ Return ReceiveRateMeasurement from send_traffic_on_tg_internal,
as that is needed for detecting whether trial is ok as ramp up.
- Udp needs quite low ramp-up rate after recent regression.
- Max scale has higher rate (so failing) to avoid session timeouts.
+ Bump copyright year.
Change-Id: I50c928659cd5b985b490a2e5fb69c5cd790600b0
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Results if NDR search is in TPS (so unidir) since
https://gerrit.fd.io/r/c/csit/+/28208
The bug was causing all reconf tests report zero loss,
as they used half of NDR load during reconfiguration.
+ Small improvements to descriptions of other traffic keywords.
Change-Id: Ib9af5861cc09e698eb762feb3f0a019571d17962
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Replace pair of traffic profiles (2n and 3n) with single nodeless one.
+ Compared to incremental suites, randomized ones add IP6_RND tag.
Change-Id: I2f0dfc9e04bbcd0f88e95b92edf2da2c73faaab6
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Jira: CSIT-1768
Change-Id: I888ae1a5754fa07297d4cdf65c2be0e3e49d89a5
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
+ Rename traffic profiles to avoid mentioning number of nodes.
+ Improve 2n rnd suite documentation slightly.
Change-Id: I82d6fb6a99133163a58d56f2acf8a7b9568ee77c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Modify initialize L2BD ACL keywords to be usable for both 2-node and
3-node topologies and update testsuites accordingly.
Add the missing macip T-rex profile.
Add classifier tests to 2n-tx2 job specs.
Change-Id: I17b84b8fc18ef9a6f275ae0238a0665ac2017f01
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech>
Change-Id: Ifb97088a26f3c2ab2ecee28ed43e6e957a844c64
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Ibafaaca4438587284d1e255c764f0701002941e8
|
|
Disconnect+connect cycle is expensive and slow.
This change tracks connected client instances
so later "connect" to the same target uses it.
Explicit disconnects are allowed (and executed before VPP ends),
but once again disconnected instances are cached and reused,
as creating a new instance is more expensive than just connect.
+ Add missing checks on interfaces being up to appropriate keyword.
+ Use appropriate keywords.
+ Add a comment explaining why a simpler keyword is not appropriate.
+ Improve VPP checks in containers.
+ Fix the vppctl check to actually work.
+ Add PAPI check to ensure VPP is really ready.
+ Delay/reorder checks to make them faster with multiple containers.
+ Leave some TODOs to improve various lifecycles later.
+ As we do not stop VPP in test/suite teardown:
+ One final disconnect is needed, added to __init__.robot teardowns.
- Import of the final disconnect keyword is ugly, but it works.
- We could use a hashable class for distinguishing node+socket pairs.
- Are we connecting to VPP inside VMs?
Change-Id: I49cd726740c3e8cae1591c7c84b85a447241228f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Jira: CSIT-1597
+ add ipsec_sa tear down action
Change-Id: I4d1e6f26c14e61e8ddc6f29fbf5528a034c84eb0
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Ticket: CSIT-1771 VPP-1207 VPP-1675
Change-Id: I8ba2d62054361e72b833943327434ea071e7e568
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites.
+ Update existing cps traffic profiles.
+ Add missing traffic profiles.
+ UDP:
+ Single burst of 32 packets was confirmed as safe enough for TRex.
+ Maybe 64 could work, but not enough testing for that.
+ Multiple bursts have lead to reduced TRex performance,
as overlaping bursts (from different client instances)
tend to fill up the buffers.
+ TCP:
+ Data size set to 11111 bytes, completely arbitrarily.
+ Results look reasonable, so I have kept that.
- MSS not set at all
- No tested support for frame size other than 64B.
- Frame size does not even factor into TCP profiles.
+ So other frame sizes are skipped in autogen.
+ Update tags in related suites.
- HOSTS_{n} and SRC_USER_{n} should be unified.
- Questionable clarification on difference between IP4BASE and SCALE.
+ Add NAT state resetters to tests that need them.
+ Resetter is called (if set) before each measurement.
+ If ramp-up is detected, resetter is not set.
+ Rename "mult" argument to "multiplier".
+ Abstracted from packets to transactions.
+ Transaction corresponds to profile.
+ TRex multiplier argument sets target rate in transactions per second.
+ The familiar STL traffic:
+ Bidirectional is considered to be 2 packets per transaction.
+ Unidirectional is considered to be 1 packet per transaction.
+ The newer ASTF traffic:
+ 4 subtypes, each has different number of packets per transaction.
+ For max rate computation:
+ Packets in the more numerous direction are considered.
+ Rely on TRex reported traffic duration for ASTF:
+ Use the server side value.
- Client side value is higher by an overhead.
- TRex is not sending traffic during that time.
+ Remove delays from traffic profiles.
- Those delays would increase the reprted traffic time.
+ Support for scale lmited trials.
+ Only for ASTF profiles, each ASTF profile has limited scale.
+ Scale defined in suite variables.
+ For TRex to send all transactions provided duration value is ignored.
+ The appropriate value is computed in TrafficGenerator.
+ An ad-hoc time constant is added to match the TRex client side time overhead.
+ The profile driver receives the computed duration.
+ Measurement for PLRsearch add a sleep if the computed duration is smaller.
+ Alternative argument for search algos if scale is limited.
+ Both need higher timeout to accomodate big scales.
+ MLRsearch can afford fewer phases.
+ Added a parameter to optionally shorten the duration.
+ Use short duration for runtime stats trial and failure stats trial.
+ Use very large keepalive values in udp profiles to avoid ka packets.
+ No polling in ASTF profile driver.
- Polling could eliminate the time overhead value.
+ But polling proved to introduce some loss, affecting the results.
+ Handle duration stretching in ASTF by stopping traffic.
+ The stop has several steps so that:
+ The traffic is really stopped entirely.
+ Late packets do not count (maybe as errors).
+ Stats are preserved to read for results (and cleared afterwards).
+ Several quantities added to ReceiveRateMeasurement:
+ Original target duration is preserved (algos need that).
+ Input estimate (tps) for early search iterations.
+ Output estimate (maybe pps) for MRR output.
+ Strict result (unsent counts as loss) for NDR.
+ Use L2 counters (opackets, ipackets) where possible.
- TRex has trouble processing packets for the L7 ones at high loads.
+ Remove warmup from profile drivers and keywords.
+ Suites should call "Send ramp-up traffic" explicitly if needed.
+ Added parsing for few more counters.
+ Both to use in formulas or just for debug purposes.
- Only 64B cases in autogen, framesize support to be added later.
+ Latency streams during search can be enabled via PERF_USE_LATENCY env var.
+ MLRsearch improvments:
+ Rename argument names to min_rate and max_rate.
+ Use relative receive rate in initial phase.
+ PLRsearch improvements:
+ Careful computation when output (pps) does not match input (tps).
+ Use geometric distribution (instead of Poisson).
+ Helps agains math errors.
+ This should improve estimate stability.
- But in practice big losses still lead to significant jumps.
+ Traffic generator improvements:
+ send_traffic_on_tg now calls the full set_rate_provider_defaults.
+ _send_traffic_on_tg_internal for the logic without provider defaults.
+ As the internal function is re-used by measure() without affecting defaults.
+ Move _parse_traffic_results just before get_measurement_result.
+ As the latter uses fields set bu the former, it is now easier to read.
+ Multiple sources for approximate duration.
+ Tried from more precise to more available.
+ Includes logic for _pps tests (added in later change).
+ Move explicit type conversions to earlier occurences.
+ Profile driver output field uses semicolons to simplify parsing.
+ Performance Robot lib file split to several smaller ones.
+ performance_actions.robot:
+ Hosts Additional Statistics Action For * keywords.
+ performance_display.robot:
+ Hosts keyword for displaying and verifying results.
+ Change test message to use the correct unit (pps or cps).
+ performance_limits.robot renamed to performance_vars.robot
+ Added many keywords, mostly for accessing test variables.
+ Moved variables for Policer into a new keyword there.
+ Some keywords need sophisticated logic.
- Other are basically Get Variable Value.
+ But in future more logic can be added, without editing callers.
+ Documentation for the new keywords acts as a documentation for test variables.
+ performance_utils.robot has the rest.
+ Eliminated arguments if the value is in test variable.
+ Small improvements to documentation.
- Still not enough cleanup with respect to arguments and test variables.
+ Keywords are sorted alphabetically now in each one.
+ Suites:
+ Unified variables table:
+ No colons in comments.
+ ${n_hosts}, ${n_ports} and use them instead hardcoded numbers.
+ Add -cps to existing cps suite names.
+ Remove "trial data overwrite".
+ Compute max rate as in STL suites.
+ Each NAT suite has ip4base suite to compare results to.
- Those act as indirect TRex calibration.
- VPP does not lose packets in those.
+ Latency in ASTF suites is disabled hard.
- As we do not support latency in ASTF profiles yet.
+ Unidirectional tests governed by suite variable, not an argument.
+ Write long argument lists vertically.
+ Prefer to use argument names.
+ In Python, also the last argument is followed by comma.
+ It makes renaming and reordering easier.
+ Similarly applies to prints with long lists of values.
+ A TODO to update api crc file comments.
Change-Id: I84729355edbec051298a9de1162107f88ff5737d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/29463
- update vpp stable to version 21.01-rc0~283-g5f4f2081c
Change-Id: I079c10e4537448c3b078f22c3fe4ed266a5e2e2c
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- Specs
- Tests
- Library
- TODO: PAL
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I8890940d56c7a9924d4a5f9365b6bb881610db14
|
|
- after merge of commit vppinfra: use heap to store bihash data |
https://gerrit.fd.io/r/c/vpp/+/29099
it is necessary to allocate 14GB of memory for main heap in case
of nat44ed test with 16.5M sessions
Change-Id: I65dccdd4f50382b1b40cdffd2d9031ffb7d3bfa1
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I3aa50ec1ef9b0445014daa31e767323060f4a03f
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|