Age | Commit message (Collapse) | Author | Files | Lines |
|
Jira: CSIT-1768
Change-Id: I888ae1a5754fa07297d4cdf65c2be0e3e49d89a5
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit 6962831a1f1905e50a3d663c0cf3ce8c2873ecc9)
Change-Id: I6496ae773b176b2abfedab843110d6f870c62888
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech>
Change-Id: Ifb97088a26f3c2ab2ecee28ed43e6e957a844c64
(cherry picked from commit 9446b9bf9c15999feec3c8a48fec428bca094500)
|
|
Jira: CSIT-1769
Change-Id: I98510cd8e627d7347f77d0032b7bac28f2e36c61
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
This reverts commit a9f54ca5080aeef17686f300a6807bf9b46b7c90.
Reason for revert: DO NOT MERGE BROKEN TESTS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-device-master-ubuntu1804-1n-skx/9918/archives/log.html.gz
Change-Id: I2ce2970a43e5dd7487eeb54d1ccbb149e06cf8fa
Signed-off-by: pmikus <pmikus@cisco.com>
|
|
Jira: CSIT-1769
Change-Id: I0ab906966f760b809b8431530c010a8f0835d62c
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
+ Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites.
+ Update existing cps traffic profiles.
+ Add missing traffic profiles.
+ UDP:
+ Single burst of 32 packets was confirmed as safe enough for TRex.
+ Maybe 64 could work, but not enough testing for that.
+ Multiple bursts have lead to reduced TRex performance,
as overlaping bursts (from different client instances)
tend to fill up the buffers.
+ TCP:
+ Data size set to 11111 bytes, completely arbitrarily.
+ Results look reasonable, so I have kept that.
- MSS not set at all
- No tested support for frame size other than 64B.
- Frame size does not even factor into TCP profiles.
+ So other frame sizes are skipped in autogen.
+ Update tags in related suites.
- HOSTS_{n} and SRC_USER_{n} should be unified.
- Questionable clarification on difference between IP4BASE and SCALE.
+ Add NAT state resetters to tests that need them.
+ Resetter is called (if set) before each measurement.
+ If ramp-up is detected, resetter is not set.
+ Rename "mult" argument to "multiplier".
+ Abstracted from packets to transactions.
+ Transaction corresponds to profile.
+ TRex multiplier argument sets target rate in transactions per second.
+ The familiar STL traffic:
+ Bidirectional is considered to be 2 packets per transaction.
+ Unidirectional is considered to be 1 packet per transaction.
+ The newer ASTF traffic:
+ 4 subtypes, each has different number of packets per transaction.
+ For max rate computation:
+ Packets in the more numerous direction are considered.
+ Rely on TRex reported traffic duration for ASTF:
+ Use the server side value.
- Client side value is higher by an overhead.
- TRex is not sending traffic during that time.
+ Remove delays from traffic profiles.
- Those delays would increase the reprted traffic time.
+ Support for scale lmited trials.
+ Only for ASTF profiles, each ASTF profile has limited scale.
+ Scale defined in suite variables.
+ For TRex to send all transactions provided duration value is ignored.
+ The appropriate value is computed in TrafficGenerator.
+ An ad-hoc time constant is added to match the TRex client side time overhead.
+ The profile driver receives the computed duration.
+ Measurement for PLRsearch add a sleep if the computed duration is smaller.
+ Alternative argument for search algos if scale is limited.
+ Both need higher timeout to accomodate big scales.
+ MLRsearch can afford fewer phases.
+ Added a parameter to optionally shorten the duration.
+ Use short duration for runtime stats trial and failure stats trial.
+ Use very large keepalive values in udp profiles to avoid ka packets.
+ No polling in ASTF profile driver.
- Polling could eliminate the time overhead value.
+ But polling proved to introduce some loss, affecting the results.
+ Handle duration stretching in ASTF by stopping traffic.
+ The stop has several steps so that:
+ The traffic is really stopped entirely.
+ Late packets do not count (maybe as errors).
+ Stats are preserved to read for results (and cleared afterwards).
+ Several quantities added to ReceiveRateMeasurement:
+ Original target duration is preserved (algos need that).
+ Input estimate (tps) for early search iterations.
+ Output estimate (maybe pps) for MRR output.
+ Strict result (unsent counts as loss) for NDR.
+ Use L2 counters (opackets, ipackets) where possible.
- TRex has trouble processing packets for the L7 ones at high loads.
+ Remove warmup from profile drivers and keywords.
+ Suites should call "Send ramp-up traffic" explicitly if needed.
+ Added parsing for few more counters.
+ Both to use in formulas or just for debug purposes.
- Only 64B cases in autogen, framesize support to be added later.
+ Latency streams during search can be enabled via PERF_USE_LATENCY env var.
+ MLRsearch improvments:
+ Rename argument names to min_rate and max_rate.
+ Use relative receive rate in initial phase.
+ PLRsearch improvements:
+ Careful computation when output (pps) does not match input (tps).
+ Use geometric distribution (instead of Poisson).
+ Helps agains math errors.
+ This should improve estimate stability.
- But in practice big losses still lead to significant jumps.
+ Traffic generator improvements:
+ send_traffic_on_tg now calls the full set_rate_provider_defaults.
+ _send_traffic_on_tg_internal for the logic without provider defaults.
+ As the internal function is re-used by measure() without affecting defaults.
+ Move _parse_traffic_results just before get_measurement_result.
+ As the latter uses fields set bu the former, it is now easier to read.
+ Multiple sources for approximate duration.
+ Tried from more precise to more available.
+ Includes logic for _pps tests (added in later change).
+ Move explicit type conversions to earlier occurences.
+ Profile driver output field uses semicolons to simplify parsing.
+ Performance Robot lib file split to several smaller ones.
+ performance_actions.robot:
+ Hosts Additional Statistics Action For * keywords.
+ performance_display.robot:
+ Hosts keyword for displaying and verifying results.
+ Change test message to use the correct unit (pps or cps).
+ performance_limits.robot renamed to performance_vars.robot
+ Added many keywords, mostly for accessing test variables.
+ Moved variables for Policer into a new keyword there.
+ Some keywords need sophisticated logic.
- Other are basically Get Variable Value.
+ But in future more logic can be added, without editing callers.
+ Documentation for the new keywords acts as a documentation for test variables.
+ performance_utils.robot has the rest.
+ Eliminated arguments if the value is in test variable.
+ Small improvements to documentation.
- Still not enough cleanup with respect to arguments and test variables.
+ Keywords are sorted alphabetically now in each one.
+ Suites:
+ Unified variables table:
+ No colons in comments.
+ ${n_hosts}, ${n_ports} and use them instead hardcoded numbers.
+ Add -cps to existing cps suite names.
+ Remove "trial data overwrite".
+ Compute max rate as in STL suites.
+ Each NAT suite has ip4base suite to compare results to.
- Those act as indirect TRex calibration.
- VPP does not lose packets in those.
+ Latency in ASTF suites is disabled hard.
- As we do not support latency in ASTF profiles yet.
+ Unidirectional tests governed by suite variable, not an argument.
+ Write long argument lists vertically.
+ Prefer to use argument names.
+ In Python, also the last argument is followed by comma.
+ It makes renaming and reordering easier.
+ Similarly applies to prints with long lists of values.
+ A TODO to update api crc file comments.
Change-Id: I84729355edbec051298a9de1162107f88ff5737d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: I409b060f64ae7c6787448ae519fd76d8384e9ffb
Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I20e01dfe83a961dc8202d33783a678d38e71cff2
|
|
Change-Id: Ib30bc4697fcba93a6723ee492a59a0523425f623
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- Leftovers from kubernetes found, but not removed here.
Change-Id: If8cb9269d0f3e69f642d7fe02c59122e17925a4d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: Ibbfbed79e473c804390802ae1ecd737b50c06aa3
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I0c9b8b07ed3a113e24a38faaddd5ed06fae67768
|
|
Change-Id: I03603108fe7128fb4bde4abddb6a0959dd5b5c73
Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
|
|
+ This will help unify driver base differences between TCs.
+ Decrease amount of logic needed for suite generator.
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I1e84ba361dc1e829f0612c58a61096e2633ce0c5
|
|
Change-Id: I61555ba566efef0a2151db9a30bf7f5d9ccad1df
Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
|
|
Change-Id: Idd715cb0d3e6b60c660b95ac618e8e114419491b
Signed-off-by: Nidhy <nidhyanandhan.a@hcl.com>
|
|
Change-Id: I0fbe4d9ef1b30a1ac1382ea43b5b52aeb8a7d39f
Signed-off-by: Nidhy <nidhyanandhan.a@hcl.com>
|
|
Ticket: CSIT-1551
+ Config copied from other suites.
+ Layer keywords prepared for additional chains.
+ Renamed "create" argument to "vlan_per_chain".
+ TrafficGenerator result extraction improvements.
+ Also unified TG type and subtype checking.
+ Throughput estimated as PDR for zero PLR.
+ Actual Qemu start is skipped.
+ Parse xstats instead of stats in trex stop scipt.
- Because stats are unconditionally cleared on each script connect.
+ Remember start values to get better ReceiveMeasurementResult at stop.
+ Expose loss count and time as appropriate test message.
+ Autogen support.
+ Multiple chain amounts and core densities.
- Only 1 additional chain.
+ Proper TAGs. Documented.
+ Clarified min framesize tags are for TG-DUT.
+ Added a TODO concerning overhead values.
+ Added 118B tag.
+ Robot performance keywords improvements.
+ Added missing return value documentation.
+ Frame size as required test variable, not an explicit argument.
+ MLRsearch keyword to store result (not into test message).
- TODO: De-duplicate: main keyword to measure, minor to set test message.
- But implementation uses the fact default arguments are different.
+ Keywords for starting and stopping background traffic.
- No checks whether start-stop pair is complete.
+ Multiple minor indentation and logging improvements.
- Missing / TODOs:
- Support also vlan_per_chain=True?
- Support dot1q without vxlan?
- Support also deletion?
- How to package config steps so -reconf can be generated from -ndrpdr?
Change-Id: If0270697f9ffa837c34ef802917220a53f156703
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- 5K
- 10K
Change-Id: Iaa2c77d07c3686e09fe327c663d7d7fc42024034
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- Add:
aes-128-gcm aes-128-gcm
aes-256-gcm aes-256-gcm
aes-128-cbc hmac-sha-256
aes-256-cbc hmac-sha-512
- Remove:
hmac-sha1
- Scale will follow in next patch
Change-Id: I789f71cf66cf61b8dbb3c6dbe9b6fdc79866ac33
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: Icff556142280ad0b6261e0a2bfb71672ee6b3807
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ All perf suites updated, as autogen change is backward incompatible.
+ Only x710 -ndrpdr suites remain in git repository.
+ Removed retry feature of run_tests.
+ Renamed topology_type and traffic_type to traffic_profile.
+ Renamed framesize to frame_size; nic_model to nic_name.
+ Reordered Variables table to start with nic_name and overhead.
+ Fixed wrong overhead value in some cbc-sha1 suites.
+ Fixed some suite tags.
++ Introduced tags to distinguish suites, such as IPSECINT.
+ Bound crypto hardware to NIC name.
+ Implemented NIC limit hiding.
+ Also search and teardown related arguments are now hidden.
+ Main measurement keywords updated to new arg handling.
+ Max rate related keywords moved to performance_limits.robot
+ Teardowns unified.
+ Generated tests are archived.
++ Generated directory is .gitignore-d.
+ Regenerator raises an exception on seeing non-compatible suite.
++ Relatively helpful message should be seen in exception.
+ Suite and template Documentation has generated parts.
+- With short NIC names only.
+ Autogen checker also upgraded.
+ Tag expressions replaced with long files to ensure analogous tests.
Change-Id: I60e9a999187e7da1f60d0eb4fb02afa14682aa46
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
* rename NICs in Taishan testbed to Intel-X520-DA2
* fix testbed NIC selection
* remove unnecessary 82599es tests
* blacklist unsupported tests on Taishan testbed
* lowercase previously created bash local variables
Change-Id: I3b7f59e2edc00929fab2f5a0fce483f236f83075
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
|
|
Change-Id: Iaa0e7fbca7de26c7dce8b389cc3494f628a2ecd0
Signed-off-by: Stanislav Chlebec <stanislav.chlebec@pantheon.tech>
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
|
|
Change-Id: I6a045225b0ff3cc4cd9880ee05861fd7bf36c61a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
Change-Id: Iece74d5e7e380db344005a6035252bbe18925646
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
|
|
Change-Id: I36ab0bc7c9eada4c3f7bb5cb8f53de7d267ff910
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: Iad77972ab6a3f4eb4c2e3c09aebfcce0fc5f999b
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: I23cedb3ad0f188e9c7570f549e397f4c42404fab
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
Jira: CSIT-1273
Change-Id: Ic2a41661c158384a5aaa7d4e73f30ffca13ddf82
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: Ia74a87346fe2c0a170b881b510f1b0558a538f3c
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Jira: CSIT-1247
Change-Id: Ia6a3f7682eb9a35040682913c0e33a1d61d1e6fa
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: Ib57014bf2461518eadcd156b16f8ee26eeb86785
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
Due to automatization of SMT detection this change is suppose to
remove static thread/core tags in favor of dynamic one. Leaving the
static tags for number of physical cores to be able to select TCs.
Change-Id: I7f99f605821f363e45c333f46d1dea786693521b
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Test case definitions
Change-Id: I03736ef003d3839859dc450745323362b21aa75e
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
CSIT-1072
CSIT-1083
CSIT-1084
Change-Id: Icaf0e09af633d301306a84251f47822e95b411a2
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
CSIT-1073
CSIT-1074
CSIT-1077
Change-Id: I248e079021c30988210fd792b32d5afdca71aadb
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
+ Add 10ge2p1x520-ethip4-ip4base-ndrpdr.robot
+ Add 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-ndrpdr.robot
+ Make sure both IMIX and 9000B test cases are there.
+ Document NDRPDR test tag.
+ Execute some of NDRPDR tests daily.
Change-Id: I66427d49cb6b8304fde4e8d91606de33b624186a
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- SRv6 with endpoint to SR-unaware Service Function via static proxy
- SRv6 with endpoint to SR-unaware Service Function via dynamic proxy
- SRv6 with endpoint to SR-unaware Service Function via masquerading proxy
Change-Id: I6a6062cb41d810bf9e27dacfd866181bfde4c693
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- Implementation of Display raw results
- Implementation of Traffic should pass with maximum rate
- Code optimizations
- L2BD, L2XC, IP4, IP6 TC
Change-Id: I0b50e995a853ed3c8f88323ccc650e47b8ed2496
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I42efd42ec577723ef2b17a1753ae744986bb66ff
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- CSIT-867: Low Level Description
- CSIT-868: wrk traffic profile - parsing
- CSIT-869: wrk implementation into CSIT
Change-Id: I65e1037f5ae05b3a5b2020e4a6c54462766ae1b4
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
NSH is optional HC component so possibility to exclude it
(e.g. for local testing) might be usefull.
Change-Id: Ib2496009f8839364e50d255fbe231f3addc45bc4
Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
|
|
Change-Id: I84f6da2bede47e7ae35e6b4911d425d2402dbd1b
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- Based on https://gerrit.fd.io/r/#/c/8146/
Change-Id: I774483c7e2b419a9b9b6be78812ecf53a5c91eab
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Follow the good practices for formatting. Update the content of release
report RST files to use the proper directives like :cite:, :footnote:,
:abbr:, :captions: etc.With these markings we will be able to format
output in proper way and style it in various sphinx output builders.
Change-Id: Ibbb538d43c3bd364a6acdcc990097a477f49de00
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I3d8cbb235918331b3c60147f7870a4aa90036748
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I51f6015681d1fc62da56cf1cca110f7a4c82ba10
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Change-Id: I949f8800b45354c205c5a5d36ab2abeeb93409dc
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Update the mechanics of script for generating report to reflect the latest changes. Update some static content.
Change-Id: If60f7b74245138cc3044a49c46ff06d5850732de
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|