Age | Commit message (Collapse) | Author | Files | Lines |
|
+ Migrate the call to main python util
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I8d00165071d4764121740c1c77819b8ffc65b3ba
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I6826add7b3032041632c3952c45a3c64409400b0
|
|
- enabling for fortville, columbiaville
- enabling experimental for mlx
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I1b7ceb54769f4a0089ac7309350499e60c5cca0a
|
|
Change-Id: I0c973ceffc13f4f3c5f959d56cb0890b87091a82
Signed-off-by: Viliam Luc <viliam.luc@pantheon.tech>
|
|
+ Extract the stats keyword into a library.
Change-Id: Icfaaa32c2a97d479ba0339acf34b865006949a89
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
This patch is to add test suites for vpp plugin crypto_sw_scheduler,
IPsec sync mode is to do crypto and packet forward work in same worker cores,
crypto_sw_scheduler can schedule crypto work to other async crypto cores to
improve whole crypto processing capability.
This test suites configure fixed 1 rx queues per port, then measure IPsec
performance with 1, 2, 3 crypto cores.
This patchset include 1, 2, 4, 8 ipsec tunnels test cases.
+Vratko help to change to count total physical cores instead of previous only
count crypto cores in test cases.
Change-Id: I0e67182e3d13273890a23703d838101900e25126
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Signed-off-by: pmikus <pmikus@cisco.com>
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I874568aebbe50c38e634438eb1201ca737bd18ad
|
|
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech>
Change-Id: Ifb97088a26f3c2ab2ecee28ed43e6e957a844c64
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Ibafaaca4438587284d1e255c764f0701002941e8
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I46750f8571d86dc13df95252e8894c4b988efd5c
|
|
+ Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites.
+ Update existing cps traffic profiles.
+ Add missing traffic profiles.
+ UDP:
+ Single burst of 32 packets was confirmed as safe enough for TRex.
+ Maybe 64 could work, but not enough testing for that.
+ Multiple bursts have lead to reduced TRex performance,
as overlaping bursts (from different client instances)
tend to fill up the buffers.
+ TCP:
+ Data size set to 11111 bytes, completely arbitrarily.
+ Results look reasonable, so I have kept that.
- MSS not set at all
- No tested support for frame size other than 64B.
- Frame size does not even factor into TCP profiles.
+ So other frame sizes are skipped in autogen.
+ Update tags in related suites.
- HOSTS_{n} and SRC_USER_{n} should be unified.
- Questionable clarification on difference between IP4BASE and SCALE.
+ Add NAT state resetters to tests that need them.
+ Resetter is called (if set) before each measurement.
+ If ramp-up is detected, resetter is not set.
+ Rename "mult" argument to "multiplier".
+ Abstracted from packets to transactions.
+ Transaction corresponds to profile.
+ TRex multiplier argument sets target rate in transactions per second.
+ The familiar STL traffic:
+ Bidirectional is considered to be 2 packets per transaction.
+ Unidirectional is considered to be 1 packet per transaction.
+ The newer ASTF traffic:
+ 4 subtypes, each has different number of packets per transaction.
+ For max rate computation:
+ Packets in the more numerous direction are considered.
+ Rely on TRex reported traffic duration for ASTF:
+ Use the server side value.
- Client side value is higher by an overhead.
- TRex is not sending traffic during that time.
+ Remove delays from traffic profiles.
- Those delays would increase the reprted traffic time.
+ Support for scale lmited trials.
+ Only for ASTF profiles, each ASTF profile has limited scale.
+ Scale defined in suite variables.
+ For TRex to send all transactions provided duration value is ignored.
+ The appropriate value is computed in TrafficGenerator.
+ An ad-hoc time constant is added to match the TRex client side time overhead.
+ The profile driver receives the computed duration.
+ Measurement for PLRsearch add a sleep if the computed duration is smaller.
+ Alternative argument for search algos if scale is limited.
+ Both need higher timeout to accomodate big scales.
+ MLRsearch can afford fewer phases.
+ Added a parameter to optionally shorten the duration.
+ Use short duration for runtime stats trial and failure stats trial.
+ Use very large keepalive values in udp profiles to avoid ka packets.
+ No polling in ASTF profile driver.
- Polling could eliminate the time overhead value.
+ But polling proved to introduce some loss, affecting the results.
+ Handle duration stretching in ASTF by stopping traffic.
+ The stop has several steps so that:
+ The traffic is really stopped entirely.
+ Late packets do not count (maybe as errors).
+ Stats are preserved to read for results (and cleared afterwards).
+ Several quantities added to ReceiveRateMeasurement:
+ Original target duration is preserved (algos need that).
+ Input estimate (tps) for early search iterations.
+ Output estimate (maybe pps) for MRR output.
+ Strict result (unsent counts as loss) for NDR.
+ Use L2 counters (opackets, ipackets) where possible.
- TRex has trouble processing packets for the L7 ones at high loads.
+ Remove warmup from profile drivers and keywords.
+ Suites should call "Send ramp-up traffic" explicitly if needed.
+ Added parsing for few more counters.
+ Both to use in formulas or just for debug purposes.
- Only 64B cases in autogen, framesize support to be added later.
+ Latency streams during search can be enabled via PERF_USE_LATENCY env var.
+ MLRsearch improvments:
+ Rename argument names to min_rate and max_rate.
+ Use relative receive rate in initial phase.
+ PLRsearch improvements:
+ Careful computation when output (pps) does not match input (tps).
+ Use geometric distribution (instead of Poisson).
+ Helps agains math errors.
+ This should improve estimate stability.
- But in practice big losses still lead to significant jumps.
+ Traffic generator improvements:
+ send_traffic_on_tg now calls the full set_rate_provider_defaults.
+ _send_traffic_on_tg_internal for the logic without provider defaults.
+ As the internal function is re-used by measure() without affecting defaults.
+ Move _parse_traffic_results just before get_measurement_result.
+ As the latter uses fields set bu the former, it is now easier to read.
+ Multiple sources for approximate duration.
+ Tried from more precise to more available.
+ Includes logic for _pps tests (added in later change).
+ Move explicit type conversions to earlier occurences.
+ Profile driver output field uses semicolons to simplify parsing.
+ Performance Robot lib file split to several smaller ones.
+ performance_actions.robot:
+ Hosts Additional Statistics Action For * keywords.
+ performance_display.robot:
+ Hosts keyword for displaying and verifying results.
+ Change test message to use the correct unit (pps or cps).
+ performance_limits.robot renamed to performance_vars.robot
+ Added many keywords, mostly for accessing test variables.
+ Moved variables for Policer into a new keyword there.
+ Some keywords need sophisticated logic.
- Other are basically Get Variable Value.
+ But in future more logic can be added, without editing callers.
+ Documentation for the new keywords acts as a documentation for test variables.
+ performance_utils.robot has the rest.
+ Eliminated arguments if the value is in test variable.
+ Small improvements to documentation.
- Still not enough cleanup with respect to arguments and test variables.
+ Keywords are sorted alphabetically now in each one.
+ Suites:
+ Unified variables table:
+ No colons in comments.
+ ${n_hosts}, ${n_ports} and use them instead hardcoded numbers.
+ Add -cps to existing cps suite names.
+ Remove "trial data overwrite".
+ Compute max rate as in STL suites.
+ Each NAT suite has ip4base suite to compare results to.
- Those act as indirect TRex calibration.
- VPP does not lose packets in those.
+ Latency in ASTF suites is disabled hard.
- As we do not support latency in ASTF profiles yet.
+ Unidirectional tests governed by suite variable, not an argument.
+ Write long argument lists vertically.
+ Prefer to use argument names.
+ In Python, also the last argument is followed by comma.
+ It makes renaming and reordering easier.
+ Similarly applies to prints with long lists of values.
+ A TODO to update api crc file comments.
Change-Id: I84729355edbec051298a9de1162107f88ff5737d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: I409b060f64ae7c6787448ae519fd76d8384e9ffb
Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
|
|
- Specs
- Tests
- Library
- TODO: PAL
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I8890940d56c7a9924d4a5f9365b6bb881610db14
|
|
- default value for main heap size set to 2G
- allow allocation of n x 2G in tests that need it
Change-Id: I02dafe6bc61649d2823486a3e983f66f5a213a88
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- some tests need to reduce rate for ramp-up phase
- some tests need to extend trail duration in ramp-up phase
- removed 2n1l-10ge2p1x710-ethip4udp-nat44det-h1-p63-s63 suite
as nat out ports are randomly selected from available port range
so T-Rex stateless is not able to provide required out2in traffic
Change-Id: I1145496610d202f81d911e68aa819844d7600918
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
- https://gerrit.fd.io/r/c/vpp/+/28829
- https://gerrit.fd.io/r/c/vpp/+/28896
- ip6 to be added later
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I7b03b8ace5c5c7582d4a2f66bfd788a3ec80a916
|
|
+ Unifying code structures
- To easily plug another DUT
+ New PCI PassThrough templates
+ Improved perf stat on cores allocated in test.
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I325f17b977314f93cb91818feddfddf3e607eb8a
|
|
- continuation of https://gerrit.fd.io/r/c/csit/+/26898 as there was
reached limit of changes (1000)
Jira: CSIT-1711
- udp synthetic profiles w/o data packets
- udp cps perf tests, phase I (no special "search cps" KW)
Part I means that we are using MRR tests to collect traffic data
until there is ready new CPS test type with corresponding algorithm.
Change-Id: I0d30feb9ecf1d0bff937152656f8eb422f831378
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
for example, below will trigger VPP with default code path test in CSIT,
"csit-2n-clx-perftest mrrAND64bANDnic_intel-xxv710ANDeth-l2bdbasemaclrnNOTdrv_avf"
and below will trigger VPP with avx512 code path test in CSIT,
"csit-2n-clx-perftest icl mrrAND64bANDnic_intel-xxv710ANDeth-l2bdbasemaclrnNOTdrv_avf"
Change-Id: Id3abf932203ce00f34f8643c5bc4d5fc4d3e6efb
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Iacd7aa49d9832233033fe18fc228ccd45c12b4db
|
|
+ FIX for Vratko's FIX
Change-Id: Ia6ff137120d84bdd2b9e02e3e935c62d1c732b7b
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
On 3n-hsw, xl-710 is numa 1.
The old code returned 0, causing performance regression.
Fixes: 14a71b74b414df7616ccb5ada3d50ecb90d96bae
Change-Id: I81d3f711b6a37438f6411e29baafae9dec1c00a4
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Merge single/double link
+ Introduce _pf{n}[0] variables so we can access physical function
same way as virtual function
+ Cleanup code by moving complex logic to python
+ Prepare code for multiple vf functions
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ic2e74a38bfa146441357de8f0916aeb638941c49
|
|
Change-Id: I53425f57fe9ecef9cff2c94642cc7cb24537a961
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
|
|
- TrafficProfile
- IPsec jumbo
- TDLK leftover
- Trex scapy server
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I0601ea75ed6e7318c46124fb02e9c61066089791
|
|
- DUT only topology (hoststack test
apps are co-located with vpp)
- Make vpp app specific keywords generic
where applicable
- Add IP4 Prefix to topology file
- Support running wrk in linux namespace
- Refactor namespace cleanup
- Remove redundant namespace creation code
- Refactor test/keyword dirs: tcp -> hoststack
- Add hoststack utility keywords
- Refactor wrk suite setup/teardown
- Update tests with recent perf infra changes
Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
|
|
Change-Id: Ie64d662e81879bd52785e0188450d998bf056bda
Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I2b1672412e19f15cb46246fffa931351cd3d518b
|
|
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I0c9b8b07ed3a113e24a38faaddd5ed06fae67768
|
|
+ This will help unify driver base differences between TCs.
+ Decrease amount of logic needed for suite generator.
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I1e84ba361dc1e829f0612c58a61096e2633ce0c5
|
|
Change-Id: I61555ba566efef0a2151db9a30bf7f5d9ccad1df
Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
|
|
Change-Id: I6c7eff8d9cbb196ccb684a45579ac59942351cfe
Signed-off-by: premkumarv <preym17@gmail.com>
|
|
Change-Id: I0fbe4d9ef1b30a1ac1382ea43b5b52aeb8a7d39f
Signed-off-by: Nidhy <nidhyanandhan.a@hcl.com>
|
|
+ Ability to get stats from CNF via SocketPAPI
- Remove obsolete functions
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I4d1b32a7279244592be96644e4f8a530c4f29a15
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I1497a9b031603d09d69a29ed7167686848d4b7c5
|
|
- Remove translate TC as they have no perf opposite
- Remove unused keywords
- Merge similar keywords / rename
- Move keywords from perf library to categories
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I6433006136e2497ad8fb17193aa20c578d836999
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I3721900b91a9c8e9db8766bdcb9771b6bcf95bca
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ideddcea56120cf770aec78ca24813baf7d05ba11
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I73878542672ab2dcfa1db046fbca4b7d931e744c
|
|
Change-Id: I0050f715011ecfa92b3ee88b301809a56abb7946
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I8c8ae1d7eed8cf28fc63ce42b2dc36dd16a5c0da
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Ticket: CSIT-1541
Ticket: VPP-1722
Ticket: CSIT-1546
+ Increase timeout to hide x520 slownes of show hardware detail.
- Install sshpass and update ssh client in virl bootstrap.
+ Added TODOs to remove when CSIT-1546 is fixed.
+ Enable default socksvr on any startup conf.
+ Improve OptionString init and repr.
- The non-socket executor still kept for stats.
+ Remove everything unrelated to stats from non-socket executor.
- Remove some debug-loooking calls to avoid failures.
TODO: Introduce proper parsing to the affected keywords.
+ Reduce logging from PAPI code to level INFO.
- Needs https://gerrit.fd.io/r/20660 to fully work.
+ Change default values for LocalExecution.run()
+ Return code check enabled by default.
Code is more readable when rc!=0 is allowed explicitly,
and the test code will now detect unexpected failures.
+ Logging disabled by default.
Output XML is large already. Important logging can be enabled explicitly.
+ Restore alphabetical order in common.sh functions.
Change-Id: I05882cb6b620ad14638f7404b5ad38c7a5de9e6c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
* replace the current VM image with kernel img
* rework keyword usage to make it consistent with performance tests
* remove resources/libraries/robot/shared/qemu.robot as it's not used
anywhere anymore
Change-Id: Ia5bc19e9e6ed9af031e4d9b5c0c89431fb49fd33
Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
|
|
- Alignment with PERF
Change-Id: Id7b2cb91c88e44cf9265c3376330f6aa579abf7a
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I233c8908c724556db09c60de6efa9237068fb4c3
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: Ibd3e99390bf004173d3d126946b95ffe9ba8f196
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ Phase II - aligning FUNC/PERF setup.
+ 2n/3n compatibility of suite setup.
+ Same structure of testcase for DEV/PERF.
+ DEVICE tests are now running VPP startup config in same
way as PERF.
+ DEVICE is using COREKEEPER!
- DEVICE tests are running topology compute twice for backward
compatibility. TODO to remove.
- Still phase III needed to align TAGS, LIBRARIES and remove OLD
setup phase.
Change-Id: Ib6e1b0ebf4abebf3cbe73b4fda5cac953c2a7270
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ Phase I - aligning FUNC/PERF
Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I5954fdd61ead9b4143f8cc4e20294cf8e199e0af
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|