Age | Commit message (Collapse) | Author | Files | Lines |
|
Since https://gerrit.fd.io/r/c/csit/+/24056
csit-vpp perf tests are failing.
Turns out the reason is too large limit_memory value,
but other improvements applied during investigation
are included as well.
+ Cat trex.log if trex fails to start.
+ Improve trex startup command formatting (remove double spaces).
+ Reorder imports.
Change-Id: I2c1106ea6f88a1a275682e73eba212d08c7947c8
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ia9832e5e65dbcf2d4f99c52fd0570d456262ab9a
|
|
+ When possible, fix the violation.
+ Else, add a comment:
+ An explanation (if not already present) and keep disable.
+ A TODO (if not already present) and remove the disable.
- This makes tox job report more pylint violations,
but any such violation is fixable and should be fixed.
- Although some need to be fixed in VPP, such as enum item long names.
Change-Id: I48604b5eda070083d79dff1439620dbd9e798e1f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
+ Ansible
+ Service for auto load during restart
+ By default created 2 instances of Docker container
+ Small modification of Trex to use prefixes
+ DNV topology to use ssh port to connect to docker
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ibc2b2f0e509d920fc3e05d012b88abc388f5c3e7
|
|
HRDhistogram output contains essential trailing equals.
Edit all fields even though we do not expect equals in others.
Change-Id: I9f4f07fc4d6f4583989a9c0e45f68bc0cd9a65fe
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Add functions for creating rdma interface
+ Fix traffic generator
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I81787b72ff5ee926ed652d350888c4f86da766f1
|
|
+ Enable hdrh in trex server.
+ Append hdrh coded output after min/avg/max/.
+ Read (not show nor decode) hdrh value in PAL.
+ Also, remove old ndrpdrdisc code.
Change-Id: I99d99f10386a621772b5419ca1f36080fa15aca7
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
TRex does not zero the server counters.
It copies the values to use as reference,
and subtracts them when asked for results.
But the reference is stored in the client (not the server).
And CSIT uses different scripts to start and stop async traffic,
which means different clients.
This patch introduces a workaround.
Async start will return xstats objects to use as reference,
and async stop will use the objects to compute the correct results.
The xstats objects are stored in TrafficGenerator instance.
Sync measurement does not export the counters, to shorten logs.
Other improvements:
+ Make stop_traffic_on_tg return measurement results directly.
+ Rename --async to --async_start as "async" is reserved in Python 3.7
+ Minor pylint, docstring and typo fixes.
Change-Id: I5fc56a0763afb7d62cfa7c0651f96b6867de3e15
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: I0c2bee5a1ff8ad29e47c4f947388e7b97dd17fff
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Rename bool unidirection to int traffic_directions.
+ Rename "untagged" to "initial" for bandwidth calculation.
+ Fix latency measurement for unidirectional traffic.
+ Remove duplicate colon in soak test message.
+ Edit PAL to accept both forms.
+ Fix minor documentation issues.
Change-Id: I6c76f2dc090ae493f2fbd7e9ccd45229d2306dea
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- HDRhistogram can be rebased and used in separate patch.
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ic7ce752344c6e1791af7cf7b7effde65ac2be680
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I5ab8fe05074a842301a511bbd3ab5e67f8d1a9f0
|
|
This is a partial fix for:
Ticket: CSIT-1434
and should fix one pylint violation.
Many files to remove SSH() from remain.
+ Use -E (together with previous -S) in ssh.py when sudo=True.
Needed in some cases, no downsides expected.
Change-Id: Ie2ad03b3c656a4d3965a61a1e5435276f99a28c0
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Ticket: CSIT-1551
+ Config copied from other suites.
+ Layer keywords prepared for additional chains.
+ Renamed "create" argument to "vlan_per_chain".
+ TrafficGenerator result extraction improvements.
+ Also unified TG type and subtype checking.
+ Throughput estimated as PDR for zero PLR.
+ Actual Qemu start is skipped.
+ Parse xstats instead of stats in trex stop scipt.
- Because stats are unconditionally cleared on each script connect.
+ Remember start values to get better ReceiveMeasurementResult at stop.
+ Expose loss count and time as appropriate test message.
+ Autogen support.
+ Multiple chain amounts and core densities.
- Only 1 additional chain.
+ Proper TAGs. Documented.
+ Clarified min framesize tags are for TG-DUT.
+ Added a TODO concerning overhead values.
+ Added 118B tag.
+ Robot performance keywords improvements.
+ Added missing return value documentation.
+ Frame size as required test variable, not an explicit argument.
+ MLRsearch keyword to store result (not into test message).
- TODO: De-duplicate: main keyword to measure, minor to set test message.
- But implementation uses the fact default arguments are different.
+ Keywords for starting and stopping background traffic.
- No checks whether start-stop pair is complete.
+ Multiple minor indentation and logging improvements.
- Missing / TODOs:
- Support also vlan_per_chain=True?
- Support dot1q without vxlan?
- Support also deletion?
- How to package config steps so -reconf can be generated from -ndrpdr?
Change-Id: If0270697f9ffa837c34ef802917220a53f156703
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- force a python2 environment in tox for pylint
- let pylint load some compiled modules for member checking
- allow locally enabled pylint rules
- ignore bad enum names
- ignore complexity for code to be refactored by other tickets
- broad excepts are acceptable sometimes
Signed-off-by: Miroslav Los <miroslav.los@pantheon.tech>
Change-Id: If4c3a1b85e8609f074e1de8ce91fc9c4fedbeb38
|
|
Change-Id: I71010c18210e88828ded0bc679997d76f34886cb
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I2c0877f4d8b6cad2694c43a4350998cd2b830680
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Extract several tracker classes and use them in Integrator.
+ Apply next_rate workarounds to focus more on critical region.
+ Rewrite stretch function, as the previous implementation was wrong.
+ Rework logging:
++ Use injected trace() function in Integrator.
++ Inject function that skips trace logging on default initialization.
++ Use the same multiprocessing-safe passing, but separate queue.
+ Set duration to 120m in Robot, but keep at 30m in Python.
+ Apply minor changes to make computations more reproducible:
++ Attempt to log everything needed by reproducibility in one line.
++ Log samples used, and make it settable as upper limit.
++ Use repr output in TRex scripts, to avoid rounding when copypasting.
+- Numpy seems to be sharing PRNG between processes.
+-- An edit to disable one thread is needed for full reproducibility.
+-- Such an edit is prepared, but will not be merged anytime soon.
Change-Id: Icf56429b34ab6de2d18a12e6c676e2d242b3ba4c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
The two types were not well distinguished before.
Error introduced in:
https://gerrit.fd.io/r/#/c/17811/84/resources/libraries/robot/performance/performance_setup.robot@255
Error hotfixed in:
https://gerrit.fd.io/r/#/c/18847/4/resources/libraries/robot/shared/default.robot@109
+ Rename some arguments and improve method docstrings.
Newly introduced argument name osi_layer should be dissimilar enough.
Change-Id: Ie0f6f97dc010fc6477f09c54574970f1d15462e2
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ All perf suites updated, as autogen change is backward incompatible.
+ Only x710 -ndrpdr suites remain in git repository.
+ Removed retry feature of run_tests.
+ Renamed topology_type and traffic_type to traffic_profile.
+ Renamed framesize to frame_size; nic_model to nic_name.
+ Reordered Variables table to start with nic_name and overhead.
+ Fixed wrong overhead value in some cbc-sha1 suites.
+ Fixed some suite tags.
++ Introduced tags to distinguish suites, such as IPSECINT.
+ Bound crypto hardware to NIC name.
+ Implemented NIC limit hiding.
+ Also search and teardown related arguments are now hidden.
+ Main measurement keywords updated to new arg handling.
+ Max rate related keywords moved to performance_limits.robot
+ Teardowns unified.
+ Generated tests are archived.
++ Generated directory is .gitignore-d.
+ Regenerator raises an exception on seeing non-compatible suite.
++ Relatively helpful message should be seen in exception.
+ Suite and template Documentation has generated parts.
+- With short NIC names only.
+ Autogen checker also upgraded.
+ Tag expressions replaced with long files to ensure analogous tests.
Change-Id: I60e9a999187e7da1f60d0eb4fb02afa14682aa46
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Motivation: Make the constants available also for Robot.
Restraint: Robotframework user guide says:
Because variable files are always imported using a file system path,
creating them as classes has some restrictions:
Python classes must have the same name as the module they are located.
Change-Id: I638ef3fe045db132e366de2e2699638b8637e45e
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- Latency is now reported in both directions.
-- For unidirection that means additional -1/-1/-1.
+ Changed function argument wrapping to save vertical space.
+ Changed trex command synthesys to save vertical space.
+ Fixed handling of command line arguments to avoid superfluous spaces.
+ More care distinguishing (0, 1) from (port_0, port_1).
- Not renaming the original port arguments.
Change-Id: I7b1736b594f6edea52eb07e5bc76ef52d0af92f8
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
The original unidirection Chhange [0] has changed the order,
breaking some call sites.
Also, use different indentation to save some vertical space.
[0] https://gerrit.fd.io/r/#/c/16696/3/resources/libraries/python/TrafficGenerator.py@551
Change-Id: Ife9bd994da73f0afcd24fc16794f822a831c4353
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Missing bits:
- Add up-to-date .rst document (in child Change).
- Prepare for releasing to PyPI.org (in child Change):
-- Either copy dependencies from MLRsearch, or list in requirements.
-- Perhaps move common dependencies to separate package
for both search to depend on.
-- All the other metadata stuff.
Jira: CSIT-1276
Change-Id: I277efdb63dbb90b30e11f4e30a82e2130ac8efc3
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Extend TG in CSIT to support to send unidirectional traffic, and then
it can be used by unidirectional test cases e.g. Load balancer.
Change-Id: Ie25bcdf7d4525a6d88d8ecd11a1b6d98275fa4d6
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
|
|
- Decrease pylint errors. Part: easy
Change-Id: I452e5b5a11e9b78c03cd173a3848babe21b93c73
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Make number of doublings configurable,
keep Python default at 1, set Robot default to 2.
Also make docstring types unique
(pylint was complaining about classes and modules having the same name).
Increase MLRsearch version to 0.2.0.
Change-Id: Ib846032e79ff52994503c0cfef2f86655502c275
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Delete the corresponding ndrpdrdisc suites.
MRR suites are also edited,
to make them less different from NDRPDR suites.
Now, there is a script to re-generate testcases of selected suites.
For that, resources/libraries/python/autogen is created,
with Regenerator holding the most of script logic.
The script looks at files matching a glob expression,
reuses a "prolog", but overwites test cases with the default template
and hardcoded set of testcase arguments.
Also, handling of overhead size has been simplified,
using new keyword Get Max Rate And Jumbo And Handle Multi Seg.
Default value for search timeout is increased to 12 minutes
to make timeout failures less frequent.
Change-Id: I394ad61f262b3078e5ca719034b15ada48987ee0
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Rename search to MLRsearch in resources/libraries/python/.
+ Make PyPI/MLRsearch/MLRsearch a symlink to the renamed directory.
+ TrafficGenerator.py to import from the renamed directory/module.
- No change to MLRsearch code.
- Version stays at 0.1.1 to point to the latest released code.
Change-Id: I4a38b0b7f8f083fbb1a93b25522560144b4ef556
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
This reverts commit 0c2c57d9965ab1db0d5404ad103871ac250339cb.
Reason for revert: unless you will fix this broken patch we have to revert this as it breaks several thing in the csit core ci/cd.
1) all tb are pypi offline this means no new pypi is automatically dowloaded. Everytime you change your version you have to update every tb. As this is not automated this will never work and put extreme overhead
2) this code is not pylint checked and not verified by any existing job breaking any kind of verificition mechanics in csit
3)there is no automation in terms of creating pypi (no job) this creates chicken egg problem that you cannot verify change unless new version is installed
Next time please think or consult with more seniored csit core developer
Unless above will be fixed i will revert on monday as now it breaks all jobs. I will set hard verification to framework setup (exception) so this will prevent next time merge.
Possible solution:
1)Move the directory back to resources and reference from csit directly not via pypi, leave the setup crap in dir and let users to create package on their own
Change-Id: Id530a9f19f668e45f73ffc9e7371bb453b59fd1b
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ Delete resources/libraries/python/search/.
+ Add MLRsearch==0.1.1 to requirements.txt.
+ Bump PyPI/MLRsearch version to 0.1.2.
Change-Id: I1f86d75ac8eb90ee3946dbac8c56280f39302f4d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Use first intermediate with goal in initial phase.
+ Measure above MRR if that got zero loss.
+ Always prioritizes NDR in internal search.
+ Rename classes.
+ Copy code for standalone PyPI publishing.
- Original files will be deleted after publish.
Change-Id: I5169d602d1e5e35a1894645cd52e70d791871608
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Also add TODOs and improve comments.
Change-Id: I50bd652c83c272c3f7662dd487ab617be2b7de08
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: I1da9a64fb06e015f2b2a0aacb46a53d70349b4b6
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
- Fail the testcase if there is no traffic pass.
Change-Id: I162187073aafa286793ce488fda9fd065bb55213
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ Place the libraries into resources/libraries/python/search/.
+ Except OptimizedTrexSearch in TrafficGenerator.py
+ Change traffic generator to support floats for duration and warmup.
+ Remove explicit type conversions where not needed.
+ Add robot keywords to performance_utils.robot
+ for calling the optimized search.
+ for reporting the resulting values.
+ for checking the minimal performance has been reached.
+ for running five second "Traffic should pass with no loss" after the search.
- Add methodology documentation in subsequent Change.
- Add simulator for testing algorithm correctness in a subsequent Change.
- Add tests using the libraries in subsequent Change.
Change-Id: Ia041008382ee4c9a562172099aea794c854d5f2f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ Docstring warnings fixed.
+ Multiline param descriptions indented by 4 spaces.
- Except the PacketVerifier.py one
- I have tried several quote-like blocks, nothing works.
- Rst warnings not fixed.
- How can I fix them? They refer to temporarily created files.
+ Other improvements:
+ Python lines no longer than 80 characters.
+ :return: -> :returns:
+ Notes before params.
+ :raises
+ closing colon after exception class.
+ Description is a sentence.
+ Present tense in conditional sentences.
+ Bumped copyright year in edited files.
Change-Id: I462c194eeecb666dc146e26858486a07c990be9b
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Currently Binary Search is using TRex warmup time for every lookup
trial. This patch is suppose to optimize Binary Search to not use
warmup time for each trial apart from first one.
Change-Id: I9b914cfac3ce558557133c266335c1f550c0b52a
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- CSIT-867: Low Level Description
- CSIT-868: wrk traffic profile - parsing
- CSIT-869: wrk implementation into CSIT
Change-Id: I65e1037f5ae05b3a5b2020e4a6c54462766ae1b4
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
Change-Id: I82681fc9ea52d72323d48ac78507ef1c571ca712
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- Preparing initialization of TRex in L7 mode
- Updating Topology files
- Adding sample ASTF profiles
Change-Id: If71f7f8f3db66425a1b543f1d29069a7543f4090
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I7e18ba8beaac507af376a859821ef7d571cbff80
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
Change-Id: Id60525182cbe5870f40aebf165e8bd86c101d267
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I506589c3785eec3a2eaa9fdb32bfc3375388d6f4
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
Change-Id: I772c9e214be2461adf58124998d272e7d795220f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
|
|
Profile name structure:
[trex|ixia]-[sl|sf]-[topo]-[packet structure]-[scale]
- Traffic generator - T-rex (trex) or Ixia (ixia)
- Stateless (sl) or statefull (sf)
- Topology - 2-node topology (2n) or 3-node topology (3n)
- Packet structure - eth | ip4/6 | udp
- Scale - e.g. ip4dst1, ip6dst10000, 4000u15p
Traffic types changed to stream profiles:
- From: 2-node-bridge
To: trex-sl-2n-ethip4-ip4src253
- From: 3-node-bridge
To: trex-sl-3n-ethip4-ip4src254
- From: 3-node-IPv4-dst-1000000
To: trex-sl-3n-ethip4-ip4dst1000000
- From: 3-node-IPv4-dst-100000
To: trex-sl-3n-ethip4-ip4dst100000
- From: 3-node-IPv4-dst-10000
To: trex-sl-3n-ethip4-ip4dst10000
- From: 3-node-IPv4-dst-1000
To: trex-sl-3n-ethip4-ip4dst1000
- From: 3-node-IPv4-dst-100
To: trex-sl-3n-ethip4-ip4dst100
- From: 3-node-IPv4-dst-1
To: trex-sl-3n-ethip4-ip4dst1
- From: 3-node-IPv4
To: trex-sl-3n-ethip4-ip4src253
- From: 3-node-IPv6-dst-1000000
To: trex-sl-3n-ethip6-ip6dst1000000
- From: 3-node-IPv6-dst-100000
To: trex-sl-3n-ethip6-ip6dst100000
- From: 3-node-IPv6-dst-10000
To: trex-sl-3n-ethip6-ip6dst10000
- From: 3-node-IPv6
To: trex-sl-3n-ethip6-ip6src253
- From: 3-node-xconnect
To: trex-sl-3n-ethip4-ip4src254
- From: 3-node-IPv4-SNAT-1000u-15p
To: trex-sl-3n-ethip4udp-1000u15p
- From: 3-node-IPv4-SNAT-100u-15p
To: trex-sl-3n-ethip4udp-100u15p
- From: 3-node-IPv4-SNAT-10u-15p
To: trex-sl-3n-ethip4udp-10u15p
- From: 3-node-IPv4-SNAT-1u-15p
To: trex-sl-3n-ethip4udp-1u15p
- From: 3-node-IPv4-SNAT-1u-1p
To: trex-sl-3n-ethip4udp-1u1p
- From: 3-node-IPv4-SNAT-2000u-15p
To: trex-sl-3n-ethip4udp-2000u15p
- From: 3-node-IPv4-SNAT-4000u-15p
To: trex-sl-3n-ethip4udp-4000u15p
Change-Id: If6a3cf204c93ff9f2cdccc5fd395e26b05483b58
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|
|
fix some code style issue and rebase the code.
change the DPDK l3fwd test case name and rebase the code.
fix the pylint errors.
change the DPDK version to 17.05
Rebase the code.
Fix the l3fwd test code like the l2fwd.
Fix the run l3fwd script issues.
Rebase the code and review.
Fix the l3fwd script issues.
verify-perf-dpdk-long
Change-Id: I5c5625ebcd5ff16f47fbee361789be3fece1ddbc
Signed-off-by: Fangyin Hu <fangyinx.hu@intel.com>
|
|
- Upgrade T-rex to v2.25 with multiple bug fixes.
Change-Id: Idc422172ce98c37ace7bbbf054a2b024c313bbf7
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
- High level definition (HLD)
- Low level definition (LLD)
- Add keywords to set SNAT
- Add tests according to HLD, LLD
Change-Id: I7bf0b1870ac9c3adb36bb6590be9a3eb4ea8aa9a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
|