aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/python/TrafficGenerator.py
AgeCommit message (Collapse)AuthorFilesLines
2019-09-12Support unidirection in all searchesVratko Polak1-29/+54
+ Rename bool unidirection to int traffic_directions. + Rename "untagged" to "initial" for bandwidth calculation. + Fix latency measurement for unidirectional traffic. + Remove duplicate colon in soak test message. + Edit PAL to accept both forms. + Fix minor documentation issues. Change-Id: I6c76f2dc090ae493f2fbd7e9ccd45229d2306dea Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-12TRex: 2.61Peter Mikus1-1/+2
- HDRhistogram can be rebased and used in separate patch. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ic7ce752344c6e1791af7cf7b7effde65ac2be680
2019-09-11Ansible: Trex installationPeter Mikus1-7/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5ab8fe05074a842301a511bbd3ab5e67f8d1a9f0
2019-08-27Stop using SSH() in TrafficGenerator.pyVratko Polak1-44/+29
This is a partial fix for: Ticket: CSIT-1434 and should fix one pylint violation. Many files to remove SSH() from remain. + Use -E (together with previous -S) in ssh.py when sudo=True. Needed in some cases, no downsides expected. Change-Id: Ie2ad03b3c656a4d3965a61a1e5435276f99a28c0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak1-73/+122
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-14DO_NOT_MERGE CSIT-1194 - Framework easy pylint improvementsMiroslav Los1-4/+4
- force a python2 environment in tox for pylint - let pylint load some compiled modules for member checking - allow locally enabled pylint rules - ignore bad enum names - ignore complexity for code to be refactored by other tickets - broad excepts are acceptable sometimes Signed-off-by: Miroslav Los <miroslav.los@pantheon.tech> Change-Id: If4c3a1b85e8609f074e1de8ce91fc9c4fedbeb38
2019-05-24CSIT-1425 Upgrade TRex to v2.54Tibor Frank1-55/+72
Change-Id: I71010c18210e88828ded0bc679997d76f34886cb Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-05-16Avoid !r format appending u on unicodeVratko Polak1-0/+8
Change-Id: I2c0877f4d8b6cad2694c43a4350998cd2b830680 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-10PLRsearch: Use stat trackers to shorten IntegratorVratko Polak1-5/+6
+ Extract several tracker classes and use them in Integrator. + Apply next_rate workarounds to focus more on critical region. + Rewrite stretch function, as the previous implementation was wrong. + Rework logging: ++ Use injected trace() function in Integrator. ++ Inject function that skips trace logging on default initialization. ++ Use the same multiprocessing-safe passing, but separate queue. + Set duration to 120m in Robot, but keep at 30m in Python. + Apply minor changes to make computations more reproducible: ++ Attempt to log everything needed by reproducibility in one line. ++ Log samples used, and make it settable as upper limit. ++ Use repr output in TRex scripts, to avoid rounding when copypasting. +- Numpy seems to be sharing PRNG between processes. +-- An edit to disable one thread is needed for full reproducibility. +-- Such an edit is prepared, but will not be merged anytime soon. Change-Id: Icf56429b34ab6de2d18a12e6c676e2d242b3ba4c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-04-17Clean up traffic_profile vs osi_layerVratko Polak1-10/+10
The two types were not well distinguished before. Error introduced in: https://gerrit.fd.io/r/#/c/17811/84/resources/libraries/robot/performance/performance_setup.robot@255 Error hotfixed in: https://gerrit.fd.io/r/#/c/18847/4/resources/libraries/robot/shared/default.robot@109 + Rename some arguments and improve method docstrings. Newly introduced argument name osi_layer should be dissimilar enough. Change-Id: Ie0f6f97dc010fc6477f09c54574970f1d15462e2 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-04-08Upgrade autogen to NICs and search typesVratko Polak1-35/+37
+ All perf suites updated, as autogen change is backward incompatible. + Only x710 -ndrpdr suites remain in git repository. + Removed retry feature of run_tests. + Renamed topology_type and traffic_type to traffic_profile. + Renamed framesize to frame_size; nic_model to nic_name. + Reordered Variables table to start with nic_name and overhead. + Fixed wrong overhead value in some cbc-sha1 suites. + Fixed some suite tags. ++ Introduced tags to distinguish suites, such as IPSECINT. + Bound crypto hardware to NIC name. + Implemented NIC limit hiding. + Also search and teardown related arguments are now hidden. + Main measurement keywords updated to new arg handling. + Max rate related keywords moved to performance_limits.robot + Teardowns unified. + Generated tests are archived. ++ Generated directory is .gitignore-d. + Regenerator raises an exception on seeing non-compatible suite. ++ Relatively helpful message should be seen in exception. + Suite and template Documentation has generated parts. +- With short NIC names only. + Autogen checker also upgraded. + Tag expressions replaced with long files to ensure analogous tests. Change-Id: I60e9a999187e7da1f60d0eb4fb02afa14682aa46 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-03-04Rename to Constants.pyVratko Polak1-1/+1
Motivation: Make the constants available also for Robot. Restraint: Robotframework user guide says: Because variable files are always imported using a file system path, creating them as classes has some restrictions: Python classes must have the same name as the module they are located. Change-Id: I638ef3fe045db132e366de2e2699638b8637e45e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-02-26De-duplicate unidirection supportVratko Polak1-125/+60
- Latency is now reported in both directions. -- For unidirection that means additional -1/-1/-1. + Changed function argument wrapping to save vertical space. + Changed trex command synthesys to save vertical space. + Fixed handling of command line arguments to avoid superfluous spaces. + More care distinguishing (0, 1) from (port_0, port_1). - Not renaming the original port arguments. Change-Id: I7b1736b594f6edea52eb07e5bc76ef52d0af92f8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-02-12Fix argument order in unidirection related KWsVratko Polak1-19/+18
The original unidirection Chhange [0] has changed the order, breaking some call sites. Also, use different indentation to save some vertical space. [0] https://gerrit.fd.io/r/#/c/16696/3/resources/libraries/python/TrafficGenerator.py@551 Change-Id: Ife9bd994da73f0afcd24fc16794f822a831c4353 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-01-29PLRsearch: Initial implementation and suitesVratko Polak1-0/+44
Missing bits: - Add up-to-date .rst document (in child Change). - Prepare for releasing to PyPI.org (in child Change): -- Either copy dependencies from MLRsearch, or list in requirements. -- Perhaps move common dependencies to separate package for both search to depend on. -- All the other metadata stuff. Jira: CSIT-1276 Change-Id: I277efdb63dbb90b30e11f4e30a82e2130ac8efc3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-01-08[V4] Add sending unidirectional traffic support for TG in CSITYulong Pei1-6/+97
Extend TG in CSIT to support to send unidirectional traffic, and then it can be used by unidirectional test cases e.g. Load balancer. Change-Id: Ie25bcdf7d4525a6d88d8ecd11a1b6d98275fa4d6 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2018-09-04Fix pylint part 1Peter Mikus1-2/+2
- Decrease pylint errors. Part: easy Change-Id: I452e5b5a11e9b78c03cd173a3848babe21b93c73 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-08-22CSIT-1222: Do two doublings in external MLRsearchVratko Polak1-2/+7
Make number of doublings configurable, keep Python default at 1, set Robot default to 2. Also make docstring types unique (pylint was complaining about classes and modules having the same name). Increase MLRsearch version to 0.2.0. Change-Id: Ib846032e79ff52994503c0cfef2f86655502c275 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-17CSIT-1186: Add srv6 NDRPDR suitesVratko Polak1-1/+1
Delete the corresponding ndrpdrdisc suites. MRR suites are also edited, to make them less different from NDRPDR suites. Now, there is a script to re-generate testcases of selected suites. For that, resources/libraries/python/autogen is created, with Regenerator holding the most of script logic. The script looks at files matching a glob expression, reuses a "prolog", but overwites test cases with the default template and hardcoded set of testcase arguments. Also, handling of overhead size has been simplified, using new keyword Get Max Rate And Jumbo And Handle Multi Seg. Default value for search timeout is increased to 12 minutes to make timeout failures less frequent. Change-Id: I394ad61f262b3078e5ca719034b15ada48987ee0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-12CSIT-1186: Consume MLRsearch in agreed upon wayVratko Polak1-3/+3
+ Rename search to MLRsearch in resources/libraries/python/. + Make PyPI/MLRsearch/MLRsearch a symlink to the renamed directory. + TrafficGenerator.py to import from the renamed directory/module. - No change to MLRsearch code. - Version stays at 0.1.1 to point to the latest released code. Change-Id: I4a38b0b7f8f083fbb1a93b25522560144b4ef556 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-06-20Revert "CSIT-986: Use MLRsearch from pip"Peter Mikus1-3/+3
This reverts commit 0c2c57d9965ab1db0d5404ad103871ac250339cb. Reason for revert: unless you will fix this broken patch we have to revert this as it breaks several thing in the csit core ci/cd. 1) all tb are pypi offline this means no new pypi is automatically dowloaded. Everytime you change your version you have to update every tb. As this is not automated this will never work and put extreme overhead 2) this code is not pylint checked and not verified by any existing job breaking any kind of verificition mechanics in csit 3)there is no automation in terms of creating pypi (no job) this creates chicken egg problem that you cannot verify change unless new version is installed Next time please think or consult with more seniored csit core developer Unless above will be fixed i will revert on monday as now it breaks all jobs. I will set hard verification to framework setup (exception) so this will prevent next time merge. Possible solution: 1)Move the directory back to resources and reference from csit directly not via pypi, leave the setup crap in dir and let users to create package on their own Change-Id: Id530a9f19f668e45f73ffc9e7371bb453b59fd1b Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-06-18CSIT-986: Use MLRsearch from pipVratko Polak1-3/+3
+ Delete resources/libraries/python/search/. + Add MLRsearch==0.1.1 to requirements.txt. + Bump PyPI/MLRsearch version to 0.1.2. Change-Id: I1f86d75ac8eb90ee3946dbac8c56280f39302f4d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-06-18CSIT-986: Implement proposed MDR improvementsVratko Polak1-6/+6
+ Use first intermediate with goal in initial phase. + Measure above MRR if that got zero loss. + Always prioritizes NDR in internal search. + Rename classes. + Copy code for standalone PyPI publishing. - Original files will be deleted after publish. Change-Id: I5169d602d1e5e35a1894645cd52e70d791871608 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-05-16CSIT-992: Fix intermediate phases MDR parameterVratko Polak1-5/+6
Also add TODOs and improve comments. Change-Id: I50bd652c83c272c3f7662dd487ab617be2b7de08 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-05-10CSIT-992: Give MDR parameters official namesVratko Polak1-13/+15
Change-Id: I1da9a64fb06e015f2b2a0aacb46a53d70349b4b6 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-05-10CSIT-1071 Add traffic evaluation criteria to MRRPeter Mikus1-2/+13
- Fail the testcase if there is no traffic pass. Change-Id: I162187073aafa286793ce488fda9fd065bb55213 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-05-04CSIT-992: Add libraries for optimized searchVratko Polak1-20/+134
+ Place the libraries into resources/libraries/python/search/. + Except OptimizedTrexSearch in TrafficGenerator.py + Change traffic generator to support floats for duration and warmup. + Remove explicit type conversions where not needed. + Add robot keywords to performance_utils.robot + for calling the optimized search. + for reporting the resulting values. + for checking the minimal performance has been reached. + for running five second "Traffic should pass with no loss" after the search. - Add methodology documentation in subsequent Change. - Add simulator for testing algorithm correctness in a subsequent Change. - Add tests using the libraries in subsequent Change. Change-Id: Ia041008382ee4c9a562172099aea794c854d5f2f Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-04-25Fix warnings reported by gen_doc.shVratko Polak1-14/+14
+ Docstring warnings fixed. + Multiline param descriptions indented by 4 spaces. - Except the PacketVerifier.py one - I have tried several quote-like blocks, nothing works. - Rst warnings not fixed. - How can I fix them? They refer to temporarily created files. + Other improvements: + Python lines no longer than 80 characters. + :return: -> :returns: + Notes before params. + :raises + closing colon after exception class. + Description is a sentence. + Present tense in conditional sentences. + Bumped copyright year in edited files. Change-Id: I462c194eeecb666dc146e26858486a07c990be9b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-03-05Performance KW optimizations IIPeter Mikus1-4/+12
Currently Binary Search is using TRex warmup time for every lookup trial. This patch is suppose to optimize Binary Search to not use warmup time for each trial apart from first one. Change-Id: I9b914cfac3ce558557133c266335c1f550c0b52a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-01-10CSIT-866: wrk onboarding in CSITTibor Frank1-0/+18
- CSIT-867: Low Level Description - CSIT-868: wrk traffic profile - parsing - CSIT-869: wrk implementation into CSIT Change-Id: I65e1037f5ae05b3a5b2020e4a6c54462766ae1b4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-01-04CSIT-885 Updagrade TRex to v2.35Peter Mikus1-2/+3
Change-Id: I82681fc9ea52d72323d48ac78507ef1c571ca712 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-01-04TRex ASTF onboarding Part IPeter Mikus1-63/+87
- Preparing initialization of TRex in L7 mode - Updating Topology files - Adding sample ASTF profiles Change-Id: If71f7f8f3db66425a1b543f1d29069a7543f4090 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-01-02Upgrade to T-rex v2.34Tibor Frank1-14/+9
Change-Id: I7e18ba8beaac507af376a859821ef7d571cbff80 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2017-09-03Update T-rex to v2.29Peter Mikus1-1/+1
Change-Id: Id60525182cbe5870f40aebf165e8bd86c101d267 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2017-08-25Update T-rex to v2.28Peter Mikus1-1/+1
Change-Id: I506589c3785eec3a2eaa9fdb32bfc3375388d6f4 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2017-06-29CSIT-687: Directory structure reorganizationTibor Frank1-8/+8
Change-Id: I772c9e214be2461adf58124998d272e7d795220f Signed-off-by: Tibor Frank <tifrank@cisco.com> Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2017-06-05CSIT-524: Traffic generator using python profilesTibor Frank1-382/+26
Profile name structure: [trex|ixia]-[sl|sf]-[topo]-[packet structure]-[scale] - Traffic generator - T-rex (trex) or Ixia (ixia) - Stateless (sl) or statefull (sf) - Topology - 2-node topology (2n) or 3-node topology (3n) - Packet structure - eth | ip4/6 | udp - Scale - e.g. ip4dst1, ip6dst10000, 4000u15p Traffic types changed to stream profiles: - From: 2-node-bridge To: trex-sl-2n-ethip4-ip4src253 - From: 3-node-bridge To: trex-sl-3n-ethip4-ip4src254 - From: 3-node-IPv4-dst-1000000 To: trex-sl-3n-ethip4-ip4dst1000000 - From: 3-node-IPv4-dst-100000 To: trex-sl-3n-ethip4-ip4dst100000 - From: 3-node-IPv4-dst-10000 To: trex-sl-3n-ethip4-ip4dst10000 - From: 3-node-IPv4-dst-1000 To: trex-sl-3n-ethip4-ip4dst1000 - From: 3-node-IPv4-dst-100 To: trex-sl-3n-ethip4-ip4dst100 - From: 3-node-IPv4-dst-1 To: trex-sl-3n-ethip4-ip4dst1 - From: 3-node-IPv4 To: trex-sl-3n-ethip4-ip4src253 - From: 3-node-IPv6-dst-1000000 To: trex-sl-3n-ethip6-ip6dst1000000 - From: 3-node-IPv6-dst-100000 To: trex-sl-3n-ethip6-ip6dst100000 - From: 3-node-IPv6-dst-10000 To: trex-sl-3n-ethip6-ip6dst10000 - From: 3-node-IPv6 To: trex-sl-3n-ethip6-ip6src253 - From: 3-node-xconnect To: trex-sl-3n-ethip4-ip4src254 - From: 3-node-IPv4-SNAT-1000u-15p To: trex-sl-3n-ethip4udp-1000u15p - From: 3-node-IPv4-SNAT-100u-15p To: trex-sl-3n-ethip4udp-100u15p - From: 3-node-IPv4-SNAT-10u-15p To: trex-sl-3n-ethip4udp-10u15p - From: 3-node-IPv4-SNAT-1u-15p To: trex-sl-3n-ethip4udp-1u15p - From: 3-node-IPv4-SNAT-1u-1p To: trex-sl-3n-ethip4udp-1u1p - From: 3-node-IPv4-SNAT-2000u-15p To: trex-sl-3n-ethip4udp-2000u15p - From: 3-node-IPv4-SNAT-4000u-15p To: trex-sl-3n-ethip4udp-4000u15p Change-Id: If6a3cf204c93ff9f2cdccc5fd395e26b05483b58 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2017-05-28 Add the DPDK l3fwd performance test case.Fangyin Hu1-1/+18
fix some code style issue and rebase the code. change the DPDK l3fwd test case name and rebase the code. fix the pylint errors. change the DPDK version to 17.05 Rebase the code. Fix the l3fwd test code like the l2fwd. Fix the run l3fwd script issues. Rebase the code and review. Fix the l3fwd script issues. verify-perf-dpdk-long Change-Id: I5c5625ebcd5ff16f47fbee361789be3fece1ddbc Signed-off-by: Fangyin Hu <fangyinx.hu@intel.com>
2017-05-23Upgrade T-rex to version 2.25Peter Mikus1-9/+2
- Upgrade T-rex to v2.25 with multiple bug fixes. Change-Id: Idc422172ce98c37ace7bbbf054a2b024c313bbf7 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2017-04-18CSIT-545: Performance tests for SNATTibor Frank1-0/+155
- High level definition (HLD) - Low level definition (LLD) - Add keywords to set SNAT - Add tests according to HLD, LLD Change-Id: I7bf0b1870ac9c3adb36bb6590be9a3eb4ea8aa9a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2017-04-10IPsec Multi-Tunnel performance test suiteKirill Rybalchenko1-1/+45
Change-Id: I4b0ba83960e50089f29cab9a30ab760241c6f566 Signed-off-by: Kirill Rybalchenko <kirill.rybalchenko@intel.com>
2017-04-03Add x710 and xl710 tests for testpmdTibor Frank1-2/+3
10ge2p1x710-eth-l2xcbase-ndrdisc 40ge2p1xl710-eth-l2xcbase-ndrdisc Change-Id: Iea411182fd41e1ae9ed9b5a17f540befc247adb9 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2017-03-31Upgrade T-rex to v2.22pmikus1-1/+1
Change-Id: I2029ba77a1c3c9ba6e128f816a353769a2cd5c8e Signed-off-by: pmikus <pmikus@cisco.com>
2017-02-09Upgrade T-rex to newer versionPeter Mikus1-2/+2
Change-Id: I8b918a3c1d8109fb64bfdeec8e5c9afe45a86d21 Signed-off-by: pmikus <pmikus@cisco.com>
2017-02-03CSIT-516: Add keywords for 2-node topologyTibor Frank1-2/+26
Change-Id: Ib5a1e207f1dec99747329a755c3c365fef4bd64c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2016-12-16Pylint fixesTibor Frank1-2/+12
- Fix PyLint errors - Fix comments in touched python modules Change-Id: I26db2d292a41969cf38b9b0bdd49c4fb15349102 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2016-11-24Fix documentation and pylint errorspmikus1-25/+58
- Fix documentation to be comliant with sphinx - Fix pylint errors Change-Id: I64acaa6c330c5a3b2975efc4120260813a2b3a92 Signed-off-by: pmikus <pmikus@cisco.com>
2016-10-26CSIT-395 Update TRex version to v2.09pmikus1-7/+9
- Update TRex to v2.09 Change-Id: I53221aff346840e0438dd1385cedd1db57ca5f3b Signed-off-by: pmikus <pmikus@cisco.com>
2016-10-19T-REX installer preserve env variablesMiroslav Miklus1-1/+1
This can be usefull for variables like http_proxy. Change-Id: I31128d2a57c961c6d888b6693b68fb57bde11436 Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-09-24CSIT-412 replace latency (N/A) string by number -1Miroslav Miklus1-11/+2
Change-Id: Id6373d3bbfcc42e315976b59aa86c6cadee411f1 Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>