aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/python/autogen
AgeCommit message (Collapse)AuthorFilesLines
2019-09-04Clarify why 9000B tests are skipped for IPsecVratko Polak1-1/+2
Ticket: VPP-1207 Change-Id: I506b4a24dfa6b10578407b261926f37215a714f4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-27- Removed 9000B test cases from ipsec test suites.Ludovit Mikula1-0/+3
- Added new traffic profiles. Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech> Change-Id: Ie06b7a1cf4d9df403d098b7a495219fd95e5869e
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak1-1/+43
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-19Autogen: Create NICs for TCP suites.Vratko Polak3-245/+270
+ Put Regenerator internal functions ourside class. + Duplicate into _default_ and _tcp_ functions. + Rework Testcase to be a single class with two factories. + Improve suite and testcase naming. Change-Id: I1f835b576c476c1a40c8c52469b1876695f9ae73 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-24NF_density dot1qip4udpvxlanPeter Mikus1-1/+2
+ NF_density compatible + Baseline VM tests compatible + Scalability NxM - 3n variant is missing in VXLAN and DOT1Q due to requests to have it asymmetrical - TODO Change-Id: Ia817d7be00622b594b451bc07756240d1a379e04 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-21Allow 9000b tests for AVFVratko Polak1-4/+0
See https://gerrit.fd.io/r/20148 Change-Id: I22856b14e3608348f2ccecb1d99d87209db2d776 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-27FIX: do not generate 9000B perf test in case of vic1385 nicJan Gelety1-0/+3
Change-Id: I7b8e30903a8bc012a3bb43bfb33755e08b56dc1d Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-05-09Add new higher density NF testsPeter Mikus1-0/+5
Change-Id: Ib2f3f3f8dc3e202dc1db53c739d9309be2ed8a60 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-26NF density tests with dtc=0.5 and dtcr=2Peter Mikus1-1/+1
Change-Id: Icff556142280ad0b6261e0a2bfb71672ee6b3807 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-08Upgrade autogen to NICs and search typesVratko Polak3-36/+128
+ All perf suites updated, as autogen change is backward incompatible. + Only x710 -ndrpdr suites remain in git repository. + Removed retry feature of run_tests. + Renamed topology_type and traffic_type to traffic_profile. + Renamed framesize to frame_size; nic_model to nic_name. + Reordered Variables table to start with nic_name and overhead. + Fixed wrong overhead value in some cbc-sha1 suites. + Fixed some suite tags. ++ Introduced tags to distinguish suites, such as IPSECINT. + Bound crypto hardware to NIC name. + Implemented NIC limit hiding. + Also search and teardown related arguments are now hidden. + Main measurement keywords updated to new arg handling. + Max rate related keywords moved to performance_limits.robot + Teardowns unified. + Generated tests are archived. ++ Generated directory is .gitignore-d. + Regenerator raises an exception on seeing non-compatible suite. ++ Relatively helpful message should be seen in exception. + Suite and template Documentation has generated parts. +- With short NIC names only. + Autogen checker also upgraded. + Tag expressions replaced with long files to ensure analogous tests. Change-Id: I60e9a999187e7da1f60d0eb4fb02afa14682aa46 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-02-27Add tox.ini and few checker scriptsVratko Polak2-35/+23
The plan is to change csit-validate-pylint-master job to fail if (and only if) tox fails. This will allow us to easily add checks, with or without the voting power. Each check produces log (ignored in .gitignore) the voting job can archive. + Made autogen quiet by default, to avoid spam in autogen checker. + Unified the way direct csit subdirectories are git-ignored. Change-Id: I6477b1ef7da6d3e30f68c5850d04900cc52f431e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-02-19Adapt autogen for SFd testsVratko Polak1-12/+23
+ Make the skipping logic in autogen more flexible. + Do not emit 4c cases when there is too much VMs/containers. + Peter says the second -eth- should be renamed away from suites. + Regenerate suites so test case names match (new) suite names. Change-Id: I95ecc47ca07b8bae6693d5d4330811ff55672fc4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-01-29PLRsearch: Initial implementation and suitesVratko Polak1-0/+5
Missing bits: - Add up-to-date .rst document (in child Change). - Prepare for releasing to PyPI.org (in child Change): -- Either copy dependencies from MLRsearch, or list in requirements. -- Perhaps move common dependencies to separate package for both search to depend on. -- All the other metadata stuff. Jira: CSIT-1276 Change-Id: I277efdb63dbb90b30e11f4e30a82e2130ac8efc3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-21Fix autogen regenerator and few suitesVratko Polak1-1/+1
+ "elif" instead of "else if". + Some suites had test cases reordered. + Some suites contained 9000B cases for avf. Change-Id: I3bf8241c317f20dbf66b497a2c8be35ef71307f5 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-20CSIT-1205 Create AVF driver test - Add Tests L2BD L2XCPeter Mikus1-6/+18
Change-Id: Ia74a87346fe2c0a170b881b510f1b0558a538f3c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-04Fix pylint part 1Peter Mikus2-4/+36
- Decrease pylint errors. Part: easy Change-Id: I452e5b5a11e9b78c03cd173a3848babe21b93c73 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-07-24CSIT-1097: Migrate L2 to NDRPDR and edit MRRVratko Polak2-11/+13
Change-Id: I738d1794f795e1225bc9709ea2ed2cf9b06d1cf4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-24CSIT-1097: Migrate Vts suite to NDRPDRVratko Polak2-4/+10
Change-Id: Iaac913bb4b15d743a2e9d565e25d31c089cee3e8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-18CSIT-1097: Ip6 NDRPDR and edited MRR suitesVratko Polak1-1/+8
Change-Id: I912017c6d1eab5530eecb2326887ffd057916ee8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-17CSIT-1186: Add srv6 NDRPDR suitesVratko Polak5-0/+245
Delete the corresponding ndrpdrdisc suites. MRR suites are also edited, to make them less different from NDRPDR suites. Now, there is a script to re-generate testcases of selected suites. For that, resources/libraries/python/autogen is created, with Regenerator holding the most of script logic. The script looks at files matching a glob expression, reuses a "prolog", but overwites test cases with the default template and hardcoded set of testcase arguments. Also, handling of overhead size has been simplified, using new keyword Get Max Rate And Jumbo And Handle Multi Seg. Default value for search timeout is increased to 12 minutes to make timeout failures less frequent. Change-Id: I394ad61f262b3078e5ca719034b15ada48987ee0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>