aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/robot/performance
AgeCommit message (Collapse)AuthorFilesLines
2019-09-12Support unidirection in all searchesVratko Polak1-61/+84
+ Rename bool unidirection to int traffic_directions. + Rename "untagged" to "initial" for bandwidth calculation. + Fix latency measurement for unidirectional traffic. + Remove duplicate colon in soak test message. + Edit PAL to accept both forms. + Fix minor documentation issues. Change-Id: I6c76f2dc090ae493f2fbd7e9ccd45229d2306dea Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-10Refactor getting telemetryPeter Mikus1-3/+4
+ Ability to get stats from CNF via SocketPAPI - Remove obsolete functions Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I4d1b32a7279244592be96644e4f8a530c4f29a15
2019-09-06Read environment variables in Constants.pyVratko Polak1-2/+3
Instead of using EnsureGlobalVariable, which is clunky to use from Python. As a consequence, all caps variables from Constants.py are used directly and tests/__init__.robot and robot_enhancements.robot are deleted. + Rename the CRC global kill switch based on ci-man review. Change-Id: I10723792475bc83352bf8c3b7f7946ecf885a194 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak2-12/+136
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Improve PLRsearch yet againVratko Polak1-1/+1
Logic improvements for rls1908 and post-pylint fixes: + Reduce search time to 30 minutes. + Use average instead alternating loads. + Rework log/exp avg/stdev from two estimates. + Introduce and use pessimistic variance of dual trackers. + Introduce safe_exp to use when None does not skip code. + Use dot relative imports (instead of disabling pylint). + Complete docstrings for simpler functions. + Append docstrings to named tuples. + Somewhat unify docstrings related to the same arguments. + Slightly improve intentation. + State named tuples as rtype where used. + Add returns and rtype to __repr__ where missing. + Return what docstring say (or update them). + Explicit copying for vector and matrix values. Change-Id: I884c68b4839c5df5e8bef95e463666599603a0ff Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-12Clean: Various librariesPeter Mikus2-666/+3
- Remove translate TC as they have no perf opposite - Remove unused keywords - Merge similar keywords / rename - Move keywords from perf library to categories Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6433006136e2497ad8fb17193aa20c578d836999
2019-08-09Clean: L2 bridge domainPeter Mikus1-815/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I3721900b91a9c8e9db8766bdcb9771b6bcf95bca
2019-08-09Clean: L2_xconnect keywordsPeter Mikus1-284/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ic15a15c426db04defc15840883a772a0eb8e4885
2019-08-09Clean: L2_patchPeter Mikus1-9/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ib3620689acb3641b2ec9a49e2ce552bab3f00eef
2019-08-09Clean: Interface KeywordsPeter Mikus1-176/+16
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ideddcea56120cf770aec78ca24813baf7d05ba11
2019-08-07Clean: IPv6 keywordsPeter Mikus1-214/+6
+ Move all base ipv6 related KW to library + Remove useless KW to save space in output.xml + Remove cross dependency in ip6.robot Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I8c5f72e9852c1017356ab6034c4d271cf7b53265
2019-08-07Clean: IPv4 keywordsPeter Mikus1-547/+25
+ Move all base ipv4 related KW to library + Remove useless KW to save space in output.xml + Remove cross dependency in ip4.robot Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: If1e3a6370d36caf4ce7411fb956a0b7d0f939b6b
2019-08-06VPPD: Dot1QPeter Mikus1-18/+0
Change-Id: I0d3c925ea4a6896a0df98db6ddaf4238e6291bf1 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-26Change: Move few KWPeter Mikus1-35/+0
Change-Id: I8c8ae1d7eed8cf28fc63ce42b2dc36dd16a5c0da Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-24NF_density dot1qip4udpvxlanPeter Mikus1-26/+41
+ NF_density compatible + Baseline VM tests compatible + Scalability NxM - 3n variant is missing in VXLAN and DOT1Q due to requests to have it asymmetrical - TODO Change-Id: Ia817d7be00622b594b451bc07756240d1a379e04 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-19FIX: COP ipv6 missing FIBPeter Mikus1-2/+2
- From unknown reason VAT was more tolerant Change-Id: Ia740843c67604a18de40d14cc448138b9f079b41 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-17Remove obsolete DPDK bondingPeter Mikus1-99/+0
- Code not used Change-Id: I312635f8038c6d4d4db21d93e43df6b3636feb34 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-17Refactor VPP Device VM vhost testsjuraj.linkes1-85/+71
* replace the current VM image with kernel img * rework keyword usage to make it consistent with performance tests * remove resources/libraries/robot/shared/qemu.robot as it's not used anywhere anymore Change-Id: Ia5bc19e9e6ed9af031e4d9b5c0c89431fb49fd33 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-07-12VPPD: Cop, iACL, PolicerPeter Mikus1-58/+0
- Alignment with PERF Change-Id: Id7b2cb91c88e44cf9265c3376330f6aa579abf7a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-12PapiExecutor always verifiesVratko Polak1-2/+2
Do not support returning unverified replies anymore. Basically, ".get_replies().verify_replies()" is now just ".get_replies()". This allows fairly large simplifications both at call sites and in PapiExecutor.py + Rename get_dumps to get_details. + Introduce get_reply and get_sw_if_index. + Rename variables holding get_*() value, + e.g. get_stats() value is stored to variable named "stats". + Rename "item" of subsequent loop to hint the type instead. + Rename "details" function argument to "verbose". + Process reply details in place, instead of building new list. - Except hybrid blocks which can return both list or single item. - Except human readable text building blocks. + Rename most similar names to sw_if_index. - Except "vpp_sw_index" and some function names. + Use single run_cli_cmd from PapiExecutor. + Do not chain methods over multiple lines. + Small space gain is not worth readability loss. + Include minor code and docstrings improvement. + Add some TODOs. Change-Id: Ib2110a3d2101a74d5837baab3a58dc46aafc6ce3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-09CSIT-1539 Aligning FUNC/PERF keywords - Phase IIIPeter Mikus1-56/+96
Change-Id: Ibd3e99390bf004173d3d126946b95ffe9ba8f196 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-19FIX: VXLAN scale testsJan Gelety1-4/+3
Change-Id: I9287d5ed6d7876b98add3fb3d523e9764094843b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-07-05Remove pkt_trace as global variablePeter Mikus1-2/+3
- Use explicit parameters Change-Id: I21cbc8cf13ec5e455b20c685c4427a53a655f71f Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-28VAT-to-PAPI: ClassifyTibor Frank1-4/+2
Change-Id: Ic06a0a65429680e6ecdc3f5288d091c2c2630921 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-16FIX: IPUtil after vpp api changesJan Gelety1-4/+4
+ do not log all ip_route_add_del api calls to PAPI history in case of scale tests + fix for SRv6 proxy tests + support of more complex structures in api calls/replies in PapiExecutor + update of vpp stable build to version 19.08-rc0~487-gb98dbb1 Change-Id: I9972e126b68ce1cb57f781e3d90ea9b3914de69e Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-25Align suite/test teardown/setupPeter Mikus2-672/+1
+ Phase II - aligning FUNC/PERF setup. + 2n/3n compatibility of suite setup. + Same structure of testcase for DEV/PERF. + DEVICE tests are now running VPP startup config in same way as PERF. + DEVICE is using COREKEEPER! - DEVICE tests are running topology compute twice for backward compatibility. TODO to remove. - Still phase III needed to align TAGS, LIBRARIES and remove OLD setup phase. Change-Id: Ib6e1b0ebf4abebf3cbe73b4fda5cac953c2a7270 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-21Align suite/test teardown/setupPeter Mikus1-250/+1
+ Phase I - aligning FUNC/PERF Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-19Add continue/fail option for enable tracesTibor Frank1-1/+1
Change-Id: I675f9664391c9f18f312773812f765f345983cc1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-18CSIT-1459: Migrate IP libraries from VAT to PAPIJan Gelety1-164/+192
Change-Id: Id84cf9c3afd53260b5bbf658a6a3870aa0c91333 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-19VAT-to-PAPI: VPPUtilsTibor Frank1-2/+0
Change-Id: If4eb592e2718c4564f1f7929ca6ff91c02f074a6 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-19Add scale ipsec tunnel testsPeter Mikus1-1/+0
- 5K - 10K Change-Id: Iaa2c77d07c3686e09fe327c663d7d7fc42024034 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-09CSIT-1521 Convert NestedVM testcase into KernelVMPeter Mikus1-418/+94
Change-Id: Ic493253dc29dad4fc7e6541e4b839ca5898a6a40 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-05Even more Crypto combinationsPeter Mikus1-6/+0
Change-Id: I10eeb4ee30a57712824e68176d92d1ecb5f0d1b0 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-05-31CSIT-1468: InterfaceUtil migration from VAT to PAPIJan Gelety1-6/+6
Change-Id: I0062710d58996be767a852dc00545fedd60a5c72 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-05-10PLRsearch: Use stat trackers to shorten IntegratorVratko Polak1-2/+2
+ Extract several tracker classes and use them in Integrator. + Apply next_rate workarounds to focus more on critical region. + Rewrite stretch function, as the previous implementation was wrong. + Rework logging: ++ Use injected trace() function in Integrator. ++ Inject function that skips trace logging on default initialization. ++ Use the same multiprocessing-safe passing, but separate queue. + Set duration to 120m in Robot, but keep at 30m in Python. + Apply minor changes to make computations more reproducible: ++ Attempt to log everything needed by reproducibility in one line. ++ Log samples used, and make it settable as upper limit. ++ Use repr output in TRex scripts, to avoid rounding when copypasting. +- Numpy seems to be sharing PRNG between processes. +-- An edit to disable one thread is needed for full reproducibility. +-- Such an edit is prepared, but will not be merged anytime soon. Change-Id: Icf56429b34ab6de2d18a12e6c676e2d242b3ba4c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-10FIX: Missing show run data in NDRPDRPeter Mikus1-0/+27
Change-Id: If4b013ef4bf4bc8f857913f3847ab3b2e2aa5a5d Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-05-06CSIT-1493 VPP restart handling codePeter Mikus1-1/+2
Change-Id: Ibe52125089f39e0ff17ec607a3ed00c61d52ab8c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-26NF density tests with dtc=0.5 and dtcr=2Peter Mikus2-64/+14
Change-Id: Icff556142280ad0b6261e0a2bfb71672ee6b3807 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-24FIX: Crypto testsPeter Mikus1-0/+9
- By adding crypto_ipsecmb.so into the list of loaded plugins Change-Id: I52de9b410441422ad8e337208f40360fbdea264d Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-17Clean up traffic_profile vs osi_layerVratko Polak2-51/+57
The two types were not well distinguished before. Error introduced in: https://gerrit.fd.io/r/#/c/17811/84/resources/libraries/robot/performance/performance_setup.robot@255 Error hotfixed in: https://gerrit.fd.io/r/#/c/18847/4/resources/libraries/robot/shared/default.robot@109 + Rename some arguments and improve method docstrings. Newly introduced argument name osi_layer should be dissimilar enough. Change-Id: Ie0f6f97dc010fc6477f09c54574970f1d15462e2 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-04-15Remove obsolete QEMU installationPeter Mikus1-141/+0
+Unify QEMU parameter to Contants. Change-Id: I95adfc97dd7008c19e60ba3fa8668818b503bee8 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-12Convert existing nf_density testpmd-mac into KernelVMPeter Mikus1-2/+2
Change-Id: I9ec1701d9012de59cc42cc5f9a4744f01695ff9a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-08Upgrade autogen to NICs and search typesVratko Polak3-453/+309
+ All perf suites updated, as autogen change is backward incompatible. + Only x710 -ndrpdr suites remain in git repository. + Removed retry feature of run_tests. + Renamed topology_type and traffic_type to traffic_profile. + Renamed framesize to frame_size; nic_model to nic_name. + Reordered Variables table to start with nic_name and overhead. + Fixed wrong overhead value in some cbc-sha1 suites. + Fixed some suite tags. ++ Introduced tags to distinguish suites, such as IPSECINT. + Bound crypto hardware to NIC name. + Implemented NIC limit hiding. + Also search and teardown related arguments are now hidden. + Main measurement keywords updated to new arg handling. + Max rate related keywords moved to performance_limits.robot + Teardowns unified. + Generated tests are archived. ++ Generated directory is .gitignore-d. + Regenerator raises an exception on seeing non-compatible suite. ++ Relatively helpful message should be seen in exception. + Suite and template Documentation has generated parts. +- With short NIC names only. + Autogen checker also upgraded. + Tag expressions replaced with long files to ensure analogous tests. Change-Id: I60e9a999187e7da1f60d0eb4fb02afa14682aa46 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-04-06FIX: Operational issuesPeter Mikus1-48/+29
- FIX MSR package installation - FIX VTS - FIX sockets by removing from TCs Change-Id: I043c60f9e3e9c39cbeaee5e09b7a8351965efd35 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-03-27FIX: change socket location for vhost-user interfaceJan Gelety1-2/+2
Change-Id: Icce908d274f9fcd92e7eaa731da094b63fd3ee0e Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-03-28Add ipsec crypto test for Denverton platformYulong Pei1-10/+10
add ipsec cryto test to support Denverton hardware crypto device C3xxx. replace HW_cryptodev keyword to HW_DH895xcc or HW_C3xxx, or add more other hardware crypto device type support in the future. Change-Id: I181bca4095757fde7ca45ab3856273ac5930d902 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2019-03-20CSIT-1386 KernelVM - Part IIPeter Mikus2-36/+83
- Implementation of KernelVM (ThinVM) - nf_density tests for KernelVM with VPP as VNF. Change-Id: Ife5c2e1ab419c55cbcd442792f940db3a41da471 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-03-18CSIT-1386 KernelVM - Part IPeter Mikus3-71/+51
- Refactor QemuUtils Full refactor of QemuUtils to prepare for thinner VMs. Change-Id: I171ba11a9e6e9faec582bd58df6819598b0ed0b5 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-03-08FIX: Do not set sriov_numvfs of QAT card to 0 in case of SW crypto testsJan Gelety1-2/+3
- when sriov_numvfs of QAT card is set to 0 for SW crypto tests and then back to 32 for subsequent HW crypto the dpdk backend is not running Change-Id: I9bc6b2549823e3318c31bda55a976199266e3535 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-03-06CSIT-1451: PapiHistoryTibor Frank1-3/+3
Change-Id: Idb51f377fe0c196fbe24aab21c294dad595adcd2 Signed-off-by: Tibor Frank <tifrank@cisco.com>