aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/robot
AgeCommit message (Collapse)AuthorFilesLines
2019-09-24Reconf tests: Fix async measurementsVratko Polak1-3/+8
TRex does not zero the server counters. It copies the values to use as reference, and subtracts them when asked for results. But the reference is stored in the client (not the server). And CSIT uses different scripts to start and stop async traffic, which means different clients. This patch introduces a workaround. Async start will return xstats objects to use as reference, and async stop will use the objects to compute the correct results. The xstats objects are stored in TrafficGenerator instance. Sync measurement does not export the counters, to shorten logs. Other improvements: + Make stop_traffic_on_tg return measurement results directly. + Rename --async to --async_start as "async" is reserved in Python 3.7 + Minor pylint, docstring and typo fixes. Change-Id: I5fc56a0763afb7d62cfa7c0651f96b6867de3e15 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-05FIX: Use vhost macs when testpmd_mac forwarding used in VMJan Gelety2-8/+58
Change-Id: I291c3dc852e7c1bde6fb6102a9168cfec9992682 Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit 474034ac24ce89cdaa94a09ae4d9ebbd3fb0f219)
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak4-60/+223
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit ff820e3d53434889aacde2373314c3be8d00f765)
2019-08-23Improve PLRsearch yet againVratko Polak1-1/+1
Logic improvements for rls1908 and post-pylint fixes: + Reduce search time to 30 minutes. + Use average instead alternating loads. + Rework log/exp avg/stdev from two estimates. + Introduce and use pessimistic variance of dual trackers. + Introduce safe_exp to use when None does not skip code. + Use dot relative imports (instead of disabling pylint). + Complete docstrings for simpler functions. + Append docstrings to named tuples. + Somewhat unify docstrings related to the same arguments. + Slightly improve intentation. + State named tuples as rtype where used. + Add returns and rtype to __repr__ where missing. + Return what docstring say (or update them). + Explicit copying for vector and matrix values. Change-Id: I884c68b4839c5df5e8bef95e463666599603a0ff Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit fbbc47359e3f7b59bbd5a84d85c673374933a50a)
2019-08-21Add: avf-dot1q testcasesPeter Mikus1-2/+6
+ Few fixes in doc + Align testcases Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5575db8255a74fd46f37d5cc2637b9b4448bcd7f (cherry picked from commit a0a1bdba6245c685898bab5007de28452bbd2d27)
2019-08-21Fix Initialize layer dot1q on nodeVratko Polak1-8/+4
By using Get Variable Value keyword. Change-Id: I3a9650b6f802d60422ffa6b6b00d444ebb050d89 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit b6cd8aba14026ee89178714171d2890a2796ca31)
2019-08-20Align NF_denisty testsPeter Mikus1-14/+27
+ Apply layer approach for interfaces Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6c1898b9833f9f42b8912f1b521e30093360d7ea (cherry picked from commit 329e0acee1cf31c6524883e15560993af76f0fad)
2019-08-19FIX: Missing ACL libraryPeter Mikus1-0/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I1497a9b031603d09d69a29ed7167686848d4b7c5 (cherry picked from commit 00b3327cf9df8ac078bde5f936bd2ac9fe666f03)
2019-08-14FIX: Remove eth_avf and portX_vifY interfaces from topo in TC/TS tear downJan Gelety1-0/+6
Change-Id: I90901c4ab14dfa76dc3e1f786a4b986479e0ba47 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-15VAT-to-PAPI: Fix HTTP/TCP testsDave Wallace2-12/+13
Change-Id: Ie728bc73eb7429d246e82dd195fc63191c3ac51d Signed-off-by: Dave Wallace <dwallacelf@gmail.com> (cherry picked from commit 6da7266754c28754c04cf37d1974e39766a261f6)
2019-08-14Move: WRK installation into ansiblePeter Mikus1-1/+1
- Remove ability to install from framework. - Control via ansible with other dependencies. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5fd0ca18b4f76c30b6c28b4dcd15e538e01cb315 (cherry picked from commit 37a9451377f7239365b5af778b8072d1585a2d2a)
2019-08-14FIX: Container VPP pid detectionPeter Mikus1-0/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I47163aa9ee0114f13ab17badde773560243ce319 (cherry picked from commit be39339d5bf121d718623885d05848e50d466347)
2019-08-14FIX: Vhost teardownPeter Mikus1-1/+3
If test failed before vnf_manager is created then teardown does fail as well due to missing variable. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0b23648bce78738cf4c377979d7a06fb488d990d (cherry picked from commit 8f41f1d040453d963440d8cc99fb2b957521120b)
2019-08-13Clean: Various librariesPeter Mikus16-901/+717
- Remove translate TC as they have no perf opposite - Remove unused keywords - Merge similar keywords / rename - Move keywords from perf library to categories Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6433006136e2497ad8fb17193aa20c578d836999 (cherry picked from commit 9c75b7bf5af74aa13cfa3aeeaf726cd965850fce)
2019-08-12FIX: if MAC address must be updated after AVF interface creationJan Gelety1-0/+6
Change-Id: I9fc26a7d1bf8d9eed620173a73febe3d874a1351 Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit 3e664d4a46aecf847659df2fe3d2572697b582a7)
2019-08-07FIX: Various typos in nf_density + missing KWPeter Mikus1-0/+14
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I73878542672ab2dcfa1db046fbca4b7d931e744c
2019-08-12Clean: L2 bridge domainPeter Mikus9-1037/+838
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I3721900b91a9c8e9db8766bdcb9771b6bcf95bca
2019-08-09Clean: L2_xconnect keywordsPeter Mikus3-286/+286
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ic15a15c426db04defc15840883a772a0eb8e4885 (cherry picked from commit 181e882d5c939cc6620545e5fde411e46fd59559)
2019-08-09Clean: L2_patchPeter Mikus2-16/+10
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ib3620689acb3641b2ec9a49e2ce552bab3f00eef (cherry picked from commit 016fb5988a63c3086b0b5b9f5ad7155c6ebef291)
2019-08-09Clean: Interface KeywordsPeter Mikus4-504/+97
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ideddcea56120cf770aec78ca24813baf7d05ba11 (cherry picked from commit 484cb0bddf8ad75129ac20b955a72f36f4241ad2)
2019-08-07Clean: IPv6 keywordsPeter Mikus2-220/+206
+ Move all base ipv6 related KW to library + Remove useless KW to save space in output.xml + Remove cross dependency in ip6.robot Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I8c5f72e9852c1017356ab6034c4d271cf7b53265
2019-08-07Clean: IPv4 keywordsPeter Mikus4-567/+539
+ Move all base ipv4 related KW to library + Remove useless KW to save space in output.xml + Remove cross dependency in ip4.robot Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: If1e3a6370d36caf4ce7411fb956a0b7d0f939b6b
2019-08-06Add: Dot1Q + L2BD + GBPPeter Mikus2-1/+101
Change-Id: I0050f715011ecfa92b3ee88b301809a56abb7946 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-08-06VPPD: Dot1QPeter Mikus2-21/+47
Change-Id: I0d3c925ea4a6896a0df98db6ddaf4238e6291bf1 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-08-06FIX: Layer dot1q logicPeter Mikus1-10/+20
Bug found by vrpolak Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ibf61f14a838d0b49220ab8079a2d999f860a18e7
2019-07-29Remove obsolete L2 KWPeter Mikus7-765/+210
Change-Id: I9a8aa6f5ccf95d632f4e677c8bf14f50b982d930 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-26Change: Move few KWPeter Mikus3-35/+34
Change-Id: I8c8ae1d7eed8cf28fc63ce42b2dc36dd16a5c0da Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-25VPPD: VXLANPeter Mikus4-92/+129
Change-Id: Iac518341e6a0fff4a31ad3692ccc5697690db8be Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-24NF_density dot1qip4udpvxlanPeter Mikus2-26/+257
+ NF_density compatible + Baseline VM tests compatible + Scalability NxM - 3n variant is missing in VXLAN and DOT1Q due to requests to have it asymmetrical - TODO Change-Id: Ia817d7be00622b594b451bc07756240d1a379e04 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-19Remove un-interested VIRL tests and related resourcesJan Gelety2-300/+0
Change-Id: I712f5dc238fdf480c7c35250148c12d03ed96d74 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-07-19FIX: COP ipv6 missing FIBPeter Mikus1-2/+2
- From unknown reason VAT was more tolerant Change-Id: Ia740843c67604a18de40d14cc448138b9f079b41 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-17Remove obsolete DPDK bondingPeter Mikus1-99/+0
- Code not used Change-Id: I312635f8038c6d4d4db21d93e43df6b3636feb34 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-17Use PapiSocketProvider for most PAPI callsVratko Polak2-1/+3
Ticket: CSIT-1541 Ticket: VPP-1722 Ticket: CSIT-1546 + Increase timeout to hide x520 slownes of show hardware detail. - Install sshpass and update ssh client in virl bootstrap. + Added TODOs to remove when CSIT-1546 is fixed. + Enable default socksvr on any startup conf. + Improve OptionString init and repr. - The non-socket executor still kept for stats. + Remove everything unrelated to stats from non-socket executor. - Remove some debug-loooking calls to avoid failures. TODO: Introduce proper parsing to the affected keywords. + Reduce logging from PAPI code to level INFO. - Needs https://gerrit.fd.io/r/20660 to fully work. + Change default values for LocalExecution.run() + Return code check enabled by default. Code is more readable when rc!=0 is allowed explicitly, and the test code will now detect unexpected failures. + Logging disabled by default. Output XML is large already. Important logging can be enabled explicitly. + Restore alphabetical order in common.sh functions. Change-Id: I05882cb6b620ad14638f7404b5ad38c7a5de9e6c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-17Refactor VPP Device VM vhost testsjuraj.linkes5-140/+176
* replace the current VM image with kernel img * rework keyword usage to make it consistent with performance tests * remove resources/libraries/robot/shared/qemu.robot as it's not used anywhere anymore Change-Id: Ia5bc19e9e6ed9af031e4d9b5c0c89431fb49fd33 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-07-12VPPD: Cop, iACL, PolicerPeter Mikus4-139/+113
- Alignment with PERF Change-Id: Id7b2cb91c88e44cf9265c3376330f6aa579abf7a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-12PapiExecutor always verifiesVratko Polak1-2/+2
Do not support returning unverified replies anymore. Basically, ".get_replies().verify_replies()" is now just ".get_replies()". This allows fairly large simplifications both at call sites and in PapiExecutor.py + Rename get_dumps to get_details. + Introduce get_reply and get_sw_if_index. + Rename variables holding get_*() value, + e.g. get_stats() value is stored to variable named "stats". + Rename "item" of subsequent loop to hint the type instead. + Rename "details" function argument to "verbose". + Process reply details in place, instead of building new list. - Except hybrid blocks which can return both list or single item. - Except human readable text building blocks. + Rename most similar names to sw_if_index. - Except "vpp_sw_index" and some function names. + Use single run_cli_cmd from PapiExecutor. + Do not chain methods over multiple lines. + Small space gain is not worth readability loss. + Include minor code and docstrings improvement. + Add some TODOs. Change-Id: Ib2110a3d2101a74d5837baab3a58dc46aafc6ce3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-11CSIT-1477: add 1n_tx2 VPP Devicejuraj.linkes1-1/+2
- add 1n_tx2 testbed - update VF reservation to meet 1n_tx2 testbed needs - update ansible with 1n_tx2 Change-Id: Ia075a913d4859f537fd0e6bff731ea88aff01dd9 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-07-11Aligning FUNC/PERF keywords - Data drivenPeter Mikus1-4/+18
Change-Id: I233c8908c724556db09c60de6efa9237068fb4c3 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-09CSIT-1539 Aligning FUNC/PERF keywords - Phase IIIPeter Mikus5-164/+157
Change-Id: Ibd3e99390bf004173d3d126946b95ffe9ba8f196 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-19FIX: VXLAN scale testsJan Gelety1-4/+3
Change-Id: I9287d5ed6d7876b98add3fb3d523e9764094843b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-07-05Remove pkt_trace as global variablePeter Mikus2-4/+4
- Use explicit parameters Change-Id: I21cbc8cf13ec5e455b20c685c4427a53a655f71f Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-05Add Additional Setup action for scapyPeter Mikus1-0/+7
Change-Id: I82f70d7c80b921da03553f6c1351a9a2f786ff0e Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-28VAT-to-PAPI: ClassifyTibor Frank2-21/+15
Change-Id: Ic06a0a65429680e6ecdc3f5288d091c2c2630921 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-27VAT-to-PAPI: IPFIX and SPANTibor Frank2-112/+3
Change-Id: I34bdc17d6350e5a441dabd9154620627780f4c12 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-16FIX: IPUtil after vpp api changesJan Gelety2-5/+5
+ do not log all ip_route_add_del api calls to PAPI history in case of scale tests + fix for SRv6 proxy tests + support of more complex structures in api calls/replies in PapiExecutor + update of vpp stable build to version 19.08-rc0~487-gb98dbb1 Change-Id: I9972e126b68ce1cb57f781e3d90ea9b3914de69e Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-25Align suite/test teardown/setupPeter Mikus11-830/+458
+ Phase II - aligning FUNC/PERF setup. + 2n/3n compatibility of suite setup. + Same structure of testcase for DEV/PERF. + DEVICE tests are now running VPP startup config in same way as PERF. + DEVICE is using COREKEEPER! - DEVICE tests are running topology compute twice for backward compatibility. TODO to remove. - Still phase III needed to align TAGS, LIBRARIES and remove OLD setup phase. Change-Id: Ib6e1b0ebf4abebf3cbe73b4fda5cac953c2a7270 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-21Align suite/test teardown/setupPeter Mikus8-435/+232
+ Phase I - aligning FUNC/PERF Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-19Add continue/fail option for enable tracesTibor Frank1-1/+1
Change-Id: I675f9664391c9f18f312773812f765f345983cc1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-19VAT-to-PAPI: VPPCountersTibor Frank1-0/+1
Change-Id: I5954fdd61ead9b4143f8cc4e20294cf8e199e0af Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-19VAT-to-PAPI: NATUtilTibor Frank1-78/+0
Change-Id: Id64bbf899e78c7c55410a8ce608393c22fa596ab Signed-off-by: Tibor Frank <tifrank@cisco.com>