aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/python/VPPUtil.py
AgeCommit message (Collapse)AuthorFilesLines
2022-06-28feat(telemetry): ReworkPeter Mikus1-1/+3
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0a84ce88b318c488ba2d519b20237c88b9f9f1e6
2021-12-15UTI: Export resultsVratko Polak1-1/+7
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-01-14perf: GENEVE tunnel test, l3 modeJan Gelety1-2/+2
Jira: CSIT-1768 Change-Id: I888ae1a5754fa07297d4cdf65c2be0e3e49d89a5 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-12-17PAPI: Cache connected client instancesVratko Polak1-3/+21
Disconnect+connect cycle is expensive and slow. This change tracks connected client instances so later "connect" to the same target uses it. Explicit disconnects are allowed (and executed before VPP ends), but once again disconnected instances are cached and reused, as creating a new instance is more expensive than just connect. + Add missing checks on interfaces being up to appropriate keyword. + Use appropriate keywords. + Add a comment explaining why a simpler keyword is not appropriate. + Improve VPP checks in containers. + Fix the vppctl check to actually work. + Add PAPI check to ensure VPP is really ready. + Delay/reorder checks to make them faster with multiple containers. + Leave some TODOs to improve various lifecycles later. + As we do not stop VPP in test/suite teardown: + One final disconnect is needed, added to __init__.robot teardowns. - Import of the final disconnect keyword is ugly, but it works. - We could use a hashable class for distinguishing node+socket pairs. - Are we connecting to VPP inside VMs? Change-Id: I49cd726740c3e8cae1591c7c84b85a447241228f Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-12-03vpp-device: GENEVE tunnel test, l3 modeJan Gelety1-0/+23
Jira: CSIT-1769 Change-Id: I98510cd8e627d7347f77d0032b7bac28f2e36c61 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-12-08Revert "vpp-device: GENEVE tunnel test, l3 mode"Peter Mikus1-23/+0
This reverts commit a9f54ca5080aeef17686f300a6807bf9b46b7c90. Reason for revert: DO NOT MERGE BROKEN TESTS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-device-master-ubuntu1804-1n-skx/9918/archives/log.html.gz Change-Id: I2ce2970a43e5dd7487eeb54d1ccbb149e06cf8fa Signed-off-by: pmikus <pmikus@cisco.com>
2020-12-03vpp-device: GENEVE tunnel test, l3 modeJan Gelety1-0/+23
Jira: CSIT-1769 Change-Id: I0ab906966f760b809b8431530c010a8f0835d62c Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-11-26Support CLI command change for event loggerVratko Polak1-2/+7
+ The old command is tried if the new one fails. Change-Id: I0afdcd7ac699595347b42f3a7d34e1281bb0dc0e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-09-25FIX: Workaround AVF interface down until VPP-1934pmikus1-1/+7
- Follow up https://jira.fd.io/browse/VPP-1934 Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Id0a26c5f67f229480332530a8531401d954f4422
2020-03-17CSIT-1597 API cleanup: vpeJan Gelety1-9/+1
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/25567 - update vpp stable to version 20.05-rc0~352 Change-Id: I205d7d3de48d480e070d3e2d3e3ab83480b69c77 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-02-04Revert "Change startup show pci to show ver"juraj.linkes1-2/+2
This reverts commit bb33f1d16f1451253f942da0c55cdea72e71c398. Change-Id: Ida80af970e400460860602db884a943e5599a849 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2020-01-15Hoststack perf infrastructure refactoringDave Wallace1-4/+3
- DUT only topology (hoststack test apps are co-located with vpp) - Make vpp app specific keywords generic where applicable - Add IP4 Prefix to topology file - Support running wrk in linux namespace - Refactor namespace cleanup - Remove redundant namespace creation code - Refactor test/keyword dirs: tcp -> hoststack - Add hoststack utility keywords - Refactor wrk suite setup/teardown - Update tests with recent perf infra changes Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-12-17Telemetry: Add more operational dataPeter Mikus1-1/+15
+ Add both NDR and PDR telemetry capture + Speedup sockets + Adjust privileges Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia6fd5d405e6fb410651d8b705c921653753aea10
2019-12-11Introduce VPP-IPsec container tests.Ludovit Mikula1-10/+24
Change-Id: Ie64d662e81879bd52785e0188450d998bf056bda Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
2019-11-29Change startup show pci to show verjuraj.linkes1-2/+2
When testing whether a socket is listetning, show pci results in a crash on the Taishan platform. Temporarily work around this by changing it to show ver. Change-Id: Id0f995373bc2f429d59de200e51959b2da408052 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-11-28Python3: resources and librariesJan Gelety1-67/+55
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-10-30Framework: Add show log as tear down actionPeter Mikus1-13/+22
+ We can see what is happening when debug level is increased + Big caveat of VPP is that it does not allow to configure default global level. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5ea1a618161cdb3170cab022d7451e52271f83c0
2019-10-04Update of VPP_STABLE_VER filesJan Gelety1-0/+4
- use new vpp ref build - ubuntu 16.04: 20.01-rc0~324-g66a332c~b8122 - use new vpp ref build - ubuntu 18.04: 20.01-rc0~324-g66a332cf1 - use new vpp ref build - centos7: 20.01-rc0~324_g66a332c~b7936 Change-Id: I82c6032742939417914f5210b1fba4464ab1e70d Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-09-17Update CRC list and support 21997/7Vratko Polak1-2/+4
- Attempt to repair IPsec LispGpe perf suite. + Collection name to reflect the current stable vpp. + Add messages found in CSIT L1 keywords. - Uncommented (instead of deleted) untestable messages. Reasons: - Honeycomb. - Messages used by unused keywords. + Listed reasons. Honeycomb not mentioned if both reasons apply. + Delete CRC items for commands not found in keywords anymore. + Add CRCs from .json.api files (as teardown is hard to execute). + Define and restore alphabetical order. + Add hints to find used API commands (not entirely reliable). + Move used commands to "cmd = " form so hints find them. + Argument to run_cli_command changed from "cmd" to "cli_cmd". + Except also struct.error where IOError is excepted. Change-Id: I61058dbe1e33296908aabd0c13433bb16cfa6adf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-13FIX: Pylint + Container mountPeter Mikus1-1/+0
- Mount is affecting only LXC container Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I84140ea5e48120792fc84182a97eb7d06c954a84
2019-08-20Interface API cleanupJan Gelety1-4/+13
CSIT code alignment with API changes in VPP introduced by patch https://gerrit.fd.io/r/c/vpp/+/18361 Change-Id: Ib0357bba79f55d297ef1086fbf3b760caca16cdb Signed-off-by: Jan Gelety <jgelety@cisco.com> Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-26Add: AVF into trace groupPeter Mikus1-1/+2
Change-Id: I3399d5fb2754eb67734e8f76a5743283fe9c074c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-17Use PapiSocketProvider for most PAPI callsVratko Polak1-10/+9
Ticket: CSIT-1541 Ticket: VPP-1722 Ticket: CSIT-1546 + Increase timeout to hide x520 slownes of show hardware detail. - Install sshpass and update ssh client in virl bootstrap. + Added TODOs to remove when CSIT-1546 is fixed. + Enable default socksvr on any startup conf. + Improve OptionString init and repr. - The non-socket executor still kept for stats. + Remove everything unrelated to stats from non-socket executor. - Remove some debug-loooking calls to avoid failures. TODO: Introduce proper parsing to the affected keywords. + Reduce logging from PAPI code to level INFO. - Needs https://gerrit.fd.io/r/20660 to fully work. + Change default values for LocalExecution.run() + Return code check enabled by default. Code is more readable when rc!=0 is allowed explicitly, and the test code will now detect unexpected failures. + Logging disabled by default. Output XML is large already. Important logging can be enabled explicitly. + Restore alphabetical order in common.sh functions. Change-Id: I05882cb6b620ad14638f7404b5ad38c7a5de9e6c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-12PapiExecutor always verifiesVratko Polak1-27/+19
Do not support returning unverified replies anymore. Basically, ".get_replies().verify_replies()" is now just ".get_replies()". This allows fairly large simplifications both at call sites and in PapiExecutor.py + Rename get_dumps to get_details. + Introduce get_reply and get_sw_if_index. + Rename variables holding get_*() value, + e.g. get_stats() value is stored to variable named "stats". + Rename "item" of subsequent loop to hint the type instead. + Rename "details" function argument to "verbose". + Process reply details in place, instead of building new list. - Except hybrid blocks which can return both list or single item. - Except human readable text building blocks. + Rename most similar names to sw_if_index. - Except "vpp_sw_index" and some function names. + Use single run_cli_cmd from PapiExecutor. + Do not chain methods over multiple lines. + Small space gain is not worth readability loss. + Include minor code and docstrings improvement. + Add some TODOs. Change-Id: Ib2110a3d2101a74d5837baab3a58dc46aafc6ce3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-25Align suite/test teardown/setupPeter Mikus1-5/+3
+ Phase II - aligning FUNC/PERF setup. + 2n/3n compatibility of suite setup. + Same structure of testcase for DEV/PERF. + DEVICE tests are now running VPP startup config in same way as PERF. + DEVICE is using COREKEEPER! - DEVICE tests are running topology compute twice for backward compatibility. TODO to remove. - Still phase III needed to align TAGS, LIBRARIES and remove OLD setup phase. Change-Id: Ib6e1b0ebf4abebf3cbe73b4fda5cac953c2a7270 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-19Add continue/fail option for enable tracesTibor Frank1-6/+23
Change-Id: I675f9664391c9f18f312773812f765f345983cc1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-19VAT-to-PAPI: VPPUtilsTibor Frank1-28/+22
Change-Id: If4eb592e2718c4564f1f7929ca6ff91c02f074a6 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-05-31CSIT-1468: InterfaceUtil migration from VAT to PAPIJan Gelety1-7/+24
Change-Id: I0062710d58996be767a852dc00545fedd60a5c72 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-05-06CSIT-1493 VPP restart handling codePeter Mikus1-76/+69
Change-Id: Ibe52125089f39e0ff17ec607a3ed00c61d52ab8c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-09Add vagrant setup for testing VPP device locallyLudovit Mikula1-1/+1
Fixed trailing whitespaces Fixed tabs vs. spaces Removed dead code Changed network interface naming Changed default number of retries for VPP startup to 120 Updated number of hugepages during provisioning Externalized CSIT related sysctl settings to 90-csit.conf Removed hardcoded local docker image name. Added new interface required for honeycomb device testing. Change-Id: I5eb38c8b1a4d96e4220dbd2e202e94f1d47e804e Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
2019-03-20CSIT-1386 KernelVM - Part IIPeter Mikus1-0/+3
- Implementation of KernelVM (ThinVM) - nf_density tests for KernelVM with VPP as VNF. Change-Id: Ife5c2e1ab419c55cbcd442792f940db3a41da471 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-03-15PAPI: Add verification of the responseTibor Frank1-7/+6
Change-Id: I86afcaeae865f0af076b8dd974386a83de07bf44 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-03-06CSIT-1450: PAPI executorVratko Polak1-60/+17
Change-Id: I4c756cc4b29901184594a728f6184c30cadd9c1a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-03-04Rename to Constants.pyVratko Polak1-1/+1
Motivation: Make the constants available also for Robot. Restraint: Robotframework user guide says: Because variable files are always imported using a file system path, creating them as classes has some restrictions: Python classes must have the same name as the module they are located. Change-Id: I638ef3fe045db132e366de2e2699638b8637e45e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-02-04CSIT-1411 Implement manual rx-placement override for bug in VPPPeter Mikus1-0/+33
Change-Id: Ie3d2b1f40a607ce5190ccfea6a372bc072d0a3b9 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-01-23Fix PyLint errorsTibor Frank1-2/+2
Change-Id: I5a369f4b9f9b478375fda4a634bf57cc9623e972 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-11-27CSIT python API introductionJan Gelety1-10/+54
Jira: CSIT-1336 Change-Id: I96d2b0221c5a7466484a82339fc132c5921532d1 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-10-16CSIT-1338 FIX: Failed to get VPP version on hostPeter Mikus1-7/+26
VPP was not initialized (shm), after service restart, quickly enough. This results into vpp_api_test console timeouts when communicating over uninitialized socket. Patch is introducing more stable test to detect service (VPP) is fully ready and initialized. Change-Id: I300a763814feba28cb8163dcd4996b53b2994fec Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-10-02CSIT-1326 VLIB: barrier sync elog tracingPeter Mikus1-7/+7
- API changed. Change-Id: If70e1a930aa14b129ad38ef544cff32ad5a08cdd Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-10-01CSIT-1326 Add barrier event tracesPeter Mikus1-0/+43
Change-Id: I0f734c2f11cc89716371554880f1d7404ea16090 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-28FIX: Single memif tests failingPeter Mikus1-0/+9
This patch suppose to fix the single memif TCs failing on SKX nodes. By stoping Host VPP during startup of Guest VPP. This way Guest VPP can take interface on single pass.o Change-Id: I88e6932bf007ce0f6a367927de5f017ffb8fdaca Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-19CSIT-1291 Improve service handling with container detectionPeter Mikus1-23/+147
Change-Id: Iaeac10676f46e0267041afee927bc6e3201146b2 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-04-25Fix warnings reported by gen_doc.shVratko Polak1-2/+2
+ Docstring warnings fixed. + Multiline param descriptions indented by 4 spaces. - Except the PacketVerifier.py one - I have tried several quote-like blocks, nothing works. - Rst warnings not fixed. - How can I fix them? They refer to temporarily created files. + Other improvements: + Python lines no longer than 80 characters. + :return: -> :returns: + Notes before params. + :raises + closing colon after exception class. + Description is a sentence. + Present tense in conditional sentences. + Bumped copyright year in edited files. Change-Id: I462c194eeecb666dc146e26858486a07c990be9b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2017-10-09CSIT-775 HC Test: CRUD over IPv6 control-plane interfaceselias1-0/+32
Adds new test suite which accesses Honeycomb's Restconf and Netconf northbound over IPv6. This is done by sending the requests to TG where an IPv4 to IPv6 tunnel is configured to the Honeycomb DUT. Change-Id: I3b7467d8108b0da9756e632851215c66ed41bb61 Signed-off-by: selias <samelias@cisco.com>
2016-12-16Pylint fixesTibor Frank1-7/+3
- Fix PyLint errors - Fix comments in touched python modules Change-Id: I26db2d292a41969cf38b9b0bdd49c4fb15349102 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2016-10-07CSIT-247: IPv6 - ip6-ipsec-lispgpe-ip6 - main fib, phy2lisp, virt2lispZdeno Olsovsky1-0/+52
Change-Id: I039e14b7c227318c70826068368430978f8f0a51 Signed-off-by: Zdeno Olsovsky <zolsovsk@cisco.com>