aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/python/ssh.py
AgeCommit message (Collapse)AuthorFilesLines
2022-12-06feat(model): Cleanup and introduce telemetrypmikus1-42/+9
- Due to divergence from original design path the RAW was never consumed. It adds too much code complexity and requires processing on both storage and compute. Removing entirely to make modeling efficient. - log (apparently SSH) section will never be consumed in the way it is coded in model. This section is also not part of model schema itself due to the point above. - Introducing telemetry section that is going to carry telemetry items required for CDash. Signed-off-by: pmikus <peter.mikus@protonmail.ch> Change-Id: I7e0256c6c9715de8ee559eed29dce96329aac97d
2021-12-15UTI: Export resultsVratko Polak1-19/+59
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-10-06docstring: Docstring warnings fixed.Viliam Luc1-3/+3
Change-Id: I8679c819b7c6ecd9e680bdc6edeafd2124def0b5 Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-07-06Telemetry: Add node infopmikus1-5/+18
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I428bf2628f5d6b87d335de415a82d6e94867d8c0
2021-06-10FIX: Pylint reducepmikus1-1/+1
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I909942dbb920df7f0fe15c0c92cda92c3cd8d8ad
2019-12-01FIX: VAT terminalPeter Mikus1-6/+8
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I98fd62e6b5367b23694017d99a1dae7d893326c9
2019-11-28Python3: resources and librariesJan Gelety1-133/+121
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-27Stop using SSH() in TrafficGenerator.pyVratko Polak1-5/+5
This is a partial fix for: Ticket: CSIT-1434 and should fix one pylint violation. Many files to remove SSH() from remain. + Use -E (together with previous -S) in ssh.py when sudo=True. Needed in some cases, no downsides expected. Change-Id: Ie2ad03b3c656a4d3965a61a1e5435276f99a28c0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-12PapiExecutor always verifiesVratko Polak1-19/+1
Do not support returning unverified replies anymore. Basically, ".get_replies().verify_replies()" is now just ".get_replies()". This allows fairly large simplifications both at call sites and in PapiExecutor.py + Rename get_dumps to get_details. + Introduce get_reply and get_sw_if_index. + Rename variables holding get_*() value, + e.g. get_stats() value is stored to variable named "stats". + Rename "item" of subsequent loop to hint the type instead. + Rename "details" function argument to "verbose". + Process reply details in place, instead of building new list. - Except hybrid blocks which can return both list or single item. - Except human readable text building blocks. + Rename most similar names to sw_if_index. - Except "vpp_sw_index" and some function names. + Use single run_cli_cmd from PapiExecutor. + Do not chain methods over multiple lines. + Small space gain is not worth readability loss. + Include minor code and docstrings improvement. + Add some TODOs. Change-Id: Ib2110a3d2101a74d5837baab3a58dc46aafc6ce3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-21SetupFramework: log thread error to consoleVratko Polak1-29/+54
Because logger.error from a non-main thread is lost. + Increase verbosity of virtualenv exception. + Introduce LocalExecution.py library. + Include multiple other style improvements. + Add some ssh.py improvements: + Accept also commands of OptionString type. + Introduce a limited form of raise_from for exception chaining. + Add include_reason argument to exec_cmd_no_error. Change-Id: I0cd92bcdaa1a5c4cc20cb99067593fc3b40d2c31 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-18CSIT-1459: Migrate IP libraries from VAT to PAPIJan Gelety1-0/+1
Change-Id: Id84cf9c3afd53260b5bbf658a6a3870aa0c91333 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-18PAPI: Reduce the amount of logged informationTibor Frank1-8/+23
Change-Id: I5cfd19a9ecca6891232d92d232a159422c0a835e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-05-06CSIT-1493 VPP restart handling codePeter Mikus1-6/+16
Change-Id: Ibe52125089f39e0ff17ec607a3ed00c61d52ab8c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-17Introduce OptionString for handling command lineVratko Polak1-10/+30
+ Convert DpdkUtil to use it. ++ Rename args to kwargs where needed. ++ Fix errors in docstrings. + Also convert and QemuUtils to use it. ++ Minor formatting edits to save space. + Add disconnect parameter to some ssh.py functions. ++ ssh.SSH.disconnect() tries to work without argument. + Exec functions in ssh.py accept OptionString commands. Change-Id: I82da71c568d120c283544c90242993fc76e9e83a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-02-08CSIT-845 Capture VPP core-dump from vpp crash on DUTsPeter Mikus1-1/+31
Change-Id: I987dcd5092d8527c9aefbe093e1ed7ae144d191b Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-01-23Fix PyLint errorsTibor Frank1-5/+7
Change-Id: I5a369f4b9f9b478375fda4a634bf57cc9623e972 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-01-10VPP_Device - add baseline tests - part IIa)Jan Gelety1-11/+51
CSIT-1372 Add following baseline tests to VPP_Device: - ip4 eth2p-ethip4-ip4base-eth-2vhost-1vm-dev - ip6 eth2p-ethip6-ip6base-eth-2vhost-1vm-dev - l2bd eth2p-eth-l2bdbasemaclrn-eth-2vhost-1vm-dev - l2xc eth2p-eth-l2xcbase-eth-2vhost-1vm-dev + fix doc strings in all vpp_device tests Change-Id: Icc5ed801f3e8f6dedaa611779bd86fc20102fa74 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-12-18Adding DMM build artifactssharath1-2/+9
Change-Id: Ia5153d8e716d506c85ec8b723331438efd66ab6a Signed-off-by: sharath <sharathkumarboyanapally@gmail.com>
2018-10-10Add VXLAN scale perf testsJan Gelety1-1/+0
Jira: CSIT-1273 Change-Id: Ic2a41661c158384a5aaa7d4e73f30ffca13ddf82 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-05CSIT-1205 Create AVF driver testPeter Mikus1-5/+20
- Add L1 KWs for SR-IOV handling (init Vfs, remove Vfs, ...) - Cleanup L1 KWs for SR-IOV bind/unbind/pci_get/... - Add L2 KWs for Test Setup/Teardown, L2patch, Create AVF interface - Add sample L2patch test fox x710, xxv710 Change-Id: If17077877455a14043617d8ea0d06cbe47b469e3 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-05Fix various pylint violationsVratko Polak1-1/+1
+ SchedUtils.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + VatHistory.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + VppCounters.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + Memif.py: ++ Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Either all return statements in a function should return an expression, or none of them should. ++ Update :return: on possible None. + Classify.py: Unnecessary "else" after "return" + ContainerUtils.py: Useless super delegation in method '__init__' + CpuUtils.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + DropRateSearch.py: Either all return statements in a function should return an expression, or none of them should. + IPv4NodeAddress.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Also improve docstrings. + IPv4Setup.py: Useless super delegation in method '__init__' + IPv6Setup.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Also improve docstrings. + IPv6Setup.py: standard import "from ipaddress import IPv6Network" should be placed before "from robot.api import logger" + MacSwap.py: Trailing newlines + NATUtil.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + NodePath.py: Unnecessary "else" after "return" + Tap.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + topology.py: Either all return statements in a function should return an expression, or none of them should. + topology.py: Unnecessary "else" after "return" ++ Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Improve docstrings + DUTSetup.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Also do not compare int(ret_code) just to access zero-ness. + ssh.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + InterfaceUtil.py: Unnecessary "else" after "return" Change-Id: Iba4244aa79661ee7df15fed5c7c6dbf04dfa88b2 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-24add the expected prompt for ubuntu root userYulong Pei1-1/+1
if ssh use root account to login ubuntu os, its prompt is ":~# ", so add it to let csit test can run with root account. Change-Id: I21f5380245324c8e48099433e3d374188e9001d9 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2018-07-19FIX: Increase timeout for VAT for longer API calls.Peter Mikus1-1/+1
Change-Id: Idab5e8d62a42513ddef97025625962b510c3a625 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-07-09CSIT-1142 2-node topology - keywordsPeter Mikus1-1/+1
- Clean up of 2-node setup KWs Change-Id: I59fc901fb57544eceb6a041b56b515cda8babf85 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-05-11CSIT-1076 Improve SSH connection handlingPeter Mikus1-25/+36
Properly detect dropped connection via predicate is_active() to prevent using dropped connection. Change-Id: I9105622f28d2f7949675b8cee1dd4384a12f72a9 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-05-09FIX: SSH connect issuePeter Mikus1-22/+14
error: [Errno 9] Bad file descriptor Change-Id: I4c5278d41e979f6af43054e3b5890fb154555494 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-05-04Fix various pylint 1.5.4 warningsVratko Polak1-13/+21
+ DUTSetup.py:424 Else clause on loop without a break statement + InterfaceUtil.py:400 Else clause on loop without a break statement + QemuUtils.py:564 Wrong continued indentation + SetupDPDKTest.py: Locally enabling broad-except + VatExecutor.py: Catching too general exception Exception + ssh.py:95 No exception type(s) specified. + HTTPRequest.py: Tolerate HTTPCodes.OK + multiple: Drop ":returns: None" from docstrings. There are still several warnings present: - R0902(too-many-instance-attributes) - R0912(too-many-branches) - R0913(too-many-arguments) - R0914(too-many-locals) - R0915(too-many-statements) - R0401(cyclic-import) And there are multiple blocks of similar lines, mainly across various Setup*Test.py files: - R0801(duplicate-code) Change-Id: I582575cb52b85d69d268e6374852f6e74bb71052 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-04-25Fix warnings reported by gen_doc.shVratko Polak1-6/+6
+ Docstring warnings fixed. + Multiline param descriptions indented by 4 spaces. - Except the PacketVerifier.py one - I have tried several quote-like blocks, nothing works. - Rst warnings not fixed. - How can I fix them? They refer to temporarily created files. + Other improvements: + Python lines no longer than 80 characters. + :return: -> :returns: + Notes before params. + :raises + closing colon after exception class. + Description is a sentence. + Present tense in conditional sentences. + Bumped copyright year in edited files. Change-Id: I462c194eeecb666dc146e26858486a07c990be9b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-04-25FIX: SSH socket reading errorPeter Mikus1-11/+16
Currently if the command timeouts due to the nature of script execution we are trying to again read from socket after timeout on seesion expired. This patch is supposed to fix the behaviour and eliminate the second read from channel after it was cloed. Change-Id: Ib0028e901eb6ff4d10ca77d2ea19f9e770e6cef4 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-04-06FIX: VAT SSH timeoutPeter Mikus1-3/+1
Currently when VAT cannot connect to VPP via direct API call, there is inner timeout of ~100s until it quits and returns RC. In our code we are setting outer timeout to 10/15s to detect if VAT is not responding. If VAT does not respond quickly enough due to e.g VPP crash, we are incorrectly reporting SSHTimout exception. This fix is suppose to set correct timeout values and also to set some of the calls like show run|hard|err / clear to detect whether they were successfull or not. + Various small library cleanup. Change-Id: I787c4baecd7e086705a4076643e255a875ea8438 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-01-10add new topology parameter: archGabriel Ganne1-1/+1
if unset, arch variable will default to "x86_64" * Note on "arm64" vs "aarch64" debian-based uses arm64 rhel-based uses aarch64 qemu binaries of both distribs uses aarch64 dpdk uses arm64 vpp uses aarch64 python machine modules uses aarch64 => prefer aarch64 to use the same nomenclature as vpp * add ARCH argument to: init_dpdk.sh, install_dpdk.sh, run_l2fwd.sh, install_tldk.sh, run_tldk.sh. default to x86_64 converts "aarch64" if needed for dpdk naming convention * fixed terminal end detection to allow "~]# " add dut node arch as param to all robot set bin calls * add --target-list flag to qemu_build.sh defaults to x86_64-softmmu * add arch flag to all the topology files * topologies/available/ (and example file) * resources/tools/virl/topologies/ * set _qemu_bin path using node['arch'] in qemu_set_node() Change-Id: If46d88d064d213d3e4c6fc584bb8e0d4b6428cb8 Signed-off-by: Gabriel Ganne <gabriel.ganne@enea.com>
2017-10-16CSIT-828: Fix the output of failed VAT executionJan Gelety1-5/+9
Change-Id: Id387bd685142dcfaa17f0236e930e40e6c0bce5a Signed-off-by: Jan Gelety <jgelety@cisco.com>
2017-10-03HC Test: increase timeout for archiving HC log fileselias1-2/+4
- add timeout parameter to ssh.scp method - increase timeout in "Archive Honeycomb Log" keyword - increase timeout for copying ODL client in setup Change-Id: I9e914d5f439fd14471b10b611efdbb6afada4c6e Signed-off-by: selias <samelias@cisco.com>
2017-06-30CSIT-619 HC Test: Honeycomb performance testing - initial commitselias1-5/+23
- keywords and scripts for HC performance testing setup - basic performance suite: operational data read - traffic script and keywords used in tests Change-Id: Ic0290be73a7c925ea2561f8cd2524c5cb83fcda2 Signed-off-by: selias <samelias@cisco.com>
2017-06-15CSIT-649 Add library for creating lxc containerPeter Mikus1-0/+23
Add RF library for manipulating lxc container Add python library for manipulating lxc container Change-Id: I02140aa879c7ebd40360d588ab1438c58cf560a1 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2017-05-02csit-validate-pylint-master/3731 for build 3731jan.hradil1-3/+4
pylint repairs deleted duplicated method from HcPersistence.py Change-Id: I426e6c15957c995ea10bd11ae2deac75b1cab922 Signed-off-by: jan.hradil <jan.hradil@pantheon.tech>
2017-04-25Vhost testsMatej Klotton1-32/+38
Change-Id: I03722afe13722941e084bc01161f7c2af30e3cb1 Signed-off-by: Matej Klotton <mklotton@cisco.com>
2017-04-18CSIT-545: Performance tests for SNATTibor Frank1-1/+1
- High level definition (HLD) - Low level definition (LLD) - Add keywords to set SNAT - Add tests according to HLD, LLD Change-Id: I7bf0b1870ac9c3adb36bb6590be9a3eb4ea8aa9a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2017-02-16Fix ssh prompts for centosMatej Klotton1-2/+4
Change-Id: I864f97b4559fc1d692bfba80266150a3d06dd6bb Signed-off-by: Matej Klotton <mklotton@cisco.com>
2017-02-16CSIT-514: Kill Qemu when it does not respondTibor Frank1-2/+7
Change-Id: Id13c7fd51e693a273637c80cb35a159434be4826 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2016-12-16Pylint fixesTibor Frank1-2/+2
- Fix PyLint errors - Fix comments in touched python modules Change-Id: I26db2d292a41969cf38b9b0bdd49c4fb15349102 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2016-10-10Ipv6 TEST fixesMiroslav Miklus1-4/+2
- Increase interactive terminal timeout - ip6 heapsize - performance lib fix Change-Id: Iac6beb041200c4b0c5cf29dceb5113df64c6c9b0 Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-10-05CSIT-176 Fix interactive SSH console deadlockpmikus1-17/+24
Change-Id: Iba291d6781238c0e880534195ff65a7b3c4bede1 Signed-off-by: pmikus <pmikus@cisco.com>
2016-10-04Fix pylint warnings in python librariesselias1-12/+25
- no functional changes - fixes 80+ PEP-8 violations Change-Id: Icf414778ec40d5cb44364fa69a876f9a1870c3c7 Signed-off-by: selias <samelias@cisco.com>
2016-10-04Interactive terminal fixesMiroslav Miklus1-1/+1
- remember if execution timed out and do not try to send "quit" on close - increase timeout of vat command execution from 10s to 60s Change-Id: I2b96a6dce1220eff45002276d1fe8a771fca2205 Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-09-29Enable paramiko ssh keepalive (10s)Miroslav Miklus1-0/+2
Change-Id: I1e46ad76f033d239ad29d3b6fe00011a4444453d Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-07-25SSH timeout problemMatej Klotton1-34/+66
In case of large output or stderr output the SSH proces got stucked, because of output buffer fills. Fix with reading from buffers. - increase traffic script default timeout - add reconnect to ssh Change-Id: Ie07107c6ab0fb19c9361d7390467b9b693fe2fdb Signed-off-by: Matej Klotton <mklotton@cisco.com>
2016-06-28Use interface key instead of interface name.Miroslav Miklus1-1/+1
JIRA: CSIT-141 Change-Id: I75cef6d570ab45ea9c4af838b6bf68cefc7c1a91 Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-04-22Reformat python libraries.Matej Klotton1-48/+54
PEP8 reformat fix typos docstrings reformat Change-Id: Ic48ba4e06490630808b8e2ab1ab0b046ec7eeed7 Signed-off-by: Matej Klotton <mklotton@cisco.com>
2016-04-08Add Vagrantfile for local testing.Stefan Kobza1-5/+12
Vagrantfile contains 3 VMs as of now, 2 DUTs 1 TG, with these notes: - login is csit/csit - by default provision script installs all deb packages from the dir where Vagrantfile is - developed for, and only tested on vbox (someone can pick up vmware) - All nodes have 1 shared mgmt network: 192.168.255.0/24 - hosts have these IP addresses in host-only network TG : 192.168.255.100 DUT1 : 192.168.255.101 DUT2 : 192.168.255.102 - script created to download MAC address information - PCI addresses are always the same for vbox (not sure about vmware) HOWTO (will create a wiki page once one is created for CSIT project): - copy Vagrantfile to separate dir on host - vagrant up --parallel sit-back-and-relax - from VM that has access to the same host-only network (192.168.255.0 above) - copy your ssh-key to csit@192.168.255.{101,102,250} using ssh-copy-id - cd ${csit_dir} - virtualenv & pip as in README - export PYTHONPATH=${csit_dir} - resources/tools/topology/update_topology.py -v -f -o topologies/available/vagrant_pci.yaml \ topologies/available/vagrant.yaml - pybot -L TRACE \ -v TOPOLOGY_PATH:topologies/available/vagrant_pci.yaml -s \ "ipv4" tests - see tests results Change-Id: Ic27626605a9c820bca977b38f4e8ca37d1504ff5 Signed-off-by: Stefan Kobza <skobza@cisco.com>