aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/python/QemuUtils.py
AgeCommit message (Collapse)AuthorFilesLines
2022-04-07Revert "feat(core): Add dpdk log level"Peter Mikus1-2/+2
This reverts commit 22bfd741614145d86751df2d61ff77af9db63627. Reason for revert: OMFG this patch is causing 90% perf degradation on AWS Change-Id: Id081f61ae849678b6b877fbf03309c4b1b0c6df6 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2022-03-21feat(core): Add dpdk log levelPeter Mikus1-2/+2
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I8af9bae81ec83a3195ae83febd978882ebc2474d
2021-12-01feat(Performance): Add 2M/1G hugepagespmikus1-6/+20
+ Add ability to switch between hugepages. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I84d8eae28ed414a32e5ba82e6c9ed10d7f0ef9cb
2021-06-10FIX: Pylint reducepmikus1-0/+2
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I909942dbb920df7f0fe15c0c92cda92c3cd8d8ad
2021-06-10Dpdk in VM: Increase num_mbufsVratko Polak1-2/+2
Since switching to rxq ratio 1:1, the previous value was not enough for 4c vhost tests. + Set the 1:1 rxq ratio also for DPDK (outside vhost) tests. - Still not sure why those tests were not failing. + Set the 1:1 rxq ratio also for VPP container tests. Ticket: CSIT-1783 Change-Id: Id9d136a638eb212a8105a6288e7847f3c35fefb0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-04-19DPDK bumppmikus1-4/+1
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I6edd980cb72111a008ae7fa19e1a4df279febdb2
2021-02-18Framework: Qemu 4.2 compatibility fixpmikus1-1/+0
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I8692b53c02773dbe5a5e02f959dfed8599cb8a4d
2021-01-04GSO: TAP/VHOST use casepmikus1-25/+70
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ibafaaca4438587284d1e255c764f0701002941e8
2020-10-29tests: Add GSO enabled testsMartin Balaz1-5/+13
Change-Id: I409b060f64ae7c6787448ae519fd76d8384e9ffb Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
2020-09-22Framework: Heapsize configurationpmikus1-6/+6
- https://gerrit.fd.io/r/c/vpp/+/28829 - https://gerrit.fd.io/r/c/vpp/+/28896 - ip6 to be added later Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I7b03b8ace5c5c7582d4a2f66bfd788a3ec80a916
2020-09-03Framework: Code aligmentspmikus1-3/+3
+ Unifying code structures - To easily plug another DUT + New PCI PassThrough templates + Improved perf stat on cores allocated in test. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I325f17b977314f93cb91818feddfddf3e607eb8a
2020-08-24Framework: Qemu alignmentspmikus1-10/+13
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2781e85f44acffb4f8d7f02326ba2ca668dad0c5
2020-08-18Framework: Qemu alignmentspmikus1-8/+21
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I20098fca8fb513accef3edc9a72bfd3c56bf9be2
2020-08-17Framework: Profiles loadpmikus1-113/+137
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Iab84aff31a23bb9d8e1165f5314004803fd8a501
2020-05-15Performance: Tests with virtio driver in VMPeter Mikus1-7/+8
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I20e01dfe83a961dc8202d33783a678d38e71cff2
2020-04-23Performance: DPDK refactorPeter Mikus1-2/+15
+ Rework BASH scripts (more code to python). + Move BASH into libraries. + Allows RDMA usage. + Fix 9000B tests. + Rename confusing l2fwd -> testpmd. + Fix suite setup. + Fix PCI whitelist to not accidentally pickup wrong interface. + Fix deprecated DPDK cli arguments. - MLX5 jumbo are disabled on NIC (i will increase separately). https://jenkins.fd.io/job/csit-dpdk-perf-verify-master-2n-clx/6/console (l3fwd still broken) - MLX5 IMIX seems to be some TRex issue with IMIX for mlx5 (i will handle separately) Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I31d1b67305fa247cb5e1f57e739d3ef30dc1a04b
2020-03-17FIX: VHOST vppPeter Mikus1-13/+7
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Id0d41a2e62ee6cce212fb2c09a0357d794426a00
2020-02-14Aarch64 fixes for vfio-pci in kernel VMjuraj.linkes1-11/+21
- Add enable_unsafe_noiommu_mode for aarch64 architecture to kernel VM init script - Add binding of pci device through /sys/bus/pci/drivers/vfio-pci/bind - Minor refactor to enhance readability of differences between architectures Change-Id: Ib0ec35bfae054582d9ffcf7a82f8f8b3e3e24856 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2020-02-14Ansible: Update DPDKPeter Mikus1-1/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia5c1e40b952c14775f2f038d3a8a293335172f47
2020-01-20FIX: Hugepages should be detected also from overcommitPeter Mikus1-1/+1
Change-Id: I36ec9d61f775791ad0632a579c4921889d1f5acf Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak1-2/+2
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-07Vhost: Add GSO optionPeter Mikus1-4/+9
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Iba9af71c293645b480203af72fcf940cbe9ccb3a
2019-11-28Python3: resources and librariesJan Gelety1-269/+270
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-15Reduce disabled pylint to address them not hidePeter Mikus1-19/+6
+ Fixed Qemu - Broad Exception should be addressed - Policer is about to be refactored - Drop search to be refactored Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I97eb992497ff6334057bf82a0f413387706e17fd
2019-08-15FIX: VM startup after socket movePeter Mikus1-1/+0
+ Still workaround for VPP bug not creating dir at startup Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I8b87ac70e5f85d538a25058dcb5453f89e21498f
2019-08-13FIX: Vhost tests in latest VPP nightlyPeter Mikus1-0/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I26ec27b00e5cd48143a2389f2ff41ed876a7a013
2019-07-23FIX: Vhost optimizationPeter Mikus1-5/+3
- Testpmd 9000B - Disable fsck - Remove redundant init mounts Change-Id: Ia063520d0a206e9eee831a5cc692e32baf5a92c8 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-17Use PapiSocketProvider for most PAPI callsVratko Polak1-0/+1
Ticket: CSIT-1541 Ticket: VPP-1722 Ticket: CSIT-1546 + Increase timeout to hide x520 slownes of show hardware detail. - Install sshpass and update ssh client in virl bootstrap. + Added TODOs to remove when CSIT-1546 is fixed. + Enable default socksvr on any startup conf. + Improve OptionString init and repr. - The non-socket executor still kept for stats. + Remove everything unrelated to stats from non-socket executor. - Remove some debug-loooking calls to avoid failures. TODO: Introduce proper parsing to the affected keywords. + Reduce logging from PAPI code to level INFO. - Needs https://gerrit.fd.io/r/20660 to fully work. + Change default values for LocalExecution.run() + Return code check enabled by default. Code is more readable when rc!=0 is allowed explicitly, and the test code will now detect unexpected failures. + Logging disabled by default. Output XML is large already. Important logging can be enabled explicitly. + Restore alphabetical order in common.sh functions. Change-Id: I05882cb6b620ad14638f7404b5ad38c7a5de9e6c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-17Refactor VPP Device VM vhost testsjuraj.linkes1-32/+46
* replace the current VM image with kernel img * rework keyword usage to make it consistent with performance tests * remove resources/libraries/robot/shared/qemu.robot as it's not used anywhere anymore Change-Id: Ia5bc19e9e6ed9af031e4d9b5c0c89431fb49fd33 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-06-09CSIT-1521 Convert NestedVM testcase into KernelVMPeter Mikus1-0/+1
Change-Id: Ic493253dc29dad4fc7e6541e4b839ca5898a6a40 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-05-06CSIT-1493 VPP restart handling codePeter Mikus1-1/+1
Change-Id: Ibe52125089f39e0ff17ec607a3ed00c61d52ab8c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-17Introduce OptionString for handling command lineVratko Polak1-149/+110
+ Convert DpdkUtil to use it. ++ Rename args to kwargs where needed. ++ Fix errors in docstrings. + Also convert and QemuUtils to use it. ++ Minor formatting edits to save space. + Add disconnect parameter to some ssh.py functions. ++ ssh.SSH.disconnect() tries to work without argument. + Exec functions in ssh.py accept OptionString commands. Change-Id: I82da71c568d120c283544c90242993fc76e9e83a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-04-15Remove obsolete QEMU installationPeter Mikus1-41/+5
+Unify QEMU parameter to Contants. Change-Id: I95adfc97dd7008c19e60ba3fa8668818b503bee8 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-12Convert existing nf_density testpmd-mac into KernelVMPeter Mikus1-5/+71
Change-Id: I9ec1701d9012de59cc42cc5f9a4744f01695ff9a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-07FIX: Qemu set affinity bugPeter Mikus1-39/+30
Change-Id: If1df58628b2150b1ea3f2f56b1e3d442176b0750 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-03-20CSIT-1386 KernelVM - Part IIPeter Mikus1-74/+186
- Implementation of KernelVM (ThinVM) - nf_density tests for KernelVM with VPP as VNF. Change-Id: Ife5c2e1ab419c55cbcd442792f940db3a41da471 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-03-18CSIT-1386 KernelVM - Part IPeter Mikus1-435/+330
- Refactor QemuUtils Full refactor of QemuUtils to prepare for thinner VMs. Change-Id: I171ba11a9e6e9faec582bd58df6819598b0ed0b5 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-03-04Rename to Constants.pyVratko Polak1-1/+1
Motivation: Make the constants available also for Robot. Restraint: Robotframework user guide says: Because variable files are always imported using a file system path, creating them as classes has some restrictions: Python classes must have the same name as the module they are located. Change-Id: I638ef3fe045db132e366de2e2699638b8637e45e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-01-10VPP_Device - add baseline tests - part IIa)Jan Gelety1-2/+5
CSIT-1372 Add following baseline tests to VPP_Device: - ip4 eth2p-ethip4-ip4base-eth-2vhost-1vm-dev - ip6 eth2p-ethip6-ip6base-eth-2vhost-1vm-dev - l2bd eth2p-eth-l2bdbasemaclrn-eth-2vhost-1vm-dev - l2xc eth2p-eth-l2xcbase-eth-2vhost-1vm-dev + fix doc strings in all vpp_device tests Change-Id: Icc5ed801f3e8f6dedaa611779bd86fc20102fa74 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-01-02VHOST vectors.Peter Mikus1-4/+7
Change-Id: I5e17c5ab5cb99af0ac3d063ffc911bf95457d3e4 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-26Fix jumbo framesTibor Frank1-1/+1
Change-Id: I8aa557fe75133d20eef9832e314357af71f1d19a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-08-20Refactor VHOST codePeter Mikus1-291/+239
CSIT-1164 Create VM vhost tests for 2-node topology CSIT-1173 Implement VM vhost KWs for 2-node topology - Cleanup QemuUtils.py Library (PEP8, some TODOs, readability, multi queue, ring size, qemu version detection) - Cleanup VHOST KWs perf (Unify running QEMU on N-node topology) - Add ability to configure Queues, Hyperthreading, automatically compute number of CPU for VM/Testpmd and RXQ for Testpmd workload. - Partial cleanup of configuration KWs (l2xc, l2bd) for 2-node - Create IPv4 2-node KW Change-Id: I8aae1355bafa651de715cd438fe706c443ea1d88 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-05-04Fix various pylint 1.5.4 warningsVratko Polak1-2/+2
+ DUTSetup.py:424 Else clause on loop without a break statement + InterfaceUtil.py:400 Else clause on loop without a break statement + QemuUtils.py:564 Wrong continued indentation + SetupDPDKTest.py: Locally enabling broad-except + VatExecutor.py: Catching too general exception Exception + ssh.py:95 No exception type(s) specified. + HTTPRequest.py: Tolerate HTTPCodes.OK + multiple: Drop ":returns: None" from docstrings. There are still several warnings present: - R0902(too-many-instance-attributes) - R0912(too-many-branches) - R0913(too-many-arguments) - R0914(too-many-locals) - R0915(too-many-statements) - R0401(cyclic-import) And there are multiple blocks of similar lines, mainly across various Setup*Test.py files: - R0801(duplicate-code) Change-Id: I582575cb52b85d69d268e6374852f6e74bb71052 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-04-25Fix warnings reported by gen_doc.shVratko Polak1-11/+12
+ Docstring warnings fixed. + Multiline param descriptions indented by 4 spaces. - Except the PacketVerifier.py one - I have tried several quote-like blocks, nothing works. - Rst warnings not fixed. - How can I fix them? They refer to temporarily created files. + Other improvements: + Python lines no longer than 80 characters. + :return: -> :returns: + Notes before params. + :raises + closing colon after exception class. + Description is a sentence. + Present tense in conditional sentences. + Bumped copyright year in edited files. Change-Id: I462c194eeecb666dc146e26858486a07c990be9b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-03-26Optimize Qemu installation to speed up vhost testsPeter Mikus1-0/+4
Currently Qemu is being installed if there is a change of qsz parameter between tests/suites. Qemu is installed always into the same directory. This patch changes the default behavior to install qemu over and install pathced version to separate directory. It also disables force install. Change-Id: I0d7493a02b026a6ae4a5ea8bacf54656de9db567 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-20Add max-pkt-len parameter to dpdk testpmd startTibor Frank1-3/+10
Change-Id: I114c187f007c10223502669c2ba85c63d09cd84a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-02-23FIX: Qemu path overridePeter Mikus1-8/+12
Currently there is bug that Qemu path is not correctly composed and cannot be properly overriden due to race condition in code. This patch is fixing functions to properly work with path. Change-Id: I276458d2d23c7a14441a4de4a56eff6431242ab9 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-01-10add new topology parameter: archGabriel Ganne1-4/+9
if unset, arch variable will default to "x86_64" * Note on "arm64" vs "aarch64" debian-based uses arm64 rhel-based uses aarch64 qemu binaries of both distribs uses aarch64 dpdk uses arm64 vpp uses aarch64 python machine modules uses aarch64 => prefer aarch64 to use the same nomenclature as vpp * add ARCH argument to: init_dpdk.sh, install_dpdk.sh, run_l2fwd.sh, install_tldk.sh, run_tldk.sh. default to x86_64 converts "aarch64" if needed for dpdk naming convention * fixed terminal end detection to allow "~]# " add dut node arch as param to all robot set bin calls * add --target-list flag to qemu_build.sh defaults to x86_64-softmmu * add arch flag to all the topology files * topologies/available/ (and example file) * resources/tools/virl/topologies/ * set _qemu_bin path using node['arch'] in qemu_set_node() Change-Id: If46d88d064d213d3e4c6fc584bb8e0d4b6428cb8 Signed-off-by: Gabriel Ganne <gabriel.ganne@enea.com>
2018-01-02FIX: Force kill QEMU in case of failed initializationPeter Mikus1-7/+6
Change-Id: Ib9c243b1bd379cf33a2b7747b7f97a8533000d60 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2017-12-12More reliable connection with nested qemu image.Thomas F Herbert1-2/+16
Under some circumstances on a busy server, the qga channel between the host and hte virtual guest has some garbage characters that cause the communications to fail. This has been seen on Centos 7.4 on a slow or busy server. This change sends a flush to qemu guest via the qga socket at 5 second intervals until there is a non empty response from guest before sending a guest ping. Change-Id: I6f21c205b289169aee9d6a4072ad4e6bafafa76f Signed-off-by: Thomas F Herbert <therbert@redhat.com>