aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2016-04-22Bridge domain tests and user keywords refactor.Jan Gelety4-81/+290
- BD tests changed to behavior-driven-test style - added documentation to BD tests - added documentation and examples to BD user KWs Change-Id: Ifad45df3ef0bb55f5295363a264fe5f4e54f5762 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2016-04-22VatExecutor updateMatus Fabian2-0/+38
Strip non-JSON data from VAT terminal output Add method for VAT command retval verification Change-Id: Ibe68020393cb1a700ad048dc78078f71064707b4 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-04-22Performance test casespmikus6-127/+443
- JIRA: CSIT-10 - Add 1518B and 9000B test for multicore tests - Fine tune threshold for short tests based on previous benchmarks - Fix SI units Change-Id: I5c4657dfd881a24b2e40c91ea80a979003754258 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-21Topology schemaTibor Frank2-2/+15
- fix Honeycomb part of the topology schema - add topology file checking using pykwalify before tests are started Change-Id: I907d41b66f5660a2ab4de8857312e9277052bae6 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2016-04-20Remove horizontal whitespace (a line) before [Command_start_exec] tag.mkonstan1-1/+0
Change-Id: I51e5616cb45b3c67e82153cc4c0f42ec9c5d75d7 Signed-off-by: mackonstan <mkonstan@cisco.com>
2016-04-20Include tags to CSIT performance boostrap scriptpmikus1-8/+44
- include tags to CSIT performance boostrap script - allows future expansion of selective test execution Change-Id: I3870f52d86ff39234eb724500d3871fed0cbdb30 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-19Make VXLAN test dot1Q as noncritical for now.Stefan Kobza1-0/+1
Change-Id: I0d41aad3fc1f96b58b83c6f0aa5a0e7606868ee4 Signed-off-by: Stefan Kobza <skobza@cisco.com>
2016-04-19Make dut_setup.sh verbose; add sleep after vpp restart.Stefan Kobza1-40/+31
- there's a race condition between VPP service restart and vpp_api_test execution; this patch addresses it with (for now) sleep after service vpp restart - above is just a workaround until VPP provides us with a notification of readiness.. Change-Id: I1f92c1d86e4f90c9432ea57c1747629f619f003e Signed-off-by: Stefan Kobza <skobza@cisco.com>
2016-04-17Multicore VPP setup for performance testingpmikus9-77/+596
- add multithread TAGS documentation - add methods to VppConfigGenerator for RSS configuration - create KW for multithread setup - create sample test case using multithread vpp setup - add Documentation into TCs Change-Id: Id40862490d49380dc76d1d3ce39314603f983fd3 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-15Update Qemu libraryMatus Fabian3-12/+110
Incorporated changes from nested VM on VIRL. Change-Id: I76ceeb2dde635c2cf0d3a53d29bf24fed72a5437 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-04-15Change the performance result messagepmikus1-6/+21
- change the result message of performance search to more verbose level (rate per stream, rate total, throughput in Gbps) Change-Id: I7562d6d33c525654131c75ebcc776cba438f68f4 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-15Update of bootstrap.sh file.Jan Gelety2-9/+10
- updated vpp build version - VIRL image switched to double-ring-nested topology - removed exclusion of 3_NODE_DOUBLE_LINK_TOPO test cases - included test cases with tags: - VM_ENV and 3_NODE_SINGLE_LINK_TOPO - VM_ENV and 3_NODE_DOUBLE_LINK_TOPO - added missing tags to TC VPP reports interfaces in Bridge Domain test suite Change-Id: Ia6a7a54ef9ad6245835b2762c303935fe39f4079 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2016-04-13VXLAN test with dot1q tagging.Matej Klotton11-89/+262
Change-Id: I3dbd12983736e338d757c580570d91680aedd83f Signed-off-by: Matej Klotton <mklotton@cisco.com>
2016-04-13Move methods from topology.py to more appropriate place.Matej Klotton12-430/+348
Change-Id: I2612a9466095b3d79a4a9d938fcdbf9f93133f69 Signed-off-by: Matej Klotton <mklotton@cisco.com>
2016-04-13Honeycomb setup and utilsTibor5-242/+412
- re-implement HTTPCodes as IntEnum rather then dictionary - add methods to manipulate data using honeycomb - get, set, delete - change the name of url file from vpp_version.url to oper_vpp_version.url - improve checking of startup and shutdown state of honeycomb - PEP8 fixes - add docstrings in all modules and classes - move logging to the lowest possible level - improve logging in exceptions - add method exec_command_sudo_log to resources.libraries.python.ssh module Change-Id: I54e0c6b45313e3a3c11bafa475488ae2b1e605c2 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2016-04-13documentation: long_xconnect test suite documentationMiroslav Miklus1-1/+16
Generate testdoc by using: python -m robot.testdoc long_xconnect.robot perf_long_xconnect.html Change-Id: I12d83de4abb1af4d94e6fd78d05c46018ee5d7d7 Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-04-13Add keywords for Honeycomb testsselias4-20/+80
-add interface dump and bridge-domain dump to Vat command templates -add low level keywords to implement vat dump commands The resulting data dumps will be used in Honeycomb testing, to verify data retrieved through Honeycomb API. Change-Id: I2a913eaf23aa13d7223576220681821007672812 Signed-off-by: selias <samuel.elias@pantheon.tech>
2016-04-12Boostrap verify installation directory removalpmikus1-1/+1
- Boostrap verify installation directory removal Change-Id: I091240f8a3f7e5074cdc2f069837963e4c8bd192 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-12Bootstrap verify performance fixpmikus1-1/+1
- Bootstrap verify performance fix Change-Id: I8f77c26aef805f5223d03e2a0bc4a97f88195ed8 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-12Bootstrap verify performance fixpmikus2-20/+36
- fix installation directory create and remove Change-Id: I0acd47b886e869958a2075889af3bbe166d7808e Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-12Bootstrap verify perfomance fixpmikus1-1/+1
- fix test tags to lowercase Change-Id: I6e01a677f15843105106300097b4d0b23e2cba5d Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-12Update of topo_installation filepmikus1-1/+1
- deb package path fix Change-Id: I94065581886c4d5b9f15ced18857ddfcde442153 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-11Fix cosmetic issues in VIRL topologiesCarsten Koester3-22/+17
Fix cosmetic issues in VIRL topologies: - Replace double whitespace by single whitespace in XML - Remove reserved but unused IP address allocations from VIRL config - Modify "start-testcase" script to handle links without IP address Change-Id: Ie447bd91172df61b30ebb8650b56bdfca0f25fd4 Signed-off-by: Carsten Koester <ckoester@cisco.com>
2016-04-11Add VIRL double-ring (load-balancing) topology with nested VM supportCarsten Koester2-0/+393
Change-Id: Idcf173b1711076a5d7e4af8fe46e302b9eae4ff6 Signed-off-by: Carsten Koester <ckoester@cisco.com>
2016-04-11Combined DropRateSearchpmikus3-147/+179
- combined search algorithm using linear and binary search - add KWs into performance library for binary and combined search - remove short_bridge_domain_binary.robot as binary and combined search KWs are in library for optional use - PEP8 fixes Change-Id: I4a6c3c07c82db4f855fb93214b0532ee0c783e0a Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-08Add Vagrantfile for local testing.Stefan Kobza7-17/+440
Vagrantfile contains 3 VMs as of now, 2 DUTs 1 TG, with these notes: - login is csit/csit - by default provision script installs all deb packages from the dir where Vagrantfile is - developed for, and only tested on vbox (someone can pick up vmware) - All nodes have 1 shared mgmt network: 192.168.255.0/24 - hosts have these IP addresses in host-only network TG : 192.168.255.100 DUT1 : 192.168.255.101 DUT2 : 192.168.255.102 - script created to download MAC address information - PCI addresses are always the same for vbox (not sure about vmware) HOWTO (will create a wiki page once one is created for CSIT project): - copy Vagrantfile to separate dir on host - vagrant up --parallel sit-back-and-relax - from VM that has access to the same host-only network (192.168.255.0 above) - copy your ssh-key to csit@192.168.255.{101,102,250} using ssh-copy-id - cd ${csit_dir} - virtualenv & pip as in README - export PYTHONPATH=${csit_dir} - resources/tools/topology/update_topology.py -v -f -o topologies/available/vagrant_pci.yaml \ topologies/available/vagrant.yaml - pybot -L TRACE \ -v TOPOLOGY_PATH:topologies/available/vagrant_pci.yaml -s \ "ipv4" tests - see tests results Change-Id: Ic27626605a9c820bca977b38f4e8ca37d1504ff5 Signed-off-by: Stefan Kobza <skobza@cisco.com>
2016-04-05Add tag support to boostrap-verify-perfpmikus2-7/+18
- support tags in vpp-verify-perfomance-* jenkins jobs Change-Id: I7e3abfe9e552a3f2fa9ebf95df9b48bcbae3561a Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-05Add "Stop and Clear QEMU" KWMatus Fabian1-1/+14
Change-Id: Ib1b3907cd30c20c89283bd20e84ed8b829540ca4 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-04-05Setup and check honeycomb on all DUTsselias12-2/+807
- methods implementing HTTP requests (PUT,GET,POST,DELETE) - methods for parsing HTTP responses - methods for honeycomb setup on DUT - updated constants.py - keywords for honeycomb setup and communication - simple honeycomb sanity test (not enabled for jenkins job runs) Change-Id: I589f0ca56cc01072b92fe9363aed16a4098aee40 Signed-off-by: selias <samuel.elias@pantheon.tech>
2016-04-05Add new tag for VPP-VM testing enviromentMatus Fabian1-0/+3
Change-Id: I2bdca4d8451bec8215b2d8f637809f8781e2e9d2 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-04-05Add VIRL topology files and template for nested virtualisationCarsten Koester2-0/+359
Change-Id: I57dbe4f179b3cfd3f51c9dec503ca45a5c0521fd Signed-off-by: Carsten Koester <ckoester@cisco.com>
2016-04-04Change send_ip_icmp recv timeout to 2secMatus Fabian1-1/+1
Traffic script send_ip_icmp recv timeout 10 sec is too long and execution rather end with ssh timeout. Change-Id: Id4df679799191fd03e2987f068f3ea9e7bf12fdf Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-04-01Add get_vhost_user_if_name_by_sockMatus Fabian1-0/+16
Change-Id: I1d7f2cdfcf29763ce94417c98d363021f1fb27a1 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-04-01Long perf testspmikus8-170/+255
- added new long tests for ipv4 and xconnect - modified long bridge domain test - added 64B, 1518B and 9000B frame sizes Change-Id: I88f75c375345f2d708df3fd24dd50d77bc63c495 Signed-off-by: pmikus <pmikus@cisco.com>
2016-04-01Bootstrap to test perf of proposed patch vpp-csit-hw-perfpmikus2-0/+139
- Based on #570 Lab reservation tool - Get the latest build of VPP - Make reservation on testbed by using existing topology files - Copy and install VPP image on all DUTs - Run performance tests Change-Id: I8034fd9c284cbe9a68e0c9116d0826965464b5f9 Signed-off-by: pmikus <pmikus@cisco.com>
2016-03-30Add linux_add/del_bridge and modify vpp_add_l2fib_entryMatus Fabian1-4/+39
linux_add_bridge - bridge two interfaces on linux node linux_del_bridge - delete bridge from linux node vpp_add_l2fib_entry - param interface should be interface name or sw_if_index Change-Id: Ia8030e24a0afe088df0dcb6c65a85ed341224206 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-03-30BestN/WorstN DropRateSearchpmikus1-29/+132
- implementation of Best-of-N/Worst-of-N result evaluation with max attempts per one rate step Change-Id: Ice1d6e93807d90761462c4df601cbe068011159a Signed-off-by: pmikus <pmikus@cisco.com>
2016-03-29Bootstrap to test csit-master against VPP release.Jan Gelety2-4/+176
- get the version of the latest VPP build - download and install the latest VPP build on nodes - run three times all test suites to check if the VPP build can be used for csit development and current csit master can be pushed to master-verified branch (see patch https://gerrit.fd.io/r/590) Change-Id: I0ad2e9fe1620cc18ccc11dce2d107177cf97ff27 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2016-03-29Fix of bootstrap.sh for vpp-csit-verify-virl jobJan Gelety1-23/+1
- moved rm -f *.deb to else statement to avoid deletion of deb packages when boostrap.sh is called from include-raw-vpp-csit-functional.sh Change-Id: I4a224a1fc332d50a27347e77dbeacbfbc98677c9 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2016-03-24Setup and run QEMU VMMatus Fabian1-2/+358
Change-Id: I22640caa9f13be09d2a837c6332d7b86cef77efc Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-03-24Add node type VM to set_interface_stateMatus Fabian1-1/+1
Change-Id: Ib70de73eec7c015f328f877fa34ead15e27eb254 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-03-24Binary DropRateSearchpmikus2-4/+134
- binary search - short perf bridge_domain with binary Change-Id: I50538738b356cc8fef4cfcb4974c43c5123827d1 Signed-off-by: pmikus <pmikus@cisco.com>
2016-03-21Lab reservation toolMiroslav Miklus2-2/+117
This tool provide client-based reservation system of tesbeds by using existing topology files. Change-Id: Ic0be944c77c9d6aa28e3fa7fa69e31d19861416f Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-03-18Remove apt-get call to install dependencies.Carsten Koester1-15/+0
Change-Id: I309a13bc857b89f1f342d659ab80b70d7a5fc686 Signed-off-by: Carsten Koester <ckoester@cisco.com>
2016-03-18Add SSH disconnectMatus Fabian1-0/+11
Change-Id: Ib0efec083aa9ee33b7d15680e5a15989827b8e57 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-03-18Fixed SSH exec_command timeoutMatus Fabian1-15/+6
Change-Id: I8cd2fcfbe774bd651ed57be0129c32d6ba68adcc Signed-off-by: Matus Fabian <matfabia@cisco.com>
2016-03-17DropRateSearch libraryMiroslav Miklus8-68/+480
- linear search - t-rex DropRateSearch implementation - long perf bridge_domain test - introduced PERFORMANCE_SHORT, PERFORMANCE_LONG test tags Change-Id: I497b72f3e6d58a67ca5a386403d1e84dcf433ec4 Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-03-14tools: vpp download and installation scriptMiroslav Miklus1-0/+40
Download and install 'RELEASE' version of vpp packages Dependecies: curl, dpkg, md5sum Change-Id: I894c8181125b84df3057afc092ed01c0a837321a Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
2016-03-14Add vxlan tests using xconnectPatrik Hrnciar7-26/+89
Change-Id: I190689d5a624f1c876df90ce29b05a2b0b8a2df4 Signed-off-by: Patrik Hrnciar <phrnciar@cisco.com>
2016-03-14Add VM to NodeTypeMatus Fabian1-0/+2
Change-Id: Ic3a313837f48a1c7bd1c70876c82799ac1994f2f Signed-off-by: Matus Fabian <matfabia@cisco.com>