aboutsummaryrefslogtreecommitdiffstats
path: root/tests/vpp/perf
AgeCommit message (Collapse)AuthorFilesLines
2020-03-12rls2001 perf: fix hoststack test packet sizesDave Wallace8-24/+25
- TCP packet size is 1460B not 9000B - QUIC packet size is 1280B not 9000B Change-Id: I6604a74fa533db4ac782782c85ea54038688627a Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-03-10Make RXQs/TXQs configurablePeter Mikus508-11/+1535
Change-Id: Ib30bc4697fcba93a6723ee492a59a0523425f623 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-03-09Fix on 2n suiteVratko Polak1-1/+1
Wrong tag said it is 3. Change-Id: I3ea6ffbec3e38a11b721318266bcb2ea451d2d5a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-03-03perf: Clean up Hoststack testsDave Wallace10-46/+347
- Update test names with clients/streams - Convert test results to JSON output * iperf3 results include bits_per_second * vpp_echo results include both client and server output which includes time in seconds and rx_data/tx_data in bytes which can be used to calculate the average bits per second. Tx and Rx data will always be the same: BPS = (client tx_data * 8) / ((client time + server time) / 2) - Fix WRK test results data formatting errors Change-Id: Ie2aeb665e3cc0739b16f97ba2628eebe6e041d22 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-02-25FIX: check if t-rex is running at test setup of all perf testsJan Gelety498-703/+703
Change-Id: I9af632035a1415666b2470c62a41d1b6acbf33c8 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-22FIX: policerJan Gelety2-4/+10
Change-Id: I23910b74c8720245b43067ac8c68880291b4b1c2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-02-04Ipsec: Use new plugin name in reconf testsVratko Polak40-40/+40
This edit is not suitable for rls2001. Change-Id: I18ea22346d5996e78034f35d74a87c125010a146 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-02-04Add more reconf tests, for IPsecVratko Polak41-1/+6101
- Not adding nf_density tests. - Not adding hardware ipsec tests. - Not adding -policy- tests. - Using old crypto_ia32_plugin.so plugin name. + Suitable for cherry-picking to rls2001. Change-Id: Ibf44d6d91e2afa2320637ecd9eb69d5d5dc364aa Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-31T-Rex: CPU pinningPeter Mikus1-2/+2
+ Detect NUMA + Pin based on numa location Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ife350f8c70e5437ac7c1413c7753f2a2f62777d9
2020-01-30perf: Add hoststack NSIM+LDPRELOAD+IPERF3 test suiteDave Wallace1-0/+66
Change-Id: Ia7a876b1aa240676e1f2d23618c1d4e09ead14f0 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-29perf: QUIC transport hoststack test suiteDave Wallace1-0/+74
Change-Id: I73f4be7ea315c7a5dcce46e1bd3034bcb0a97ee2 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-29IPSEC: Change plugin namingPeter Mikus100-100/+100
https://gerrit.fd.io/r/c/vpp/+/24574 Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ic2b7c925ceba1e16de87c64003ecceeea69c681c
2020-01-28Ipsec: Unify first line of Local Template docVratko Polak62-94/+90
Seeing differences when diff-ing between suites is distracting. + Bump copyright year, even for files with no change. Change-Id: Iaca79647821dd8233bdbe6b0ac8b14fdb04060a8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28perf: hoststack iperf3 test tuningDave Wallace1-18/+11
Change-Id: I53425f57fe9ecef9cff2c94642cc7cb24537a961 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-28Fix two auth_alg valuesVratko Polak2-2/+2
Change-Id: I0e85fc958779df3d5dbacf1ad1e3898268a832ec Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28Update overheads for IPsec CBC testsVratko Polak32-64/+64
Updated to the values as seen in packet trace. Even if VPP creates wrongly sized packets (compared to RFCs), the overhead should correspond to the actual packet size present, in order to correctly prevent linerate on DUT-DUT link. The new overhead values are 62 (256SHA) and 78 (512SHA). The GCM value is already correct, at 54 bytes, so density tests are ok. - The lispgpe test is not updated, as it currently fails. We will update overhead there in (or after) 24578. Change-Id: I5cc6920205f37ddc80e76804fabd90b67174addf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28FIX: dot1qip4vxlan testsPeter Mikus24-24/+24
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ieb6e6010108987f99f55730149e3e4b7f1a7fc21
2020-01-18FIX: VTS testsJan Gelety3-6/+6
Change-Id: I594d248c58dcdcdeceea57af2dd25e2b2e08247f Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-23FIX: nfv_densityJan Gelety185-370/+370
- use correct osi_layer=L2 (so L2 spoofing check is switched off in case of avf driver) - add pci address information to eth interface in topology file - nfv_density chain_ipsec tests work only with DPDK in current implementation Change-Id: I233c6e5634a14581960c7459b87f11fcee8365bd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-17perf: add TCP Iperf3+LDPRELOAD test suiteDave Wallace1-0/+71
Change-Id: Icff49fb31cce342a2a4ae799e844ec91f9e5e366 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-10Trending: new daily setJan Gelety42-353/+777
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Support suite tags in autogenVratko Polak457-0/+457
+ Include a script to add suite tags to many suites at once. + Add suite tags also to device tests (not covered by autogen). Change-Id: I514ee6178e22999b43460028fe2696738b012f04 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak197-3907/+184
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-15Hoststack perf infrastructure refactoringDave Wallace3-6/+10
- DUT only topology (hoststack test apps are co-located with vpp) - Make vpp app specific keywords generic where applicable - Add IP4 Prefix to topology file - Support running wrk in linux namespace - Refactor namespace cleanup - Remove redundant namespace creation code - Refactor test/keyword dirs: tcp -> hoststack - Add hoststack utility keywords - Refactor wrk suite setup/teardown - Update tests with recent perf infra changes Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-12-20VTS: Unify the testsPeter Mikus3-58/+36
- Converting to 2n as they were always 2n (with l2xc on 3rd node) - Removing KW and converting to layered approach Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ie349c50f72eb362815e7c5ede076d421ab386e76
2019-12-16FIX: NAT44 bugPeter Mikus1-1/+2
+ From trending ... Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0ebb5429f0ad731d42aa43855cd99ae73b1461fc
2019-12-13Python3: refactor ':FOR' statementsDave Wallace2-2/+4
Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I76835e3d3acf6955e328f30427f9dd0098947e41
2019-12-11Introduce VPP-IPsec container tests.Ludovit Mikula36-0/+5268
Change-Id: Ie64d662e81879bd52785e0188450d998bf056bda Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
2019-12-05Python3: IPv4 FIB scale tests with random flowsBalaji Venkatraman3-0/+400
Signed-off-by: Balaji Venkatraman <balajiv@cisco.com> Change-Id: Iadd9e8da655f3ccbd36167c1bbee37a774c3bc1f Signed-off-by: Balaji Venkatraman <balajiv@cisco.com>
2019-12-01Convert SRv6 from LXC to DockerPeter Mikus3-9/+9
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I7c948be40aaf2eb03434cec5548a89aaf0f1eca6
2019-11-28Python3: resources and librariesJan Gelety464-4145/+4148
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-11-14FIX: AVF IP6basePeter Mikus1-1/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ibc3350acab54100f03f540397bfcbd6e7455ea2c
2019-11-13Add vpp loadbalancer l3dsr/nat4 mode test suiteshaiyanx1.zhang3-1/+262
Change-Id: I03603108fe7128fb4bde4abddb6a0959dd5b5c73 Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
2019-11-11Add accidentally deleted NAT setup commandVratko Polak1-0/+1
One suite was missing "And Add NAT to all DUTs". Change-Id: I6e9c3de9f236a080cfe829db2d155a6f6ba3b3f4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-11-11FIX: 2lbvpplacpPeter Mikus4-4/+0
- These testcases are non standard so they need to be refactored to be able to use the interface layers Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I9400846ff04f1864babd6d7fa153f0c3708cb105
2019-10-30FIX: IPSecHWPeter Mikus8-16/+0
- IPsec proto is not for GCM Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: If6a9721574a72dacb4486c837100e4f2ed2bab19
2019-10-25Introduce pre-initialize driver layerPeter Mikus445-2324/+3011
+ This will help unify driver base differences between TCs. + Decrease amount of logic needed for suite generator. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I1e84ba361dc1e829f0612c58a61096e2633ce0c5
2019-10-19Tests: Introduce driver layerPeter Mikus445-437/+1294
+ Small bugfixes in various suites Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I051af0a6da7340fe3f617400370c3846ca5241ab
2019-10-09Add vpp loadbalancer maglev mode test suitehaiyanX1.zhang2-0/+145
Change-Id: I61555ba566efef0a2151db9a30bf7f5d9ccad1df Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
2019-10-04FIX: IPv6 AVF dot1qPeter Mikus2-2/+4
Change-Id: Id199eb774850153fb225434d4a3d194706fdfdb3 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-09-26CSIT-1471: Policer - VAT API to PAPIpremkumarv6-18/+36
Change-Id: I6c7eff8d9cbb196ccb684a45579ac59942351cfe Signed-off-by: premkumarv <preym17@gmail.com>
2019-09-17Update CRC list and support 21997/7Vratko Polak1-0/+6
- Attempt to repair IPsec LispGpe perf suite. + Collection name to reflect the current stable vpp. + Add messages found in CSIT L1 keywords. - Uncommented (instead of deleted) untestable messages. Reasons: - Honeycomb. - Messages used by unused keywords. + Listed reasons. Honeycomb not mentioned if both reasons apply. + Delete CRC items for commands not found in keywords anymore. + Add CRCs from .json.api files (as teardown is hard to execute). + Define and restore alphabetical order. + Add hints to find used API commands (not entirely reliable). + Move used commands to "cmd = " form so hints find them. + Argument to run_cli_command changed from "cmd" to "cli_cmd". + Except also struct.error where IOError is excepted. Change-Id: I61058dbe1e33296908aabd0c13433bb16cfa6adf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-13Add: AVF IPv6Peter Mikus18-22/+1326
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6cb4ef356047bdc341a70a4e7b8d0b3f54eb7f9d
2019-09-05FIX: Use vhost macs when testpmd_mac forwarding used in VMJan Gelety2-2/+2
Change-Id: I291c3dc852e7c1bde6fb6102a9168cfec9992682 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-09-02Add: Testpmd VM testPeter Mikus28-36/+2033
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Id39667515e304f4d92f230f82d3f1dbc1e22ac32
2019-08-27- Removed 9000B test cases from ipsec test suites.Ludovit Mikula65-780/+0
- Added new traffic profiles. Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech> Change-Id: Ie06b7a1cf4d9df403d098b7a495219fd95e5869e
2019-08-25FIX: Copy-paste typo in ↵Jan Gelety1-1/+1
2n-10ge2p1x710-eth-l2bd-6ch-48vh-24vm1t-vppip4-ndrpdr.robot Change-Id: Ibe231547e26d9b379554eec8ca6a9b4d7bc1224b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak24-24/+1944
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Update CRC list commentsVratko Polak16-16/+16
+ Add testcases and tag expressions for perf occurrences. + Add TNL_1 tag for single tunnel IPsec suites. - Some CRCs were missing before, but form a pair with a listed CRC. - Probably the other half of the pair is not used. - Will be fixed in subsequent Change. Change-Id: I5754210755c13c41226b71045813714a65a1cdf3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-21Add: 2-node ip4 oacl testsJan Gelety78-12/+372
Change-Id: I786fe9f75777e1cd0daee70bc350185dd7fd17e1 Signed-off-by: Jan Gelety <jgelety@cisco.com>