aboutsummaryrefslogtreecommitdiffstats
path: root/resources
AgeCommit message (Collapse)AuthorFilesLines
2020-01-30perf: Add hoststack NSIM+LDPRELOAD+IPERF3 test suiteDave Wallace5-15/+29
Change-Id: Ia7a876b1aa240676e1f2d23618c1d4e09ead14f0 Signed-off-by: Dave Wallace <dwallacelf@gmail.com> (cherry picked from commit d79b753c1fde3cd831f60ac5cc0034f613c44b43)
2020-01-20Use vpp 20.01-release pacakgesJan Gelety1-5/+8
Signed-off-by: Jan Gelety <jgelety@cisco.com> Change-Id: Ibd1b78fd5bb9424564f27497a19176527c604be4
2020-01-30PAL: Latency processingTibor Frank1-16/+44
Very ugly solution for now. It will be replaced by using JSON structure for test results soon. Change-Id: If5b63de94bf02e7fc2a518c1f6a94f42dcec9e07 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 20e972ce413e727b0a3d89d04460534cb454f107)
2020-01-29Display target_tr for ndr/pdr resultsVratko Polak1-5/+5
Small differences in Tx packet counts can cause the previous way (which used transmit_rate, computed from packets sent and nominal duration) to report lower bounds as higher than upper bounds. Also, reconf tests need the target (not actual) rate. Change-Id: If5222dabd9762b8d9c2e22d875c12ad355bc6838 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 401c24914d84058d06ef9b7a4f893662dfeb0dcb)
2020-01-29Ndrpdr: Remove unused latency measurementsVratko Polak1-23/+14
+ Improve keyword Documentation. + Reorder measurements, higher loads first. Change-Id: I1aeab1c8ada268422e9f421e075aff2ac9eec23f Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 703d013c83c9a4ce2283f1a0c16fe1511f1144d3)
2020-01-29Restore PDR stat run for NDRPDRVratko Polak1-2/+6
Before [0], the NDRPDR test ended with two measurements that perform runtime stats collection, each 2 seconds long. [0] then decreased that duration to one second, and removed the PDR measurement. This change re-adds the PDR measurement, while keeping the duration at 1 second. [0] https://gerrit.fd.io/r/c/csit/+/24427 Change-Id: I76828f11863a9e278cdd0cf60f2792d5b614ce43 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit f7699715f37ee6846bceb1fa7baf51366a683737)
2020-01-29perf: QUIC transport hoststack test suiteDave Wallace3-14/+43
Change-Id: I73f4be7ea315c7a5dcce46e1bd3034bcb0a97ee2 Signed-off-by: Dave Wallace <dwallacelf@gmail.com> (cherry picked from commit 074ae9732d06f962a65d91147b55abf86b42b6d9)
2020-01-29perf: hoststack iperf3 test tuningDave Wallace11-62/+135
Change-Id: I53425f57fe9ecef9cff2c94642cc7cb24537a961 Signed-off-by: Dave Wallace <dwallacelf@gmail.com> (cherry picked from commit b340b5c9925b43869b91197850504760757a0bfe)
2020-01-29Report: Fix: Add new NFV dataTibor Frank1-1/+1
Change-Id: Iae3557db30c5a95cb26d2e9b5acf24df2f64e429 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 3bc72c6e268474a6bed725e5371d66f0e369ab16)
2020-01-29Report: Add new NFV dataTibor Frank1-106/+1542
Change-Id: Ic5a20940466545611150cfceaed58cee702636c4 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 8f6b774bfb9d5728ce1b1dafc11ff1d00a05d90c)
2020-01-29Report: Add NFV data for existing elementsTibor Frank1-81/+57
Change-Id: Ib76dbf36f6eb0234f6947fc43bd6c31e12a1d8e9 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 33741e9e4d961c7ca805ef635982cd210a44a3b4)
2020-01-28Check interval validity in reconf testsVratko Polak1-0/+2
Without this, if the search part returns min_rate, then reconf part reports few seconds with a small loss count. But min_rate means there is a bug or misconfiguration, so the tests should fail, instead of returning misleading results. Change-Id: I566608b40ae11d6aaa3fe022a32da97901e1999c Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 6ac704e7388409c6aa719e50f83d0e08ebd671a0)
2020-01-28Perform some cleanup discovered by GPL workVratko Polak90-35/+23
+ Remove one __init__.robot file with no function. + Remove executable flags from traffic profiles. + Improve indentation in some traffic profiles. Change-Id: I099e63069003fba788976a9470a909ccff51a88c Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit ceaa2959217752716496b6257595d12cbcb0c657)
2020-01-28FIX: dot1qip4vxlan testsPeter Mikus7-28/+56
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ieb6e6010108987f99f55730149e3e4b7f1a7fc21 (cherry picked from commit 97dbe2feecc6d592db7427c4f1a72c1ac31990d1)
2020-01-28Report: Add RC2 dataTibor Frank1-74/+59
Change-Id: I29dc3e34293d8bcc940e2adc3034e32b3806c71c Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit a4882cb1547e2abf424bb53f424439989fc3b6e7)
2020-01-28PAL: Add latency by percentile graphTibor Frank3-4/+243
Change-Id: I38c03e089fd709afc848ecec55474d5c6925a58e Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 6716e51e3a27cfd2850a399c9ccc2818c2038eaf)
2020-01-27Increase min_rate for NDRPDR testsVratko Polak1-2/+4
Latency measurements break for loads less than 9 kpps per direction. Setting min_rate to 90 kpps prevents 10% latency trial from breaking. 90 kpps should be enough for any type of test. When some test finds a lower value, this patch will make the fail with a message saying the loss rate at minimal load is too high (instead of a cryptic error from 10% measurement without this patch). Change-Id: I97b8428aa5044da23343b86447857b3a101f6baa Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit f7452e826cd83fc5ded28c1268f6f4aab252ca2d)
2020-01-27Report: Add RC1 dataTibor Frank1-0/+21
Change-Id: I3d6d9c56456a12435096f7c61f744519279a850b Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit cdb023e3eb911ad3367fccec8ba1a2935a18c7fd)
2020-01-27Report: Add RC1 dataTibor Frank1-7/+0
Change-Id: I7ebbabd1f56b892b6f88a6a183632cb11fc2bc3c Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 9afeffb373c05a5853cbfc5a61456c69af423d9e)
2020-01-27Report: Add RC1 dataTibor Frank1-500/+399
Change-Id: I7ab49f6a910b9273b78e89096e06761ff7537ec9 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 2221396d9ebacf7a5333d63ed329a38f9507838b)
2020-01-24Attempt to fix journalctlPeter Mikus1-3/+3
- AWK - ECHO - pipe + Much simpler Change-Id: I4876f4eae68c96571dc38fd21e53ba269f5f888e Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-01-23FIX: nfv_densityJan Gelety2-36/+38
- use correct osi_layer=L2 (so L2 spoofing check is switched off in case of avf driver) - add pci address information to eth interface in topology file - nfv_density chain_ipsec tests work only with DPDK in current implementation Change-Id: I233c6e5634a14581960c7459b87f11fcee8365bd Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit f31b70ee422ed0dc4e5d9b60061a1c2cfc904684)
2020-01-23FIX: Various issuesPeter Mikus5-81/+15
- TrafficProfile - IPsec jumbo - TDLK leftover - Trex scapy server Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0601ea75ed6e7318c46124fb02e9c61066089791 (cherry picked from commit 6ccc6d6811a173b8cc3816c51a2dbef8934b3ffd)
2020-01-23Report: Add diagrams for testbedsTibor Frank1-1/+1
- Diagrams for testbeds: - 3n-dnv - 3n-tsh - 2n-clx - Replace s/Cascadelake/Cascade Lake Change-Id: I77e7659a0aba4766a28577f940b7e44e60cbd82d Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 25a8fe0aa594ca010b7f1aad449a2af2b6625bd7)
2020-01-22PAL: Process HDRH LatencyTibor Frank1-16/+76
Change-Id: Ifc48869ccb6c05f4b5af6cd97e05f009c7632dc7 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 654bfa4c33dca6d6eeba6636199de0ff80f75652)
2020-01-21Ndrpdr: Add latency trials for smaller loadsVratko Polak2-11/+82
Change-Id: I46bd4b58bdd02a3deed38c158214e1e9977dde05 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 8840972e7bbc45735f6e82e41ad369728c1b370e)
2020-01-20reduce test cases for 2n-dnv and 3n-dnv mrr daily testYulong Pei2-202/+2
since dnv platform is to use by low end market product, no need care more about too much flows performance test. Change-Id: Id7204b0353cbf968e641f14ed980f01af75971b2 Signed-off-by: Yulong Pei <yulong.pei@intel.com> (cherry picked from commit b693dedec9077ddb08f50ebefc9d84addcdc047a)
2020-01-20revert "Report: Remove 'avf' from TC name in plots"Tibor Frank1-9/+6
Change-Id: I9b10b8f2f84daad543ac9b84671753785dfb23ad Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 87f81f123c054908df6770f11a947b9fe6ab0e93)
2020-01-17perf: add TCP Iperf3+LDPRELOAD test suiteDave Wallace9-55/+284
Change-Id: Icff49fb31cce342a2a4ae799e844ec91f9e5e366 Signed-off-by: Dave Wallace <dwallacelf@gmail.com> (cherry picked from commit 6518c73a0e5cdaf12ae01d247a65aec287d01625)
2020-01-17Fix unreservationVratko Polak1-2/+3
It got broken by the previous reservation fix. Change-Id: I5d9091b7fa4fcc37148c0223fe8eb83f7e5b6770 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit ac33ed374cb82c9258e4cf57e150f77750362bbc)
2020-01-17Report: Remove 'avf' from TC name in plotsTibor Frank1-5/+7
Change-Id: Ie1d04347de123032a5d424747d216f90022bc813 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit c26ab619b300e6bf38c447c0600a5739ca46e27b) (cherry picked from commit 21591b3ec26ce457f8374acb970b0504cd373b55)
2020-01-17FIX: remove 4drc nfv_density chain ipsec tests from 3n-hsw daily jobJan Gelety1-12/+0
- not enough cpu cores reported Change-Id: If3ddbbf4f5af741f328c45ac6b8c874e61d348ab Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit 8b64d3f942a28a01df1daac3ccee42828021a410)
2020-01-17FIX: missing write of mac_dst for direction 0 --> 1 in cnf trex profilesJan Gelety34-1/+137
Change-Id: I3acf5220c24f77bc4bfc451919bf9b5345a988ef Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit 9159c81f65c71cf77e9f1d59241e9ebcbbe9abc4)
2020-01-17Fix reservation if connect failsVratko Polak1-4/+8
Change-Id: I136cc51cbd5a01451e04a39c5d8e5554d6d39305 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 1b9d4210b9b466fedab3a5891ec16a34e67a78ee)
2020-01-17Report: Prepare data setsTibor Frank2-639/+517
Change-Id: Ice90ffe8563d6283050f2b229fc5e50ab64ac253 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit e864945ddfd65b99b3311fddc9260dba3a02b1bc)
2020-01-17FIX: L2XC with vhost and VLANJan Gelety1-0/+3
Change-Id: Ia87abadba431099eeacd8b50ce5d116597a1f393 Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit 633e1f40e5adcb38458fbd6d1209afc1240244a5)
2020-01-16Python3: Migration of files under traffic-profiles/trexBalaji Venkatraman130-4745/+10715
Change-Id: Iae16225d3d3daa4e1e9b177228fb89efe4aa5405 Signed-off-by: Balaji Venkatraman <balajiv@cisco.com>
2020-01-16Report: Latency by Percentile DistributionTibor Frank1-2/+7
- legend Change-Id: I35477ce5c2cabfdcef68424854a8574bd3655e55 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-16Report: Latency by Percentile DistributionTibor Frank2-5/+5
- Set X-axis Change-Id: Ica300c7b1352c824bf51ac26e7485c9295ac60dc Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Report: Latency by Percentile DistributionTibor Frank1-1/+7
- Set the position of the Lagend Change-Id: I1bcea6002bd8beeba3300c9976d1603a234c3c14 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Report: Add Latency by Percentile DistributionTibor Frank2-4/+3299
Change-Id: I5291db8117da611497e910aa7cd2fbb2276bfe65 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Ansible: Increase TG hugepagesPeter Mikus20-57/+17
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ifa87e372ed45f6dfc883723acbb325e34fb814a8
2020-01-15Remove everything related to TLDKVratko Polak7-527/+0
- Leftovers from kubernetes found, but not removed here. Change-Id: If8cb9269d0f3e69f642d7fe02c59122e17925a4d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Trending: new daily setJan Gelety15-266/+3158
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Support suite tags in autogenVratko Polak3-22/+175
+ Include a script to add suite tags to many suites at once. + Add suite tags also to device tests (not covered by autogen). Change-Id: I514ee6178e22999b43460028fe2696738b012f04 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak24-179/+278
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-15Report: Current vs. Previous Release - NFV TestsTibor Frank1-0/+172
Change-Id: I00cbde38b8f900efe3e3123a942de2288381ea3b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Hoststack perf infrastructure refactoringDave Wallace16-74/+1057
- DUT only topology (hoststack test apps are co-located with vpp) - Make vpp app specific keywords generic where applicable - Add IP4 Prefix to topology file - Support running wrk in linux namespace - Refactor namespace cleanup - Remove redundant namespace creation code - Refactor test/keyword dirs: tcp -> hoststack - Add hoststack utility keywords - Refactor wrk suite setup/teardown - Update tests with recent perf infra changes Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-14FIX: VXLAN scalePeter Mikus1-3/+5
+ Another stupid arp -> neighbor rename Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia873d6c2a485659033f71cdf546782db74787d13
2020-01-14FIX: NF_density testsPeter Mikus1-2/+2
+ ARP -> Neighbor Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I74044bd7db6cb4b1bd3648a424e9c8f1473be8d1