aboutsummaryrefslogtreecommitdiffstats
path: root/resources
AgeCommit message (Collapse)AuthorFilesLines
2020-01-30PAL: Latency processingTibor Frank1-16/+44
Very ugly solution for now. It will be replaced by using JSON structure for test results soon. Change-Id: If5b63de94bf02e7fc2a518c1f6a94f42dcec9e07 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-29Display target_tr for ndr/pdr resultsVratko Polak1-5/+5
Small differences in Tx packet counts can cause the previous way (which used transmit_rate, computed from packets sent and nominal duration) to report lower bounds as higher than upper bounds. Also, reconf tests need the target (not actual) rate. Change-Id: If5222dabd9762b8d9c2e22d875c12ad355bc6838 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-29Ndrpdr: Remove unused latency measurementsVratko Polak1-23/+14
+ Improve keyword Documentation. + Reorder measurements, higher loads first. Change-Id: I1aeab1c8ada268422e9f421e075aff2ac9eec23f Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-29perf: QUIC transport hoststack test suiteDave Wallace3-14/+43
Change-Id: I73f4be7ea315c7a5dcce46e1bd3034bcb0a97ee2 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-29Report: Fix: Add new NFV dataTibor Frank1-1/+1
Change-Id: Iae3557db30c5a95cb26d2e9b5acf24df2f64e429 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-29Report: Add new NFV dataTibor Frank1-106/+1542
Change-Id: Ic5a20940466545611150cfceaed58cee702636c4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-29IPSEC: Change plugin namingPeter Mikus2-2/+2
https://gerrit.fd.io/r/c/vpp/+/24574 Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ic2b7c925ceba1e16de87c64003ecceeea69c681c
2020-01-29Report: Add NFV data for existing elementsTibor Frank1-81/+57
Change-Id: Ib76dbf36f6eb0234f6947fc43bd6c31e12a1d8e9 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-28perf: hoststack iperf3 test tuningDave Wallace11-62/+135
Change-Id: I53425f57fe9ecef9cff2c94642cc7cb24537a961 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-28Check interval validity in reconf testsVratko Polak1-0/+2
Without this, if the search part returns min_rate, then reconf part reports few seconds with a small loss count. But min_rate means there is a bug or misconfiguration, so the tests should fail, instead of returning misleading results. Change-Id: I566608b40ae11d6aaa3fe022a32da97901e1999c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28Perform some cleanup discovered by GPL workVratko Polak90-35/+23
+ Remove one __init__.robot file with no function. + Remove executable flags from traffic profiles. + Improve indentation in some traffic profiles. Change-Id: I099e63069003fba788976a9470a909ccff51a88c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28FIX: dot1qip4vxlan testsPeter Mikus7-28/+56
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ieb6e6010108987f99f55730149e3e4b7f1a7fc21
2020-01-28Report: Add RC2 dataTibor Frank1-74/+59
Change-Id: I29dc3e34293d8bcc940e2adc3034e32b3806c71c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-28PAL: Add latency by percentile graphTibor Frank3-4/+243
Change-Id: I38c03e089fd709afc848ecec55474d5c6925a58e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-27Increase min_rate for NDRPDR testsVratko Polak1-2/+4
Latency measurements break for loads less than 9 kpps per direction. Setting min_rate to 90 kpps prevents 10% latency trial from breaking. 90 kpps should be enough for any type of test. When some test finds a lower value, this patch will make the fail with a message saying the loss rate at minimal load is too high (instead of a cryptic error from 10% measurement without this patch). Change-Id: I97b8428aa5044da23343b86447857b3a101f6baa Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-27Report: Add RC1 dataTibor Frank1-0/+21
Change-Id: I3d6d9c56456a12435096f7c61f744519279a850b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-27Report: Add RC1 dataTibor Frank1-7/+0
Change-Id: I7ebbabd1f56b892b6f88a6a183632cb11fc2bc3c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-27Restore PDR stat run for NDRPDRVratko Polak1-2/+6
Before [0], the NDRPDR test ended with two measurements that perform runtime stats collection, each 2 seconds long. [0] then decreased that duration to one second, and removed the PDR measurement. This change re-adds the PDR measurement, while keeping the duration at 1 second. [0] https://gerrit.fd.io/r/c/csit/+/24427 Change-Id: I76828f11863a9e278cdd0cf60f2792d5b614ce43 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-27Report: Add RC1 dataTibor Frank1-500/+399
Change-Id: I7ab49f6a910b9273b78e89096e06761ff7537ec9 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-24Attempt to fix journalctlPeter Mikus1-3/+3
- AWK - ECHO - pipe + Much simpler Change-Id: I4876f4eae68c96571dc38fd21e53ba269f5f888e Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-01-23FIX: nfv_densityJan Gelety2-36/+38
- use correct osi_layer=L2 (so L2 spoofing check is switched off in case of avf driver) - add pci address information to eth interface in topology file - nfv_density chain_ipsec tests work only with DPDK in current implementation Change-Id: I233c6e5634a14581960c7459b87f11fcee8365bd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-23FIX: Various issuesPeter Mikus5-81/+15
- TrafficProfile - IPsec jumbo - TDLK leftover - Trex scapy server Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0601ea75ed6e7318c46124fb02e9c61066089791
2020-01-23Report: Add diagrams for testbedsTibor Frank1-1/+1
- Diagrams for testbeds: - 3n-dnv - 3n-tsh - 2n-clx - Replace s/Cascadelake/Cascade Lake Change-Id: I77e7659a0aba4766a28577f940b7e44e60cbd82d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-22Generate tests in deterministic orderVratko Polak1-1/+2
Change-Id: I47e45f395976fd65a8da0e93bc33588cc213c79e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-22fix loadbalancer running error(Caused by code upgrades)haiyanx1.zhang1-3/+3
Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com> Change-Id: I39bd1d9ad59971bb8cba6e664ba5509abba6a527
2020-01-22PAL: Process HDRH LatencyTibor Frank1-16/+76
Change-Id: Ifc48869ccb6c05f4b5af6cd97e05f009c7632dc7 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-21Ndrpdr: Add latency trials for smaller loadsVratko Polak2-11/+82
Change-Id: I46bd4b58bdd02a3deed38c158214e1e9977dde05 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-20reduce test cases for 2n-dnv and 3n-dnv mrr daily testYulong Pei2-202/+2
since dnv platform is to use by low end market product, no need care more about too much flows performance test. Change-Id: Id7204b0353cbf968e641f14ed980f01af75971b2 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2020-01-20FIX: Hugepages should be detected also from overcommitPeter Mikus2-148/+70
Change-Id: I36ec9d61f775791ad0632a579c4921889d1f5acf Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-01-20revert "Report: Remove 'avf' from TC name in plots"Tibor Frank1-9/+6
Change-Id: I9b10b8f2f84daad543ac9b84671753785dfb23ad Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-17perf: add TCP Iperf3+LDPRELOAD test suiteDave Wallace9-55/+284
Change-Id: Icff49fb31cce342a2a4ae799e844ec91f9e5e366 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-17Fix unreservationVratko Polak1-2/+3
It got broken by the previous reservation fix. Change-Id: I5d9091b7fa4fcc37148c0223fe8eb83f7e5b6770 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-17Report: Remove 'avf' from TC name in plotsTibor Frank1-5/+7
Change-Id: Ie1d04347de123032a5d424747d216f90022bc813 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-16FIX: remove 4drc nfv_density chain ipsec tests from 3n-hsw daily jobJan Gelety1-12/+0
- not enough cpu cores reported Change-Id: If3ddbbf4f5af741f328c45ac6b8c874e61d348ab Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-17FIX: missing write of mac_dst for direction 0 --> 1 in cnf trex profilesJan Gelety34-1/+137
Change-Id: I3acf5220c24f77bc4bfc451919bf9b5345a988ef Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-17Fix reservation if connect failsVratko Polak1-4/+8
Change-Id: I136cc51cbd5a01451e04a39c5d8e5554d6d39305 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-17Report: Prepare data setsTibor Frank2-639/+517
Change-Id: Ice90ffe8563d6283050f2b229fc5e50ab64ac253 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-16FIX: L2XC with vhost and VLANJan Gelety1-0/+3
Change-Id: Ia87abadba431099eeacd8b50ce5d116597a1f393 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-16Python3: Migration of files under traffic-profiles/trexBalaji Venkatraman130-4745/+10715
Change-Id: Iae16225d3d3daa4e1e9b177228fb89efe4aa5405 Signed-off-by: Balaji Venkatraman <balajiv@cisco.com>
2020-01-16Report: Latency by Percentile DistributionTibor Frank1-2/+7
- legend Change-Id: I35477ce5c2cabfdcef68424854a8574bd3655e55 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-16Report: Latency by Percentile DistributionTibor Frank2-5/+5
- Set X-axis Change-Id: Ica300c7b1352c824bf51ac26e7485c9295ac60dc Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Report: Latency by Percentile DistributionTibor Frank1-1/+7
- Set the position of the Lagend Change-Id: I1bcea6002bd8beeba3300c9976d1603a234c3c14 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Report: Add Latency by Percentile DistributionTibor Frank2-4/+3299
Change-Id: I5291db8117da611497e910aa7cd2fbb2276bfe65 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Ansible: Increase TG hugepagesPeter Mikus20-57/+17
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ifa87e372ed45f6dfc883723acbb325e34fb814a8
2020-01-15Remove everything related to TLDKVratko Polak7-527/+0
- Leftovers from kubernetes found, but not removed here. Change-Id: If8cb9269d0f3e69f642d7fe02c59122e17925a4d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Trending: new daily setJan Gelety15-266/+3158
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Support suite tags in autogenVratko Polak3-22/+175
+ Include a script to add suite tags to many suites at once. + Add suite tags also to device tests (not covered by autogen). Change-Id: I514ee6178e22999b43460028fe2696738b012f04 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak24-179/+278
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-15Report: Current vs. Previous Release - NFV TestsTibor Frank1-0/+172
Change-Id: I00cbde38b8f900efe3e3123a942de2288381ea3b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Hoststack perf infrastructure refactoringDave Wallace16-74/+1057
- DUT only topology (hoststack test apps are co-located with vpp) - Make vpp app specific keywords generic where applicable - Add IP4 Prefix to topology file - Support running wrk in linux namespace - Refactor namespace cleanup - Remove redundant namespace creation code - Refactor test/keyword dirs: tcp -> hoststack - Add hoststack utility keywords - Refactor wrk suite setup/teardown - Update tests with recent perf infra changes Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>