aboutsummaryrefslogtreecommitdiffstats
path: root/resources
AgeCommit message (Collapse)AuthorFilesLines
2019-09-04Clarify why 9000B tests are skipped for IPsecVratko Polak1-1/+2
Ticket: VPP-1207 Change-Id: I506b4a24dfa6b10578407b261926f37215a714f4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-04Hotfix: Bump version and CRCsVratko Polak1-7/+7
Naginator caused a false Verified+1, so a benign API change had been merged prematurely. Change-Id: I8852ca684cba4876d46049ca34204ebbccf96b2e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-04docs: added 3n-tsh testbed tests to report_test_groups.txtMaciek Konstantynowicz1-6/+42
Change-Id: Ib2249ce6c28f4c07878eb5fc2919c56dd8637376 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-09-04Report: ixgbe for tshTibor Frank1-313/+313
Change-Id: I707a654b8853926f4cb1b78f3c65b55925e95ac1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-04Report: Add dataTibor Frank2-162/+50
Change-Id: I6a32e0bdd1a693550214e7bea49b76313449f24d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: Comparison tablesTibor Frank1-10/+14
Change-Id: I108e1292c5de8024060c220b877b668b8843c68f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: Comparison tablesTibor Frank1-6/+6
Change-Id: I90f635f62573154e37a319ca1cbb760401d17c05 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: Comparison tablesTibor Frank2-62/+98
Change-Id: Ib4132c4d25f6a7c936793f403877a5819a6dd55a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: add dataTibor Frank2-45/+208
Change-Id: I51fd0b9e07d09224452f842689925d1166246f19 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03CSIT-1590: Performance comparison analysisTibor Frank1-4/+6
Change-Id: I802b5b9cbeb486587f395aeca76cbe3df68b40c4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03CSIT-1590: Performance comparison analysisTibor Frank2-77/+81
Change-Id: I09d502fcd7b286e41b141e43933bcf09262a8749 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02PAL: list all sel tests in table_performance_comparisonTibor Frank1-3/+17
Change-Id: I99ccbca3e56a58a662c569d195ecdef20d3ea348 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02PAL: list all sel tests in table_performance_comparison_nicTibor Frank2-51/+18
Change-Id: I21bb76aab4a1e627a13057f896ea7bfc34c03463 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02Report: Add dataTibor Frank1-667/+657
Change-Id: If799a7a1d1b8f6ea59ebfba3dd4724b09e7680f2 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02Report: Add 3n-tshTibor Frank2-126/+1512
Change-Id: I6027b635191c7e64397915df8944699bb320daf3 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-30Report: functional tests static contentJan Gelety1-2/+2
Change-Id: I17238ff6d1760f6a956459e072ba0303abef3199 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-30Report: Add dataTibor Frank1-0/+28
Change-Id: Ie9350530879fef7d2ae8c0b58f6fd799fde9fff8 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-29presentation: fixed suite names 10ge>25ge, 10ge>40ge based on nic model.Maciek Konstantynowicz1-229/+229
Change-Id: I90dd569ce42fe6bf29751ca29086fe458d40a095 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-08-28Report: Add dataTibor Frank1-0/+28
Change-Id: I5ae5fa7cb40ddc31e3fe3fd0d6182bf54c84ad00 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: Add dataJan Gelety1-7/+44
Change-Id: I66c91e083138b50a575cbbd32d309d0c198e6a71 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-28NEW: Added trex profiles for 60k ip4 destinations.Ludovit Mikula4-0/+412
Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech> Change-Id: Ib9e42914ad5ab6bf0febea3edbeea8d09c8918b1
2019-08-27Delete unused SR keywordsVratko Polak3-143/+4
Trim both Python (L1) and Robot (L2) keywords, as no test performs SR related deletions. Change-Id: Ic76d870c3f58e927ea222b350e149d0770985072 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-27Report: Add dataTibor Frank1-0/+13
Change-Id: I534cbb4141b60609387f894e51c429e368520a23 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-27- Removed 9000B test cases from ipsec test suites.Ludovit Mikula32-0/+3196
- Added new traffic profiles. Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech> Change-Id: Ie06b7a1cf4d9df403d098b7a495219fd95e5869e
2019-08-27Stop using SSH() in TrafficGenerator.pyVratko Polak2-49/+34
This is a partial fix for: Ticket: CSIT-1434 and should fix one pylint violation. Many files to remove SSH() from remain. + Use -E (together with previous -S) in ssh.py when sudo=True. Needed in some cases, no downsides expected. Change-Id: Ie2ad03b3c656a4d3965a61a1e5435276f99a28c0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-26Report: tablesTibor Frank1-3/+3
Change-Id: Ideec743ae10909d91625ea5cfc7901624459f455 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: NFVTibor Frank1-108/+108
Change-Id: Idf369766cb9acf82475b1d86860d72077b88dc35 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: TablesTibor Frank2-37/+304
Change-Id: I5084163758c5f15f482c6c8e08ee70c4a4f50431 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: Modify comparison tables, add dataTibor Frank1-1159/+1007
Change-Id: I6d5e7cc0618a5b1889ab54c020fd98c661d20989 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: Select tests by name, specificationsTibor Frank2-18533/+3243
Change-Id: Ibcc5e67d49fba9c02b07af462a74d2a062412979 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak8-149/+416
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Improve PLRsearch yet againVratko Polak6-51/+167
Logic improvements for rls1908 and post-pylint fixes: + Reduce search time to 30 minutes. + Use average instead alternating loads. + Rework log/exp avg/stdev from two estimates. + Introduce and use pessimistic variance of dual trackers. + Introduce safe_exp to use when None does not skip code. + Use dot relative imports (instead of disabling pylint). + Complete docstrings for simpler functions. + Append docstrings to named tuples. + Somewhat unify docstrings related to the same arguments. + Slightly improve intentation. + State named tuples as rtype where used. + Add returns and rtype to __repr__ where missing. + Return what docstring say (or update them). + Explicit copying for vector and matrix values. Change-Id: I884c68b4839c5df5e8bef95e463666599603a0ff Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Update CRC list commentsVratko Polak1-15/+39
+ Add testcases and tag expressions for perf occurrences. + Add TNL_1 tag for single tunnel IPsec suites. - Some CRCs were missing before, but form a pair with a listed CRC. - Probably the other half of the pair is not used. - Will be fixed in subsequent Change. Change-Id: I5754210755c13c41226b71045813714a65a1cdf3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Report: Specify groups of tests for graphsTibor Frank1-0/+432
Groups of tests for graphs: - Packet Throughput - Packet Latency - Speedup Multi-Core Change-Id: Ib37c1231abb63c234fb1e98d059bb3338133da0c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-21Framework: simplify CpuUtilsMiroslav Los1-34/+26
Signed-off-by: Miroslav Los <miroslav.los@pantheon.tech> Change-Id: I7682d802f405070e34f72bc4a80cbe6a63b03619
2019-08-21Add: avf-dot1q testcasesPeter Mikus3-4/+126
+ Few fixes in doc + Align testcases Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5575db8255a74fd46f37d5cc2637b9b4448bcd7f
2019-08-20Fix Initialize layer dot1q on nodeVratko Polak1-8/+4
By using Get Variable Value keyword. Change-Id: I3a9650b6f802d60422ffa6b6b00d444ebb050d89 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-20Bump stable vpp version to 20.01-rc0~68Vratko Polak1-0/+1
After API change. CSIT: https://gerrit.fd.io/r/c/csit/+/21245 VPP: https://gerrit.fd.io/r/c/vpp/+/18361 + Add ip_probe_neighbor_reply CRC, not sure how that got missing. Change-Id: I790875dab771981af3b9eceac5425d4d2f374984 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-20Interface API cleanupJan Gelety6-112/+257
CSIT code alignment with API changes in VPP introduced by patch https://gerrit.fd.io/r/c/vpp/+/18361 Change-Id: Ib0357bba79f55d297ef1086fbf3b760caca16cdb Signed-off-by: Jan Gelety <jgelety@cisco.com> Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-20Align NF_denisty testsPeter Mikus1-14/+27
+ Apply layer approach for interfaces Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6c1898b9833f9f42b8912f1b521e30093360d7ea
2019-08-19VPP perf per patch: Reverse the order of testsVratko Polak1-4/+5
If current VPP is broken, we want to fail early. If there is API change, we are more interested in current results, even if parent is likely to fail. Change-Id: I552485bbb9fe0053bae39de7939f94a5971bb63b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-19Autogen: Create NICs for TCP suites.Vratko Polak4-257/+281
+ Put Regenerator internal functions ourside class. + Duplicate into _default_ and _tcp_ functions. + Rework Testcase to be a single class with two factories. + Improve suite and testcase naming. Change-Id: I1f835b576c476c1a40c8c52469b1876695f9ae73 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-19PAL: Select tests by name, functionsTibor Frank3-0/+751
- throughput - latency - speedup Change-Id: Ia8b33815f5bbb7b0fb50b23e7655068928733832 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-19Report: Add dataTibor Frank1-28/+130
- DPDK Change-Id: Ibc45c49e34b2b0abe0f0a9f3304ee478c174c01e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-19PAL: Get DPDK version from output_info.xmlTibor Frank1-2/+2
Change-Id: I922c8143df39025cc0a78047f3cef7b9f5673b4b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-19Framework: MLRSearch refactor of ndrpdrMiroslav Los1-100/+179
Signed-off-by: Miroslav Los <miroslav.los@pantheon.tech> Change-Id: I86bc69f8b2f38b58cb3eb030bec7d0ce5c79d040
2019-08-19Framework: Refactor complex functions in PLRSearchMiroslav Los2-49/+68
Signed-off-by: Miroslav Los <miroslav.los@pantheon.tech> Change-Id: Ie2f19a2e3b37e8d85656ab31ece59b89c76bea25
2019-08-19FIX: Missing ACL libraryPeter Mikus1-0/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I1497a9b031603d09d69a29ed7167686848d4b7c5
2019-08-19Write DPDK version to logTibor Frank1-1/+1
Change-Id: Ia9ec0d420c8db9e71234f74e4013c21d039931b2 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-19FIX: Topology reservationPeter Mikus2-51/+19
Last attempt [0] is doing more harm then good. If testbed is down but recovered quickly enough (reboot, temporary ssh connectivity issue, ...) it never gets back into pool making other testbeds overloaded and queued. This patch is suppose to partially revert previous behavior until proper patch will follow. [0] https://gerrit.fd.io/r/c/csit/+/21148 Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I8203946d10d3e7dd51e97519d679246b5dae59e3