aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2019-09-04Doc: Report release notes (perf)Peter Mikus1-0/+5
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I8c1667efec67febd375818bc1dfe1d9bebc70708
2019-09-04docs: added methodology terminology section to report.Maciek Konstantynowicz2-0/+79
Change-Id: Id787d12b577165a1081dba84b610c32df57a7c44 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-09-04docs: report updates in physical_testbeds section.Maciek Konstantynowicz1-86/+163
Change-Id: Id31e6ed294dc80619a108296be580925509bb7d7 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-09-04docs: updated report intro in packet_throughput_graphsMaciek Konstantynowicz1-5/+68
Change-Id: I3bf70cc7e48ef2134841bc4bf9300cda6a118cb3 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-09-04docs: added 3n-tsh testbed tests to report_test_groups.txtMaciek Konstantynowicz1-6/+42
Change-Id: Ib2249ce6c28f4c07878eb5fc2919c56dd8637376 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-09-04Report: ixgbe for tshTibor Frank35-806/+834
Change-Id: I707a654b8853926f4cb1b78f3c65b55925e95ac1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-04Report: Add dataTibor Frank2-162/+50
Change-Id: I6a32e0bdd1a693550214e7bea49b76313449f24d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: Comparison tablesTibor Frank1-10/+14
Change-Id: I108e1292c5de8024060c220b877b668b8843c68f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: Comparison tablesTibor Frank1-6/+6
Change-Id: I90f635f62573154e37a319ca1cbb760401d17c05 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: Comparison tablesTibor Frank2-62/+98
Change-Id: Ib4132c4d25f6a7c936793f403877a5819a6dd55a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03Report: add dataTibor Frank2-45/+208
Change-Id: I51fd0b9e07d09224452f842689925d1166246f19 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03CSIT-1590: Performance comparison analysisTibor Frank1-4/+6
Change-Id: I802b5b9cbeb486587f395aeca76cbe3df68b40c4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-03CSIT-1590: Performance comparison analysisTibor Frank2-77/+81
Change-Id: I09d502fcd7b286e41b141e43933bcf09262a8749 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02PAL: list all sel tests in table_performance_comparisonTibor Frank1-3/+17
Change-Id: I99ccbca3e56a58a662c569d195ecdef20d3ea348 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02PAL: list all sel tests in table_performance_comparison_nicTibor Frank2-51/+18
Change-Id: I21bb76aab4a1e627a13057f896ea7bfc34c03463 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02Report: Add dataTibor Frank4-946/+933
Change-Id: If799a7a1d1b8f6ea59ebfba3dd4724b09e7680f2 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-09-02Update of VPP_STABLE_VER filesJan Gelety3-3/+3
- use new vpp ref build - ubuntu 16.04: 20.01-rc0~129_g5c89fbf~b7728 - use new vpp ref build - ubuntu 18.04: 20.01-rc0~129-g5c89fbf28 - use new vpp ref build - centos7: 20.01-rc0~129_g5c89fbf~b7728 Change-Id: Ie11a1fddd8f2e1c27e45a4b231a09fbdeced9c24 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-09-02Add: Testpmd VM testPeter Mikus28-36/+2033
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Id39667515e304f4d92f230f82d3f1dbc1e22ac32
2019-09-02Report: Add 3n-tshTibor Frank69-188/+4912
Change-Id: I6027b635191c7e64397915df8944699bb320daf3 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-30Report: functional tests static contentJan Gelety3-99/+72
Change-Id: I17238ff6d1760f6a956459e072ba0303abef3199 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-30Report: Add dataTibor Frank1-0/+28
Change-Id: Ie9350530879fef7d2ae8c0b58f6fd799fde9fff8 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-29presentation: fixed suite names 10ge>25ge, 10ge>40ge based on nic model.Maciek Konstantynowicz1-229/+229
Change-Id: I90dd569ce42fe6bf29751ca29086fe458d40a095 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-08-28Report: Add dataTibor Frank1-0/+28
Change-Id: I5ae5fa7cb40ddc31e3fe3fd0d6182bf54c84ad00 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: Add dataJan Gelety1-7/+44
Change-Id: I66c91e083138b50a575cbbd32d309d0c198e6a71 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-28docs: created full list of tests for report detailed results.Maciek Konstantynowicz3-0/+673
Change-Id: I82d298399b5899736c8e5ebb7204dbbddace79f4 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-08-28docs: created select list of tests for report and trending.Maciek Konstantynowicz3-0/+335
Change-Id: I2ad255c35490e52d9a55e24e64539549e6bdd36b Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-08-28NEW: Added trex profiles for 60k ip4 destinations.Ludovit Mikula4-0/+412
Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech> Change-Id: Ib9e42914ad5ab6bf0febea3edbeea8d09c8918b1
2019-08-27DOC: 1908rls release notes for VPP Hoststack TCP performance tests.Dave Wallace1-0/+4
Change-Id: I56d0519b5cbdfb8a60aab85a9a4d721fdfe022ce Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-08-27Bump DPDK versionVratko Polak1-1/+1
VPP has bumped it in https://gerrit.fd.io/r/c/vpp/+/21406 Change-Id: I32a950134f5e93b3c68e3748ab9aaaec2dde2d5a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-27docs: added csit tests job specs.Maciek Konstantynowicz1-0/+141
Change-Id: I271b1c700f72204dcf03bee6f2cd30b25ca5663e Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2019-08-27Delete unused SR keywordsVratko Polak3-143/+4
Trim both Python (L1) and Robot (L2) keywords, as no test performs SR related deletions. Change-Id: Ic76d870c3f58e927ea222b350e149d0770985072 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-27Report: Add dataTibor Frank1-0/+13
Change-Id: I534cbb4141b60609387f894e51c429e368520a23 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-27- Removed 9000B test cases from ipsec test suites.Ludovit Mikula97-780/+3196
- Added new traffic profiles. Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech> Change-Id: Ie06b7a1cf4d9df403d098b7a495219fd95e5869e
2019-08-27Stop using SSH() in TrafficGenerator.pyVratko Polak2-49/+34
This is a partial fix for: Ticket: CSIT-1434 and should fix one pylint violation. Many files to remove SSH() from remain. + Use -E (together with previous -S) in ssh.py when sudo=True. Needed in some cases, no downsides expected. Change-Id: Ie2ad03b3c656a4d3965a61a1e5435276f99a28c0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-26Report: Comp TablesTibor Frank1-30/+30
Change-Id: I2cedef4dc274dc45ebc82911d53f3d5d14ffa381 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: tablesTibor Frank1-3/+3
Change-Id: Ideec743ae10909d91625ea5cfc7901624459f455 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: NFVTibor Frank7-297/+293
Change-Id: Idf369766cb9acf82475b1d86860d72077b88dc35 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: TablesTibor Frank2-37/+304
Change-Id: I5084163758c5f15f482c6c8e08ee70c4a4f50431 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-26Report: Modify comparison tables, add dataTibor Frank9-1481/+1298
Change-Id: I6d5e7cc0618a5b1889ab54c020fd98c661d20989 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-25FIX: Copy-paste typo in ↵Jan Gelety1-1/+1
2n-10ge2p1x710-eth-l2bd-6ch-48vh-24vm1t-vppip4-ndrpdr.robot Change-Id: Ibe231547e26d9b379554eec8ca6a9b4d7bc1224b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-26Report: Select tests by name, specificationsTibor Frank184-32279/+8271
Change-Id: Ibcc5e67d49fba9c02b07af462a74d2a062412979 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak33-187/+2392
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Improve PLRsearch yet againVratko Polak6-51/+167
Logic improvements for rls1908 and post-pylint fixes: + Reduce search time to 30 minutes. + Use average instead alternating loads. + Rework log/exp avg/stdev from two estimates. + Introduce and use pessimistic variance of dual trackers. + Introduce safe_exp to use when None does not skip code. + Use dot relative imports (instead of disabling pylint). + Complete docstrings for simpler functions. + Append docstrings to named tuples. + Somewhat unify docstrings related to the same arguments. + Slightly improve intentation. + State named tuples as rtype where used. + Add returns and rtype to __repr__ where missing. + Return what docstring say (or update them). + Explicit copying for vector and matrix values. Change-Id: I884c68b4839c5df5e8bef95e463666599603a0ff Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Update CRC list commentsVratko Polak17-31/+55
+ Add testcases and tag expressions for perf occurrences. + Add TNL_1 tag for single tunnel IPsec suites. - Some CRCs were missing before, but form a pair with a listed CRC. - Probably the other half of the pair is not used. - Will be fixed in subsequent Change. Change-Id: I5754210755c13c41226b71045813714a65a1cdf3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Report: Specify groups of tests for graphsTibor Frank1-0/+432
Groups of tests for graphs: - Packet Throughput - Packet Latency - Speedup Multi-Core Change-Id: Ib37c1231abb63c234fb1e98d059bb3338133da0c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-08-21Add: 2-node ip4 oacl testsJan Gelety78-12/+372
Change-Id: I786fe9f75777e1cd0daee70bc350185dd7fd17e1 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-21Framework: simplify CpuUtilsMiroslav Los1-34/+26
Signed-off-by: Miroslav Los <miroslav.los@pantheon.tech> Change-Id: I7682d802f405070e34f72bc4a80cbe6a63b03619
2019-08-21Add: avf-dot1q testcasesPeter Mikus13-71/+595
+ Few fixes in doc + Align testcases Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5575db8255a74fd46f37d5cc2637b9b4448bcd7f
2019-08-20Fix Initialize layer dot1q on nodeVratko Polak1-8/+4
By using Get Variable Value keyword. Change-Id: I3a9650b6f802d60422ffa6b6b00d444ebb050d89 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-20Bump stable vpp version to 20.01-rc0~68Vratko Polak4-3/+4
After API change. CSIT: https://gerrit.fd.io/r/c/csit/+/21245 VPP: https://gerrit.fd.io/r/c/vpp/+/18361 + Add ip_probe_neighbor_reply CRC, not sure how that got missing. Change-Id: I790875dab771981af3b9eceac5425d4d2f374984 Signed-off-by: Vratko Polak <vrpolak@cisco.com>