aboutsummaryrefslogtreecommitdiffstats
path: root/docs/job_specs/report_coverage
AgeCommit message (Collapse)AuthorFilesLines
2023-01-19fix(docs): Cleanup job_specspmikus117-4911/+0
Signed-off-by: pmikus <peter.mikus@protonmail.ch> Change-Id: Iccccda3b7909adb9fc330b2d3454a7c429e4cf8f
2022-12-15feat(core): DNV decommissionpmikus18-1083/+0
- Static content will be removed separately Signed-off-by: pmikus <peter.mikus@protonmail.ch> Change-Id: I0992e941e1c24552837eaf7e8d3f6564b3cb21c8
2022-11-07job-spec: Add wireguard test to regressionYulong Pei2-0/+216
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I36f864b9ee36bf2d8107bd92101b639bc48a555e
2022-10-26fix(job_specs): T-Rex jobspmikus2-22/+46
Signed-off-by: pmikus <peter.mikus@protonmail.ch> Change-Id: I1641dcd2bfcacec25af92b9b85c3dad3bdf2f0e0
2022-09-23job-spec: Add trex 1n-awsTibor Frank2-46/+44
- iterative and coverage - mrr and ndrpdr - mlr weekly Signed-off-by: Tibor Frank <tifrank@cisco.com> Change-Id: Icb19efc5c831a0fa2e0cf82065b74c4af5407082
2022-08-22fix(core): 3n-snr codePeter Mikus10-470/+470
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I3bc6b42d2135085ea9dfc5a1b2bb53495094428a
2022-08-08Revert "job-spec: Add VSAP and Hoststack to weekly and coverage"Peter Mikus2-74/+0
This reverts commit dc200e7f96297b3bccecb88b4e951b142617734b. Reason for revert: This is breaking results due to test_type variable set to vsap without supporting model and results itself. It makes the output not usable for trending. Change-Id: I1c4fee87e9bcaed6fc010ebbf570f65208519fad Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
2022-08-02job-spec: Add VSAP and Hoststack to weekly and coverageTibor Frank2-0/+74
- VSAP: run on 2n-clx - Hoststack: run on 3n-icx Change-Id: I9fb54353b0cb61192769eb46e5e8e6849f6f8887 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2022-07-20fix(jobspec): switch tcp tput back to 100bVratko Polak5-86/+86
Change-Id: Id16649044ac0385d55766197fb921c38009c0da8 Ticket: CSIT-1830 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-07-13feat(docs): Update 3n-snr job_specsPeter Mikus27-81/+696
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I1c7ef573038514f1be08ddfc9220df94101c16fb
2022-06-30job_specs: add IPsec perf tests and GTPU perf tests to regressionYulong Pei1-0/+48
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: If70f910003d8208e08d4dc101ac70b33e7e4162b
2022-06-15feat(jobspec): Avoid tests affected by CSIT-1830Vratko Polak5-86/+86
+ Replace TCP TPUT 100b tests on xxv710 NICs with 1518b ones. + Keep UDP TPUT tests at 100b for now. + Add both 100b and 1518b TCP TPUT tests to Denverton. + Its x553 NIC is not affected. + That gives us some comparison on how packet size affects performance. Ticket: CSIT-1830 Change-Id: I5544f7ba84d4813148bcad2f9d64634939492647 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-06-15feat(tests): Remove PPS tests.Vratko Polak3-40/+40
+ Delete nat44 pps suites. + Rename ip4base suites, testcases, suite tags (also tg) to -tput. + Also robot tags like UDP_PPS and TCP_PPS. - Keep traffic types and traffic profile names with pps. + They can be renamed in a different change if needed. + Update job specs. - Do not update methodology yet (avoid conflict with another Change). Ticket: CSIT-1829 Change-Id: Ic0dac441f6e8253edce88e057ab7fad1d3814c24 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-05-30feat(docs): Add e810 for 2n-clxPeter Mikus17-85/+215
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I607ccd5d4b28dca576e40b79285bb2b31a65d76c
2022-05-19fix(jobspec): use 100b for tput and pps testsVratko Polak12-172/+172
For TCP tests, 1518b is closer to what the real traffic would be, but as usual we want smaller sizes to stress VPP before filling line. CPS tests remain at 64b. Fixes: 1daa6fdc0bae284dee1b61f34534e59b60b7526a Change-Id: Id16056b63006bb569a600fa1d9e318fd2853a5c8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-04-04feat(core): update 3n-altJuraj Linkeš18-0/+1029
Update ansible, topologies, bash scripts and job specs. Change-Id: I834b991f281e0b29b4af9dc37fe8028460298d83 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2022-01-24feat(core): Add ICXPeter Mikus53-0/+4070
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ic676d00bd7d3d5d8c244bfd0eb7b10bc74498505
2021-11-19job_specs: update 2n-tx2 daily and reportJuraj Linkeš14-56/+0
Add newly added spd outbound tests with and without flowcache. These new test may result in too many mlr tests, so remove the least needed tests, which are 2m and 1m scale tests. This also unifies which scale tests we run between af_xdp and dpdk tests. Also cleanup all 2n-tx2 job specs. Change-Id: I99bedefcef86f4b921f359f1ebf6c42ebdd8c20f Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-10-27fix(docs): AWS job specspmikus2-5/+20
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I34b1475f0cabe8a6d9f260c77d5a3b10f7ead61c
2021-10-21job-specs: Fix 2n-skx trex coverage job specTibor Frank1-32/+28
Change-Id: I84e998b8dc80b1def40b765325956c6c43e1c6b1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-10-18feat(job-specs): Adjust AWS executionpmikus2-24/+29
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I7cd62b7da0514b23c632a1f319c2ec6fe653bd9a
2021-10-13job-spec: Remove 3n TRex job specsTibor Frank1-50/+0
- These tests do not work and they are not needed. - Merge after 34036: CSIT: Remove 3n TRex jobs | https://gerrit.fd.io/r/c/ci-management/+/34036 Change-Id: Id94059626f17f346c92240e6ab29da12d427545a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-09-27job_specs: trex tests addedViliam Luc2-0/+100
Change-Id: I9d0f226d8de0b4ee4f69a14809c9010dc1fb91bb Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-09-27fix(Job Specs): AWS test case specificationspmikus4-58/+74
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I5c1eea7bb897df957e253c677453ecc2e9c5f14e
2021-07-08job_specs: 3n-tsh reduce number of report coverage testsMaciek Konstantynowicz18-480/+31
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. - -100flows, -100kflows in all acl tests. Change-Id: Ib359c08366768f362c70c184707ab9fde46e1930 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-07-08job_specs: 3n-dnv reduce number of report coverage testsMaciek Konstantynowicz10-59/+9
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. Change-Id: Icd9aa0b0807b16efc26fc94d6633ab61a6c47d9a Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-07-08job_specs: 2n-zn2 reduce number of report coverage testsMaciek Konstantynowicz17-336/+15
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. - nat44 -pps tests, replaced by -tput tests. - nat44 h1-p1-s1 single session tests, unessential combination. - nat44 h4096-p63-s258048 tests, non-essential scale combination. Change-Id: Ia67492b2fd132a64f05a1e8de4cfc136a015fef3 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-07-08job_specs: 3n-skx reduce number of report coverage testsMaciek Konstantynowicz37-1686/+60
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. - -100flows, -100kflows in all acl tests. Change-Id: I3525987726c347be5ffca03dbb82c2b1cf675c2b Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-07-07job_specs: 2n-tx2 reduce number of report coverage testsMaciek Konstantynowicz14-311/+49
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. - nat44 -pps tests, replaced by -tput tests. - nat44 h1-p1-s1 single session tests, unessential combination. - nat44 h4096-p63-s258048 tests, non-essential scale combination. Change-Id: Ia2187fa0ba9c8836840aa2fb7d0c335a3b0d91b1 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-07-07job_specs: 2n-skx reduce number of report coverage testsMaciek Konstantynowicz19-766/+89
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. - nat44 -pps tests, replaced by -tput tests. - nat44 h1-p1-s1 single session tests, unessential combination. - nat44 h4096-p63-s258048 tests, non-essential scale combination. Change-Id: Iae8612bd09980350b0cf5eb90ea4833296c73fc8 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-07-07job_specs: 2n-dnv reduce number of report coverage testsMaciek Konstantynowicz6-39/+6
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. Change-Id: Ifb7d256ea0ff4afe18f80baf1e9058d3775e85d4 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-07-07job_specs: 2n-clx reduce number of report coverage testsMaciek Konstantynowicz19-727/+94
Remove: - All vts tests, obsolete use cases. - dot1q tests apart from dot1q-l2bd, superfluous combinations. - nat44 -pps tests, replaced by -tput tests. - nat44 h1-p1-s1 single session tests, unessential combination. - nat44 h4096-p63-s258048 tests, non-essential scale combination. Change-Id: I2e2ff96dd727977f55e38f7a388f1afe68096aa9 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-06-16job_specs: reduce number of report coverage tests for ipsecMaciek Konstantynowicz8-597/+23
Remove: - ipsec tests - ethip4ipsectptlispgpe. - policy-aes128gcm. - policy-aes128cbc-hmac256sha. - policy-aes128cbc-hmac512sha. - int-aes128cbc-hmac256sha. - scale of - 400tnlsw. - 5000tnlsw. - 20000tnlsw. - 60000tnlsw. Change-Id: I1443f35b7bf852fd665e8856d05bf04b9827913f Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-06-02Hsw deprecation: Delete 3n-hsw job specsVratko Polak37-3013/+0
+ Verify jobs remain available for 3n-hsw. - PAL edits are independent, will be done elsewhere. - Autogen edits to come later. Change-Id: I2d0d1b15cbe49627fa9c5188c70f94156b438b77 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-04-15AWS: Add jobspecsTomas Alexy2-0/+58
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech> Change-Id: I93c3d2a25bddc6623c131b5cd1742bb55b1ee7c3
2021-04-09job-spec: Fix job-specsTibor Frank1-4/+4
- remove redundant empty lines - fix the number of tests in job-spec and its duration Change-Id: I50fc0f151dedf7d2e9858ecc3959ab42a243bb1d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-03-03job-specs: Update durations of jobsTibor Frank152-612/+612
- coverage - 2n-clx - 2n-skx - 2n-zn2 - 2n-tx2 - 2n-dnv - 3n-skx - 3n-hsw - 3n-tsh - 3n-dnv - iterative - 2n-clx - 2n-skx - 2n-zn2 - 2n-tx2 - 2n-dnv - 3n-skx - 3n-hsw - 3n-tsh - 3n-dnv Change-Id: Ica79cb68a83bd7cd1f74044c0c464b345778476b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-02-24job-spec: Add test suites for crypto sw scheduler engineTibor Frank3-0/+652
Change-Id: I68af8c56f3bd33d0f122f56f1ae60993beb67969 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-02-19FIX: Use x710 for gso testspmikus3-39/+39
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I13892dda60a6e130ebe7c074ac0a3b429e08ef32
2021-02-19FIX: Naming convention compliancepmikus3-36/+36
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Idcee623abbc5568fc6ca49113bc2ff8add179046
2021-02-19Performance: Add GSO tests to report executionpmikus3-0/+135
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I35b2f786105103f19907d74c4f6cc77d7cd3a5bb
2021-02-09Report: Hide nat44 tput testsTibor Frank1-62/+62
Change-Id: Ia01a4a7b0b70547655ab9c6e463040bc97f1727f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-02-08job-spec: fix report_coverage/2n-tx2/l2-04.mdTibor Frank1-13/+13
- replace eth-l2bdbasemaclrn-macip-iacl1sl-10kflows by eth-l2bdbasemaclrn-macip-iacl50sl-10kflows Change-Id: Ieaaea1e59533059ceaffe1e3c767d51b4ab7c8d0 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-01-27job-specs: Add rnd testsTibor Frank8-29/+487
Change-Id: I877f46608853401c5226845737ace691e3f393b1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-01-20job_specs: add geneve testsJan Gelety3-0/+255
Geneve tests added to report_iterative and report_coverage job specifications. - tests with 1024 geneve tunnels added only to report_coverage jobs until fast PAPI is finished Change-Id: I11a12d3c626dd2daf6287440131795daa7f75e48 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2021-01-13job_specs: update 2n-tx2 coverage specsJuraj Linkeš14-342/+507
Start out with l2, l3, classifier and acl tests. Change-Id: I0cd781f1858d5461b793f782f8bc6fcde5263b69 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-01-04API: deprecated COP APIsTomas Alexy18-268/+268
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech> Change-Id: Ifb97088a26f3c2ab2ecee28ed43e6e957a844c64
2020-12-23job_specs: add 2x-tx2 job specsJuraj Linkeš13-1/+927
Add job specs for the new 2n-tx2 performance testbed. Change-Id: Ife1db4caba3ad17286727fed143e0c0556c4a2a4 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-11-02job-specs: Update the durationsTibor Frank7-28/+28
Change-Id: Ibed3ad934bc4fd03013404e18b5a0d6917499bf8 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-29job-spec: Add all NAT44 testsTibor Frank6-305/+1095
- the specs: - docs/job_specs/report_iterative/2n-clx/vpp-mlr-00.md - docs/job_specs/report_iterative/2n-clx/vpp-mrr-00.md - docs/job_specs/report_iterative/2n-skx/vpp-mlr-00.md - docs/job_specs/report_iterative/2n-skx/vpp-mrr-00.md take too long. We should consider to remove some tests. - the coverage specs were split to three parts Change-Id: I39e2032c76266e159a781d1225cff1a5119d4c85 Signed-off-by: Tibor Frank <tifrank@cisco.com>