aboutsummaryrefslogtreecommitdiffstats
path: root/resources
AgeCommit message (Collapse)AuthorFilesLines
2021-12-15UTI: Export resultsVratko Polak22-42/+1374
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-12-15Tox: Add checker for CSIT model versionVratko Polak3-0/+84
+ Implementation stub so checker can check already. + Also add documentation stub for the implemented model. + Checker checks also for bumps in documentation version. - Not comparing implementation and documentation version yet. Change-Id: I4d19c00315a1c171de325c4494c28f5210635f32 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-12-15trending: legend positioningViliam Luc1-2/+2
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: If655d953eb8317f0cba6b3cb35bec232031df7dc
2021-12-14trending: fix broken trendingViliam Luc1-6/+20
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I1ecbfa48f3765884720f0490b9b9da1f69bd2b1a
2021-12-13trending: regression and progression add info to emailViliam Luc2-15/+55
adding trend in Mpps, runs for trend in #, trend change in % + dashboard - removed Short-Term change and added # of runs for trend Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: Ib02d2a2224fc52b79832560241b0530aa2eaaf77
2021-12-13doc: fixing documentation structureViliam Luc2-16/+20
When building documentation using sphinx we see ~1200 similar warnings [0] [0] - https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-verify-tox-master-ubuntu2004-x86_64/3289/doc_verify.log.gz These warning are harmless and can be fixed later Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I1ac1099d38935971d47491dde905715345d3935c
2021-12-13Report: Hide empty latency graph(s) for 3n-icxTibor Frank1-1/+1
Change-Id: I56868d2e62cea8e3bbc6c53c989bb8a8968bc6fa Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-12-10Report: Add 3n-icx - fixesTibor Frank1-1/+1
Change-Id: I130405cd96630e8384d67da02a8bfd1449ff4c85 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-12-09Report: Add 3n-icxTibor Frank2-277/+562
Change-Id: Ic9a90274f5e9791c34560725aaf29266c2a89ce4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-12-06PLRsearch: Update docscringsVratko Polak1-3/+11
Previous code changes updated code comments, but not docstring. Change-Id: I5be3fd07620cc97c9088efceb72c2f68ab103915 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-12-03Trending: Fix 3n-tsh-x520-64b-memif-base-ixgbe graphsTibor Frank1-3/+3
Change-Id: Iec877a74701409c87f941b540494e1f313b32911 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-12-02feat(AWS): Remove obsolete codepmikus3-323/+0
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I97bba8f2092bc0554702a3acac10594426bd79c3
2021-12-01feat(Performance): Add 2M/1G hugepagespmikus8-55/+48
+ Add ability to switch between hugepages. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I84d8eae28ed414a32e5ba82e6c9ed10d7f0ef9cb
2021-11-29Trending: Add ethip4ipsecNspe tests for 2n-tx2Tibor Frank2-0/+15
Change-Id: I59315e7ab10b6773cc17c72bae7e73774755efb5 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-25Pal: Process oper data only from coverage buildsTibor Frank1-6/+16
Signed-off-by: Tibor Frank <tifrank@cisco.com> Change-Id: I9340f4a34d95320e833c209fdec4bff7649522f0 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-25PAPI: Include more details in PAPI fail messageVratko Polak1-3/+4
This helps to distinguish different symptoms for PAL failure alerting. Change-Id: Idd8e8f4b427641fa055516bdf17715f6b967ca72 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-22Report: Add 2n-icx dataTibor Frank3-106/+107
Change-Id: Iaa369a5243b9c77760549f6dae3dd16d8139509a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-22Report: Hide 2n-icx empty graphsTibor Frank3-106/+105
Change-Id: I258891f482d87d930d5df8f2dc93b0507f798b8f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-20Report: Remove 2n-icx build nr 3Tibor Frank1-3/+3
- seems to be corrupted Change-Id: I7f7cb780c0a94601a2c2808d4ef384cb24c3fd99 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-19PAL: Add boxpoints to all box graphsTibor Frank4-121/+113
Change-Id: Ic316149bab15c6fe1efccb53df0afa1cc78b3672 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-18Report: Add dataTibor Frank2-4/+4
Change-Id: I1dcddef704898cfa4ff9a84934fd11b8cf7b139a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-16PAL: Add box points to all box graphsTibor Frank3-17/+28
- enabled only for aws throughput and soak Change-Id: I3c4266e35d98d8319ef1e5c365734338195a40d9 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-15Report: Add boxpoints to aws tput graphsTibor Frank2-13/+24
Change-Id: Idaeba305747863178a5eebc1d42f389a14d6084e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-15Report: Add dataTibor Frank3-4/+6
Change-Id: Ie384ab9db8f3ba909ec0b6eaf39a04c5c7f3157a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-12Report: Show min/avg/max latency for aws - fixesTibor Frank2-13/+11
Change-Id: I119fa64da4b178cb6a8abe63fb16e0bc7f18fd70 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-12Report: Show min/avg/max latency for awsTibor Frank2-10/+19
Change-Id: I4acbe6b20054881f95bf4bab03958ad2dab60629 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-12PAL: Round report comparisons to 1 digit moreVratko Polak1-4/+4
Change-Id: Ifc2fdd04270628053646a6b3d8b15bc730325eb6 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-12Report: Add dataTibor Frank2-18/+22
Change-Id: I2ee6005d499c4d93cca72cffd3373ced16d7c994 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-11Report: Add dataTibor Frank1-1/+4
Change-Id: Ifea5d3b84dbc0df4d8365ed1c1b4dfdd4c49f140 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-11Report: Add DPDK speedup graphsTibor Frank1-0/+120
Change-Id: I0c763ae0371515955037f0aaa867251b5aaec604 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-11Report: Add 4t2c graphs for awsTibor Frank2-10/+10
Change-Id: I4fd625bc5139ebcb8430a4228986eafe39672532 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-11feat(Terraform): Minio S3 gateway proxypmikus1-4/+4
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I9543a856a2e5791b61accd7a183bc5a2bf2e6187
2021-11-10Report: Add stats tables, aws testsTibor Frank2-213/+234
Change-Id: I212b0e6e70008e15ced0167ab34e613cc0a1aea4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-10PAL: Fix heatmap graphTibor Frank1-2/+1
Change-Id: I4992c8f4ae63aa25794cdab98532722c45263e8e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-10Report: Fixes in dynamic contentTibor Frank2-5/+27
Change-Id: If90ba8c66427061341888dcfc946450efe66010b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-10Fix autogen.sh checkerVratko Polak1-2/+2
Previously, the FAIL message was not printed. Change-Id: Id3fc6c126edd42b55cec73294696a0426a73d634 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-10Report: Add dataTibor Frank1-0/+8
Change-Id: Ib00909a62739e488ad809463c08cec14b4b96e91 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-09Report: Add dataTibor Frank2-4/+8
Change-Id: I21eedf3ecabe0672daaaf87819960cd986420fa0 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-09Report: 2n-icx: disable elements without dataTibor Frank1-272/+272
Change-Id: Id908e094129e6becd70a63667ebe61a6433db07e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-09Report: Add 2n-icx, part 2Tibor Frank1-458/+547
- packet throughput - speedup - latency Change-Id: If871dcc1888e7a49ac2fd2ba8e3632a87fa035bc Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-09Report: Add dataTibor Frank1-0/+12
Change-Id: Iec138a93b18f02046d7c13b7243463048ff695ee Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-08Report: Add 2n-icx, part 1Tibor Frank2-278/+278
Change-Id: Idf1e56231afa14648d7c3db661eb9a04ef7a5c85 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-08Report: Remove tests removed by 34207Tibor Frank1-778/+90
Change-Id: Ibe960b8dd4340c2c5b8a01fc490cab906dc330fd Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-08vpp_device: bind to vfio-pci before running testsJuraj Linkeš3-19/+111
In rare cases, binding the whole /dev/vfio folder will result in unusable VFs: notice dpdk EAL: Cannot open /dev/vfio/151: Device or resource busy [0], section 4.3.1. provides some clues as to what's going on and how to avoid the failure. Mounting /dev/vfio reset the file descriptors of all devices under /dev/vfio. Vfio-pci creates a device when an interface is bound to it. The rare failure then occurs when /dev/vfio is mounted while a process is using the file descriptors result in that process using invalid file descriptors (or file descriptors belonging to a different VF). Fix the issue by binding i40e and ice VFs to vfio-pci before containers are created and make sure that the VFs are not unbound later in testing. Only bind DUT VFs since the TG uses the kernel driver. [0]: https://connect.redhat.com/sites/default/files/2021-03/Cloud Native Network Function Requirements.pdf Ticket: CSIT-1794 Change-Id: I83db91b29d16669fb034b141ad247f6f796fdf64 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-11-08Report: Add dataTibor Frank2-13/+51
Change-Id: I76a36cb19d6c121b1d668d1ff8f71085baa2ed2e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-05Report: Add dataTibor Frank1-12/+31
Change-Id: Ia1709ebd3543ac4ee5fa63cefa6b18411c3c58f6 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-04Report: update tablesTibor Frank2-128/+134
- statistics - latency comparison Change-Id: I84069c26e07ced08bad92e19e6cdb0953cd24d3b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-04Report: Add dataTibor Frank2-14/+27
Change-Id: I9fce2c74eb564a80b7b7002538376ec6aefaeac2 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-03Report: Add rls dataTibor Frank2-319/+361
Change-Id: I7868b419fa47c6130826948bbdfbb288abb8d1e4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-11-02fix(autogen): VSAP testcase namingpmikus1-0/+1
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ic6d440ffdf7e27c9e6ce65605178163652758f77