aboutsummaryrefslogtreecommitdiffstats
path: root/docs
AgeCommit message (Collapse)AuthorFilesLines
2020-12-23job_specs: add 2x-tx2 job specsJuraj Linkeš20-1/+1342
Add job specs for the new 2n-tx2 performance testbed. Change-Id: Ife1db4caba3ad17286727fed143e0c0556c4a2a4 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-12-16report: added report static content for new testbed type AMD EPYC Zen2 7532Maciek Konstantynowicz4-3/+1142
Change-Id: I9099cce7c4da15a1b3a9aafd37825b284c5ed7fe Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2020-12-14Report: Add dataTibor Frank1-0/+3
Change-Id: I298c3bdec3f9254a2644401a9ffb7ff09c4ecbd2 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-12-14Improve release notesVratko Polak2-3/+10
+ Add a link to NAT44 methodology. + Mention TCP tests are not using 64B packets. Change-Id: Ief6334e84abdf4d099dfbc455cfc7c9b2179ccfe Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-12-13Ansible: Add 10.32.8.17 into nomad poolpmikus1-16/+16
+ Extra storage capacity Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ifbdc26581776e9885b0eb9929d8ac1f731282426
2020-12-03vpp-device: GENEVE tunnel test, l3 modeJan Gelety1-0/+12
Jira: CSIT-1769 Change-Id: I98510cd8e627d7347f77d0032b7bac28f2e36c61 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-12-09Report: Add dataTibor Frank3-6/+4
Change-Id: Ic7d559d30c2c7d314ba029dfe7c1db99e25e20f4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-12-08Revert "vpp-device: GENEVE tunnel test, l3 mode"Peter Mikus1-12/+0
This reverts commit a9f54ca5080aeef17686f300a6807bf9b46b7c90. Reason for revert: DO NOT MERGE BROKEN TESTS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-device-master-ubuntu1804-1n-skx/9918/archives/log.html.gz Change-Id: I2ce2970a43e5dd7487eeb54d1ccbb149e06cf8fa Signed-off-by: pmikus <pmikus@cisco.com>
2020-12-03vpp-device: GENEVE tunnel test, l3 modeJan Gelety1-0/+12
Jira: CSIT-1769 Change-Id: I0ab906966f760b809b8431530c010a8f0835d62c Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-12-07Report: Add 2n-zn2 turbo boost on/off dataTibor Frank57-6/+5563
Change-Id: I1eb0475d30048d6b42458d1fb696d7fac93ac948 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-12-04Report: Configure 2009.50Tibor Frank1-1/+4
Change-Id: Ia8e71df86d561c1d9bc388fa6b70ef850c94e66e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-12-01Intel E810 hardware onboardingpmikus1-14/+35
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ia055011626212472f65d18a5c1a10440973ecd48
2020-11-30Trending: Fix links to NAT44 filesTibor Frank12-24/+24
Change-Id: Ia19cfa32430c7f12e0093d0b540d23876ca9da8a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-30Trending: Fix filenamesTibor Frank23-0/+0
Change-Id: Idf754d624443ceef1db5ea5e1d4ab2a9f1c5649b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-30Trending: Reorganize the NAT44 chaptersTibor Frank41-922/+1050
Change-Id: Ifec550ae69c6454d0adbc08d3e49435c3cca380f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-26Report: Reorganize the NAT44 chaptersTibor Frank43-1151/+1864
- packet throughput - speed-up Change-Id: I5d49ae2da54170e48c51db47e035d2121471be0d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-25Report: Update release notes.Vratko Polak1-3/+7
+ Expand details on NAT stateless tests added. + Clarify situation with jumbo ipsec. Change-Id: Id6fb7fa48b5fadca7798ec1da7e1baf44d0827aa Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-24ansible: update 3n-tsh hugepagesJuraj Linkeš1-8/+12
Fix "Not enough availablehuge pages: 1483!". Also update 3n-tsh docs. Change-Id: I1d37a66af1e2363f77fdbd87d238e8ff5535b011 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-11-24lab: ThunderX2 updatesJuraj Linkeš1-27/+100
Update after switching one 1n-tx2 and the idle ThunderX2 servers between racks. Update the idle ThunderX2 specs to a new perf testbed, 2n-tx2. Add Server-Type-B12 which is a modified Server-Type-B2 with one extra NIC (needed for 2n-tx2). Change-Id: I51af358f1feb476652eddfe82b5af1d0d70ac259 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-11-23Report: Configure rls2009.48Tibor Frank1-0/+3
Change-Id: Ib7c4bc51bea43dafa2bd788c709c800caeb75988 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-23Report: Add 2n-zn2Tibor Frank74-80/+5642
Change-Id: Ifaba0d689a279ae650ae30531b21533cc6927262 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-20Trending: Add 2n-zn2Tibor Frank26-0/+1539
Change-Id: I49e2c50829af8f538155b6262ec2872a68560858 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-20Add and reorder methodology sectionsVratko Polak1-3/+4
+ DUT state chapter was not reachable. + Small reorder to put edits together and sooner. Change-Id: I2735c522e92e277298e299176b4cdc9be4044c26 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-19Remove csit-nat44ed-cps-spec.mdVratko Polak1-499/+0
If some details are still needed, re-add them to docs/report/introduction/methodology_nat44.rst Change-Id: I8796dcafff83407eda28719148b9527d2963e6f8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-19Methodology: Add profile details to nat44 docVratko Polak1-8/+230
Change-Id: I5dc427693f78e4c712ab5cae587d7849dc2fec16 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-19Methodology: Edit nat44 test spec docVratko Polak1-33/+97
- Missing profile specifics and transaction counters. Change-Id: I6f7378e5fe9d639599c38545b0503354a8a65198 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-19Methodology: DUT state considerationsVratko Polak1-0/+150
Change-Id: I4bdb61cf665579c6b49d79c6fb281b4c0fb5ec89 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-19Methodology: Trex modes and transactionsVratko Polak3-49/+199
Change-Id: I43423dea499bce3a298dbbba752c2aee2a322836 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-19Trending: Add NAT44 testsTibor Frank6-314/+774
Change-Id: I2601a786f58b1d89cc9a90cb16ff20e6331abbfe Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-12Report: Add NAT44 dataTibor Frank1-3/+6
- reorder NAT44 graphs Change-Id: Ied762a85fb1ee69c7d12971e3e15822571a18451 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-11Report: Add comparison soak vs ndrTibor Frank1-2/+0
Change-Id: I19308ed936246be5b4e61c6e219591d16095cb56 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-11job-spec: update the runs durationsTibor Frank2-8/+8
Change-Id: I6de49b81bd4b75d40c792b044dc46024279d6651 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-09job-specs: Align mrr-daily with report iter for 2n-clx and 2n-skxTibor Frank2-1288/+2
Change-Id: I783f0f055462daa1e8047c6905a17495b8473135 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-06Report: Add NAT44 LatencyTibor Frank2-18/+97
Change-Id: I2c247c5362396bcb4c8b42b1b46a2601091ca968 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-05Report: Configure rls2009.46Tibor Frank1-0/+17
Change-Id: I7886285de8edbf66e03e1b22d85c2e7a4c4d7224 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-04docs: nat44ed test spec for udp and tcp cps tests using trex astfMaciek Konstantynowicz1-0/+499
Change-Id: I277f462521273947d374e79a687e7f616ad1f13b Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2020-11-04job-spec: Split mrr daily from report iterativeTibor Frank6-240/+1526
- enable all NAT44 tests for report iterative Change-Id: Ifc820c3d0b657637bb4569f59ddaecd9aefb8d1d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-03Report: Fix bugs in NAT44 graphsTibor Frank4-824/+126
Change-Id: I6de094fc0cb2acdfb6eeb22e6d85a65d2e9f1478 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-02job-specs: Update the durationsTibor Frank16-64/+64
Change-Id: Ibed3ad934bc4fd03013404e18b5a0d6917499bf8 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-02Report: Add dataTibor Frank1-0/+5
Change-Id: I800273d04c3f956e58be727cbf5befa61377fa2b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-11-01Report: NAT44 LatencyTibor Frank2-106/+61
Change-Id: Ifca9e4f5783eea37df30ddfedc80c988f2163ad8 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-30Report: Add NAT44 dataTibor Frank1-1/+1
Change-Id: I463bf61d41484738ba5c614b473bcb5457e6abe1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-30Report: Add NATT44 testsTibor Frank5-162/+863
Change-Id: I3920f881286565029f35bcfa700f36c6701db457 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-29Support existing test types with ASTFVratko Polak1-27/+74
+ Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites. + Update existing cps traffic profiles. + Add missing traffic profiles. + UDP: + Single burst of 32 packets was confirmed as safe enough for TRex. + Maybe 64 could work, but not enough testing for that. + Multiple bursts have lead to reduced TRex performance, as overlaping bursts (from different client instances) tend to fill up the buffers. + TCP: + Data size set to 11111 bytes, completely arbitrarily. + Results look reasonable, so I have kept that. - MSS not set at all - No tested support for frame size other than 64B. - Frame size does not even factor into TCP profiles. + So other frame sizes are skipped in autogen. + Update tags in related suites. - HOSTS_{n} and SRC_USER_{n} should be unified. - Questionable clarification on difference between IP4BASE and SCALE. + Add NAT state resetters to tests that need them. + Resetter is called (if set) before each measurement. + If ramp-up is detected, resetter is not set. + Rename "mult" argument to "multiplier". + Abstracted from packets to transactions. + Transaction corresponds to profile. + TRex multiplier argument sets target rate in transactions per second. + The familiar STL traffic: + Bidirectional is considered to be 2 packets per transaction. + Unidirectional is considered to be 1 packet per transaction. + The newer ASTF traffic: + 4 subtypes, each has different number of packets per transaction. + For max rate computation: + Packets in the more numerous direction are considered. + Rely on TRex reported traffic duration for ASTF: + Use the server side value. - Client side value is higher by an overhead. - TRex is not sending traffic during that time. + Remove delays from traffic profiles. - Those delays would increase the reprted traffic time. + Support for scale lmited trials. + Only for ASTF profiles, each ASTF profile has limited scale. + Scale defined in suite variables. + For TRex to send all transactions provided duration value is ignored. + The appropriate value is computed in TrafficGenerator. + An ad-hoc time constant is added to match the TRex client side time overhead. + The profile driver receives the computed duration. + Measurement for PLRsearch add a sleep if the computed duration is smaller. + Alternative argument for search algos if scale is limited. + Both need higher timeout to accomodate big scales. + MLRsearch can afford fewer phases. + Added a parameter to optionally shorten the duration. + Use short duration for runtime stats trial and failure stats trial. + Use very large keepalive values in udp profiles to avoid ka packets. + No polling in ASTF profile driver. - Polling could eliminate the time overhead value. + But polling proved to introduce some loss, affecting the results. + Handle duration stretching in ASTF by stopping traffic. + The stop has several steps so that: + The traffic is really stopped entirely. + Late packets do not count (maybe as errors). + Stats are preserved to read for results (and cleared afterwards). + Several quantities added to ReceiveRateMeasurement: + Original target duration is preserved (algos need that). + Input estimate (tps) for early search iterations. + Output estimate (maybe pps) for MRR output. + Strict result (unsent counts as loss) for NDR. + Use L2 counters (opackets, ipackets) where possible. - TRex has trouble processing packets for the L7 ones at high loads. + Remove warmup from profile drivers and keywords. + Suites should call "Send ramp-up traffic" explicitly if needed. + Added parsing for few more counters. + Both to use in formulas or just for debug purposes. - Only 64B cases in autogen, framesize support to be added later. + Latency streams during search can be enabled via PERF_USE_LATENCY env var. + MLRsearch improvments: + Rename argument names to min_rate and max_rate. + Use relative receive rate in initial phase. + PLRsearch improvements: + Careful computation when output (pps) does not match input (tps). + Use geometric distribution (instead of Poisson). + Helps agains math errors. + This should improve estimate stability. - But in practice big losses still lead to significant jumps. + Traffic generator improvements: + send_traffic_on_tg now calls the full set_rate_provider_defaults. + _send_traffic_on_tg_internal for the logic without provider defaults. + As the internal function is re-used by measure() without affecting defaults. + Move _parse_traffic_results just before get_measurement_result. + As the latter uses fields set bu the former, it is now easier to read. + Multiple sources for approximate duration. + Tried from more precise to more available. + Includes logic for _pps tests (added in later change). + Move explicit type conversions to earlier occurences. + Profile driver output field uses semicolons to simplify parsing. + Performance Robot lib file split to several smaller ones. + performance_actions.robot: + Hosts Additional Statistics Action For * keywords. + performance_display.robot: + Hosts keyword for displaying and verifying results. + Change test message to use the correct unit (pps or cps). + performance_limits.robot renamed to performance_vars.robot + Added many keywords, mostly for accessing test variables. + Moved variables for Policer into a new keyword there. + Some keywords need sophisticated logic. - Other are basically Get Variable Value. + But in future more logic can be added, without editing callers. + Documentation for the new keywords acts as a documentation for test variables. + performance_utils.robot has the rest. + Eliminated arguments if the value is in test variable. + Small improvements to documentation. - Still not enough cleanup with respect to arguments and test variables. + Keywords are sorted alphabetically now in each one. + Suites: + Unified variables table: + No colons in comments. + ${n_hosts}, ${n_ports} and use them instead hardcoded numbers. + Add -cps to existing cps suite names. + Remove "trial data overwrite". + Compute max rate as in STL suites. + Each NAT suite has ip4base suite to compare results to. - Those act as indirect TRex calibration. - VPP does not lose packets in those. + Latency in ASTF suites is disabled hard. - As we do not support latency in ASTF profiles yet. + Unidirectional tests governed by suite variable, not an argument. + Write long argument lists vertically. + Prefer to use argument names. + In Python, also the last argument is followed by comma. + It makes renaming and reordering easier. + Similarly applies to prints with long lists of values. + A TODO to update api crc file comments. Change-Id: I84729355edbec051298a9de1162107f88ff5737d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-10-29job-spec: Add all NAT44 testsTibor Frank10-458/+1791
- the specs: - docs/job_specs/report_iterative/2n-clx/vpp-mlr-00.md - docs/job_specs/report_iterative/2n-clx/vpp-mrr-00.md - docs/job_specs/report_iterative/2n-skx/vpp-mlr-00.md - docs/job_specs/report_iterative/2n-skx/vpp-mrr-00.md take too long. We should consider to remove some tests. - the coverage specs were split to three parts Change-Id: I39e2032c76266e159a781d1225cff1a5119d4c85 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-29tests: Add GSO enabled testsMartin Balaz1-0/+4
Change-Id: I409b060f64ae7c6787448ae519fd76d8384e9ffb Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
2020-10-28Report: Configure rls2009.45Tibor Frank1-0/+8
Change-Id: I8918f09411254b64abb65a165a0e284f735b306e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-28Report: Add dataTibor Frank1-5/+0
Change-Id: I2394b42e177758a8f3ce4aeeb912b264cf3b457c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-21Report: Configure 2009.44Tibor Frank1-0/+23
Change-Id: I7f7e50610e71af7f2b0dc1eed267a421ffd8aef0 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-10-19Tests: Remove GBPpmikus10-119/+8
- Specs - Tests - Library - TODO: PAL Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I8890940d56c7a9924d4a5f9365b6bb881610db14