aboutsummaryrefslogtreecommitdiffstats
path: root/docs/content/methodology/suite_generation.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/content/methodology/suite_generation.md')
-rw-r--r--docs/content/methodology/suite_generation.md124
1 files changed, 0 insertions, 124 deletions
diff --git a/docs/content/methodology/suite_generation.md b/docs/content/methodology/suite_generation.md
deleted file mode 100644
index 4fa9dee0ce..0000000000
--- a/docs/content/methodology/suite_generation.md
+++ /dev/null
@@ -1,124 +0,0 @@
----
-title: "Suite Generation"
-weight: 19
----
-
-# Suite Generation
-
-CSIT uses robot suite files to define tests.
-However, not all suite files available for Jenkins jobs
-(or manually started bootstrap scripts) are present in CSIT git repository.
-They are generated only when needed.
-
-## Autogen Library
-
-There is a code generation layer implemented as Python library called "autogen",
-called by various bash scripts.
-
-It generates the full extent of CSIT suites, using the ones in git as templates.
-
-## Sources
-
-The generated suites (and their contents) are affected by multiple information
-sources, listed below.
-
-### Git Suites
-
-The suites present in git repository act as templates for generating suites.
-One of autogen design principles is that any template suite should also act
-as a full suite (no placeholders).
-
-In practice, autogen always re-creates the template suite with exactly
-the same content, it is one of checks that autogen works correctly.
-
-### Regenerate Script
-
-Not all suites present in CSIT git repository act as template for autogen.
-The distinction is on per-directory level. Directories with
-regenerate_testcases.py script usually consider all suites as templates
-(unless possibly not included by the glob patten in the script).
-
-The script also specifies minimal frame size, indirectly, by specifying protocol
-(protocol "ip4" is the default, leading to 64B frame size).
-
-### Constants
-
-Values in Constants.py are taken into consideration when generating suites.
-The values are mostly related to different NIC models and NIC drivers.
-
-### Python Code
-
-Python code in resources/libraries/python/autogen contains several other
-information sources.
-
-#### Testcase Templates
-
-The test case part of template suite is ignored, test case lines
-are created according to text templates in Testcase.py file.
-
-#### Testcase Argument Lists
-
-Each testcase template has different number of "arguments", e.g. values
-to put into various placeholders. Different test types need different
-lists of the argument values, the lists are in regenerate_glob method
-in Regenerator.py file.
-
-#### Iteration Over Values
-
-Python code detects the test type (usually by substrings of suite file name),
-then iterates over different quantities based on type.
-For example, only ndrpdr suite templates generate other types (mrr and soak).
-
-#### Hardcoded Exclusions
-
-Some combinations of values are known not to work, so they are excluded.
-Examples: Density tests for too much CPUs; IMIX for ASTF.
-
-## Non-Sources
-
-Some information sources are available in CSIT repository,
-but do not affect the suites generated by autogen.
-
-### Testbeds
-
-Overall, no information visible in topology yaml files is taken into account
-by autogen.
-
-#### Testbed Architecture
-
-Historically, suite files are agnostic to testbed architecture, e.g. ICX or ALT.
-
-#### Testbed Size
-
-Historically, 2-node and 3-node suites have diferent names, and while
-most of the code is common, the differences are not always simple enough.
-Autogen treat 2-node and 3-node suites as independent templates.
-
-TRex suites are intended for a 1-node circuit of otherwise 2-node or 3-node
-testbeds, so they support all 3 robot tags.
-They are also detected and treated differently by autogen,
-mainly because they need different testcase arguments (no CPU count).
-Autogen does nothing specifically related to the fact they should run
-only in testbeds/NICs with TG-TG line available.
-
-#### Other Topology Info
-
-Some bonding tests need two (parallel) links between DUTs.
-Autogen does not care, as suites are agnostic.
-Robot tag marks the difference, but the link presence is not explicitly checked.
-
-### Job specs
-
-Information in job spec files depend on generated suites (not the other way).
-Autogen should generate more suites, as job spec is limited by time budget.
-More suites should be available for manually triggered verify jobs,
-so autogen covers that.
-
-### Bootstrap Scripts
-
-Historically, bootstrap scripts perform some logic,
-perhaps adding exclusion options to Robot invocation
-(e.g. skipping testbed+NIC combinations for tests that need parallel links).
-
-Once again, the logic here relies on what autogen generates,
-autogen does not look into bootstrap scripts.