feat(docs): Add Methodology

Signed-off-by: pmikus <peter.mikus@protonmail.ch>
Change-Id: I5b2e4c14cc258d821b630d2e54b23a8468820764

1 files changed, 125 insertions, 0 deletions

diff --git a/docs/content/methodology/suite_generation.md b/docs/content/methodology/suite_generation.md
new file mode 100644
index 0000000000..351e70a7d1
--- /dev/null
+++ b/docs/content/methodology/suite_generation.md
@@ -0,0 +1,125 @@
+---
+bookToc: false
+title: "Suite Generation"
+weight: 19
+---
+
+# Suite Generation
+
+CSIT uses robot suite files to define tests.
+However, not all suite files available for Jenkins jobs
+(or manually started bootstrap scripts) are present in CSIT git repository.
+They are generated only when needed.
+
+## Autogen Library
+
+There is a code generation layer implemented as Python library called "autogen",
+called by various bash scripts.
+
+It generates the full extent of CSIT suites, using the ones in git as templates.
+
+## Sources
+
+The generated suites (and their contents) are affected by multiple information
+sources, listed below.
+
+### Git Suites
+
+The suites present in git repository act as templates for generating suites.
+One of autogen design principles is that any template suite should also act
+as a full suite (no placeholders).
+
+In practice, autogen always re-creates the template suite with exactly
+the same content, it is one of checks that autogen works correctly.
+
+### Regenerate Script
+
+Not all suites present in CSIT git repository act as template for autogen.
+The distinction is on per-directory level. Directories with
+regenerate_testcases.py script usually consider all suites as templates
+(unless possibly not included by the glob patten in the script).
+
+The script also specifies minimal frame size, indirectly, by specifying protocol
+(protocol "ip4" is the default, leading to 64B frame size).
+
+### Constants
+
+Values in Constants.py are taken into consideration when generating suites.
+The values are mostly related to different NIC models and NIC drivers.
+
+### Python Code
+
+Python code in resources/libraries/python/autogen contains several other
+information sources.
+
+#### Testcase Templates
+
+The test case part of template suite is ignored, test case lines
+are created according to text templates in Testcase.py file.
+
+#### Testcase Argument Lists
+
+Each testcase template has different number of "arguments", e.g. values
+to put into various placeholders. Different test types need different
+lists of the argument values, the lists are in regenerate_glob method
+in Regenerator.py file.
+
+#### Iteration Over Values
+
+Python code detects the test type (usually by substrings of suite file name),
+then iterates over different quantities based on type.
+For example, only ndrpdr suite templates generate other types (mrr and soak).
+
+#### Hardcoded Exclusions
+
+Some combinations of values are known not to work, so they are excluded.
+Examples: Density tests for too much CPUs; IMIX for ASTF.
+
+## Non-Sources
+
+Some information sources are available in CSIT repository,
+but do not affect the suites generated by autogen.
+
+### Testbeds
+
+Overall, no information visible in topology yaml files is taken into account
+by autogen.
+
+#### Testbed Architecture
+
+Historically, suite files are agnostic to testbed architecture, e.g. ICX or ALT.
+
+#### Testbed Size
+
+Historically, 2-node and 3-node suites have diferent names, and while
+most of the code is common, the differences are not always simple enough.
+Autogen treat 2-node and 3-node suites as independent templates.
+
+TRex suites are intended for a 1-node circuit of otherwise 2-node or 3-node
+testbeds, so they support all 3 robot tags.
+They are also detected and treated differently by autogen,
+mainly because they need different testcase arguments (no CPU count).
+Autogen does nothing specifically related to the fact they should run
+only in testbeds/NICs with TG-TG line available.
+
+#### Other Topology Info
+
+Some bonding tests need two (parallel) links between DUTs.
+Autogen does not care, as suites are agnostic.
+Robot tag marks the difference, but the link presence is not explicitly checked.
+
+### Job specs
+
+Information in job spec files depend on generated suites (not the other way).
+Autogen should generate more suites, as job spec is limited by time budget.
+More suites should be available for manually triggered verify jobs,
+so autogen covers that.
+
+### Bootstrap Scripts
+
+Historically, bootstrap scripts perform some logic,
+perhaps adding exclusion options to Robot invocation
+(e.g. skipping testbed+NIC combinations for tests that need parallel links).
+
+Once again, the logic here relies on what autogen generates,
+autogen does not look into bootstrap scripts.