diff options
Diffstat (limited to 'docs/report/introduction/methodology_autogen.rst')
-rw-r--r-- | docs/report/introduction/methodology_autogen.rst | 139 |
1 files changed, 0 insertions, 139 deletions
diff --git a/docs/report/introduction/methodology_autogen.rst b/docs/report/introduction/methodology_autogen.rst deleted file mode 100644 index 5453775b7a..0000000000 --- a/docs/report/introduction/methodology_autogen.rst +++ /dev/null @@ -1,139 +0,0 @@ -.. _autogen_methodology: - -Suite Generation -^^^^^^^^^^^^^^^^ - -CSIT uses robot suite files to define tests. -However, not all suite files available for Jenkins jobs -(or manually started bootstrap scripts) are present in CSIT git repository. -They are generated only when needed. - -Autogen Library -~~~~~~~~~~~~~~~ - -There is a code generation layer implemented as Python library called "autogen", -called by various bash scripts. - -It generates the full extent of CSIT suites, using the ones in git as templates. - -Sources -~~~~~~~ - -The generated suites (and their contents) are affected by multiple information -sources, listed below. - -Git Suites -`````````` - -The suites present in git repository act as templates for generating suites. -One of autogen design principles is that any template suite should also act -as a full suite (no placeholders). - -In practice, autogen always re-creates the template suite with exactly -the same content, it is one of checks that autogen works correctly. - -Regenerate Script -````````````````` - -Not all suites present in CSIT git repository act as template for autogen. -The distinction is on per-directory level. Directories with -regenerate_testcases.py script usually consider all suites as templates -(unless possibly not included by the glob patten in the script). - -The script also specifies minimal frame size, indirectly, by specifying protocol -(protocol "ip4" is the default, leading to 64B frame size). - -Constants -````````` - -Values in Constants.py are taken into consideration when generating suites. -The values are mostly related to different NIC models and NIC drivers. - -Python Code -``````````` - -Python code in resources/libraries/python/autogen contains several other -information sources. - -Testcase Templates -__________________ - -The test case part of template suite is ignored, test case lines -are created according to text templates in Testcase.py file. - -Testcase Argument Lists -_______________________ - -Each testcase template has different number of "arguments", e.g. values -to put into various placeholders. Different test types need different -lists of the argument values, the lists are in regenerate_glob method -in Regenerator.py file. - -Iteration Over Values -_____________________ - -Python code detects the test type (usually by substrings of suite file name), -then iterates over different quantities based on type. -For example, only ndrpdr suite templates generate other types (mrr and soak). - -Hardcoded Exclusions -____________________ - -Some combinations of values are known not to work, so they are excluded. -Examples: Density tests for too much CPUs; IMIX for ASTF. - -Non-Sources -~~~~~~~~~~~ - -Some information sources are available in CSIT repository, -but do not affect the suites generated by autogen. - -Testbeds -```````` - -Overall, no information visible in topology yaml files is taken into account -by autogen. - -Testbed Architecture -____________________ - -Historically, suite files are agnostic to testbed architecture, e.g. ICX or ALT. - -Testbed Size -____________ - -Historically, 2-node and 3-node suites have diferent names, and while -most of the code is common, the differences are not always simple enough. -Autogen treat 2-node and 3-node suites as independent templates. - -TRex suites are intended for a 1-node circuit of otherwise 2-node or 3-node -testbeds, so they support all 3 robot tags. -They are also detected and treated differently by autogen, -mainly because they need different testcase arguments (no CPU count). -Autogen does nothing specifically related to the fact they should run -only in testbeds/NICs with TG-TG line available. - -Other Topology Info -___________________ - -Some bonding tests need two (parallel) links between DUTs. -Autogen does not care, as suites are agnostic. -Robot tag marks the difference, but the link presence is not explicitly checked. - -Job specs -````````` - -Information in job spec files depend on generated suites (not the other way). -Autogen should generate more suites, as job spec is limited by time budget. -More suites should be available for manually triggered verify jobs, -so autogen covers that. - -Bootstrap Scripts -````````````````` - -Historically, bootstrap scripts perform some logic, -perhaps adding exclusion options to Robot invocation -(e.g. skipping testbed+NIC combinations for tests that need parallel links). - -Once again, the logic here relies on what autogen generates, -autogen does not look into bootstrap scripts. |