aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report/introduction/csit_design.rst
diff options
context:
space:
mode:
authorTibor Frank <tifrank@cisco.com>2017-10-20 13:35:53 +0200
committerTibor Frank <tifrank@cisco.com>2017-10-23 13:00:36 +0000
commit5d31c77092fd784e21cd50d604be30924ec9a7c5 (patch)
tree77300cb891555c8fc14dbb69d641c013e39a45f6 /docs/report/introduction/csit_design.rst
parent890c00d6b809d0d908903fbf1a0ec5ceca8b54ad (diff)
Report: CSIT Framework Documentation
Change-Id: I2eba1183a5acdd1fb34efab16fc5c89e38dedb14 Signed-off-by: Tibor Frank <tifrank@cisco.com>
Diffstat (limited to 'docs/report/introduction/csit_design.rst')
-rw-r--r--docs/report/introduction/csit_design.rst192
1 files changed, 0 insertions, 192 deletions
diff --git a/docs/report/introduction/csit_design.rst b/docs/report/introduction/csit_design.rst
deleted file mode 100644
index 7798081a59..0000000000
--- a/docs/report/introduction/csit_design.rst
+++ /dev/null
@@ -1,192 +0,0 @@
-.. _csit-design:
-
-CSIT Design
-===========
-
-FD.io CSIT system design needs to meet continuously expanding
-requirements of FD.io projects including VPP, related sub-systems (e.g.
-plugin applications, DPDK drivers) and FD.io applications (e.g. DPDK
-applications), as well as growing number of compute platforms running
-those applications. With CSIT project scope and charter including both
-FD.io continuous testing AND performance trending/comparisons, those
-evolving requirements further amplify the need for CSIT framework
-modularity, flexibility and usability.
-
-Design Hierarchy
-----------------
-
-CSIT follows a hierarchical system design with SUTs and DUTs at the
-bottom level of the hierarchy, presentation level at the top level and a
-number of functional layers in-between. The current CSIT system design
-including CSIT framework is depicted in the figure below.
-
-.. raw:: latex
-
- \begin{figure}[H]
- \centering
- \includesvg[width=0.90\textwidth]{csit_design_picture}
- \label{fig:csit_design_picture}
- \end{figure}
-
-.. figure:: csit_design_picture.svg
- :alt: FD.io CSIT system design
- :align: center
-
- *FD.io CSIT system design*
-
-A brief bottom-up description is provided here:
-
-#. SUTs, DUTs, TGs
-
- - SUTs - Systems Under Test;
- - DUTs - Devices Under Test;
- - TGs - Traffic Generators;
-
-#. Level-1 libraries - Robot and Python
-
- - Lowest level CSIT libraries abstracting underlying test environment, SUT,
- DUT and TG specifics;
- - Used commonly across multiple L2 KWs;
- - Performance and functional tests:
-
- - L1 KWs (KeyWords) are implemented as RF libraries and Python
- libraries;
-
- - Performance TG L1 KWs:
-
- - All L1 KWs are implemented as Python libraries:
-
- - Support for TRex only today;
- - CSIT IXIA drivers in progress;
-
- - Performance data plane traffic profiles:
-
- - TG-specific stream profiles provide full control of:
-
- - Packet definition – layers, MACs, IPs, ports, combinations thereof
- e.g. IPs and UDP ports;
- - Stream definitions - different streams can run together, delayed,
- one after each other;
- - Stream profiles are independent of CSIT framework and can be used
- in any T-rex setup, can be sent anywhere to repeat tests with
- exactly the same setup;
- - Easily extensible – one can create a new stream profile that meets
- tests requirements;
- - Same stream profile can be used for different tests with the same
- traffic needs;
-
- - Functional data plane traffic scripts:
-
- - Scapy specific traffic scripts;
-
-#. Level-2 libraries - Robot resource files:
-
- - Higher level CSIT libraries abstracting required functions for executing
- tests;
- - L2 KWs are classified into the following functional categories:
-
- - Configuration, test, verification, state report;
- - Suite setup, suite teardown;
- - Test setup, test teardown;
-
-#. Tests - Robot:
-
- - Test suites with test cases;
- - Functional tests using VIRL environment:
-
- - VPP;
- - Honeycomb;
- - NSH_SFC;
-
- - Performance tests using physical testbed environment:
-
- - VPP;
- - DPDK-Testpmd;
- - DPDK-L3Fwd;
-
- - Tools:
-
- - Documentation generator;
- - Report generator;
- - Testbed environment setup ansible playbooks;
- - Operational debugging scripts;
-
-Test Lifecycle Abstraction
---------------------------
-
-A well coded test must follow a disciplined abstraction of the test
-lifecycles that includes setup, configuration, test and verification. In
-addition to improve test execution efficiency, the commmon aspects of
-test setup and configuration shared across multiple test cases should be
-done only once. Translating these high-level guidelines into the Robot
-Framework one arrives to definition of a well coded RF tests for FD.io
-CSIT. Anatomy of Good Tests for CSIT:
-
-#. Suite Setup - Suite startup Configuration common to all Test Cases in suite:
- uses Configuration KWs, Verification KWs, StateReport KWs;
-#. Test Setup - Test startup Configuration common to multiple Test Cases: uses
- Configuration KWs, StateReport KWs;
-#. Test Case - uses L2 KWs with RF Gherkin style:
-
- - prefixed with {Given} - Verification of Test setup, reading state: uses
- Configuration KWs, Verification KWs, StateReport KWs;
- - prefixed with {When} - Test execution: Configuration KWs, Test KWs;
- - prefixed with {Then} - Verification of Test execution, reading state: uses
- Verification KWs, StateReport KWs;
-
-#. Test Teardown - post Test teardown with Configuration cleanup and
- Verification common to multiple Test Cases - uses: Configuration KWs,
- Verification KWs, StateReport KWs;
-#. Suite Teardown - Suite post-test Configuration cleanup: uses Configuration
- KWs, Verification KWs, StateReport KWs;
-
-RF Keywords Functional Classification
--------------------------------------
-
-CSIT RF KWs are classified into the functional categories matching the test
-lifecycle events described earlier. All CSIT RF L2 and L1 KWs have been grouped
-into the following functional categories:
-
-#. Configuration;
-#. Test;
-#. Verification;
-#. StateReport;
-#. SuiteSetup;
-#. TestSetup;
-#. SuiteTeardown;
-#. TestTeardown;
-
-RF Keywords Naming Guidelines
------------------------------
-
-Readability counts: "..code is read much more often than it is written."
-Hence following a good and consistent grammar practice is important when
-writing :abbr:`RF (Robot Framework)` KeyWords and Tests. All CSIT test cases
-are coded using Gherkin style and include only L2 KWs references. L2 KWs are
-coded using simple style and include L2 KWs, L1 KWs, and L1 python references.
-To improve readability, the proposal is to use the same grammar for both
-:abbr:`RF (Robot Framework)` KW styles, and to formalize the grammar of English
-sentences used for naming the :abbr:`RF (Robot Framework)` KWs. :abbr:`RF (Robot
-Framework)` KWs names are short sentences expressing functional description of
-the command. They must follow English sentence grammar in one of the following
-forms:
-
-#. **Imperative** - verb-object(s): *"Do something"*, verb in base form.
-#. **Declarative** - subject–verb–object(s): *"Subject does something"*, verb in
- a third-person singular present tense form.
-#. **Affirmative** - modal_verb-verb-object(s): *"Subject should be something"*,
- *"Object should exist"*, verb in base form.
-#. **Negative** - modal_verb-Not-verb-object(s): *"Subject should not be
- something"*, *"Object should not exist"*, verb in base form.
-
-Passive form MUST NOT be used. However a usage of past participle as an
-adjective is okay. See usage examples provided in the Coding guidelines
-section below. Following sections list applicability of the above
-grammar forms to different :abbr:`RF (Robot Framework)` KW categories. Usage
-examples are provided, both good and bad.
-
-Coding guidelines
------------------
-
-Coding guidelines can be found on `Design optimizations wiki page
-<https://wiki.fd.io/view/CSIT/Design_Optimizations>`_.