perf: add TCP Nginx+LDPRELOAD suites

1. Suite steup add download nginx
2. Add nginx-1.14.2/1.15.0 ldp test suite
3. Add NginxUtils,NginxConfigGenerator method
4. Taskset the PID of nginx to the unused cores in VPP and these cores are under NIC's NUMA ID
5. cleanup add Kill Processes - nohup

Signed-off-by: xizhanx <xix.zhang@intel.com>
Change-Id: Idbf0e4ec3bf63e88281a8e3e34f52e00a6801c85
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

5 files changed, 203 insertions, 3 deletions

diff --git a/resources/libraries/robot/hoststack/hoststack.robot b/resources/libraries/robot/hoststack/hoststack.robot
index 075cc2b8bf..30363f9b91 100644
--- a/resources/libraries/robot/hoststack/hoststack.robot
+++ b/resources/libraries/robot/hoststack/hoststack.robot
@@ -1,4 +1,4 @@
-# Copyright (c) 2020 Cisco and/or its affiliates.
+# Copyright (c) 2021 Cisco and/or its affiliates.
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:
@@ -15,10 +15,13 @@
 | Library | resources.libraries.python.InterfaceUtil
 | Library | resources.libraries.python.IPUtil
 | Library | resources.libraries.python.HoststackUtil
+| Library | resources.libraries.python.NginxUtil
 | Library | resources.libraries.python.NsimUtil
+| Library | resources.tools.ab.ABTools
 | Variables | resources/libraries/python/Constants.py
 | Resource | resources/libraries/robot/ip/ip4.robot
 | Resource | resources/libraries/robot/nsim/nsim.robot
+| Resource | resources/libraries/robot/nginx/default.robot
 |
 | Documentation | L2 keywords to set up VPP to test hoststack.
 
@@ -106,6 +109,16 @@
 | ... | ip_address=${EMPTY}
 | ... | parallel=${1}
 | ... | time=${20}
+| &{nginx_server_attr}=
+| ... | role=server
+| ... | cpu_cnt=${1}
+| ... | cfg_vpp_feature=${Empty}
+| ... | namespace=default
+| ... | vcl_config=vcl_iperf3.conf
+| ... | ld_preload=${True}
+| ... | transparent_tls=${False}
+| ... | json=${True}
+| ... | ip_version=${4}
 
 *** Keywords ***
 | Set VPP Hoststack Attributes
@@ -539,3 +552,72 @@
 | | ... | ${vpp_nsim_attr} | ${iperf3_client}
 | | Then Set test message | ${client_output}
 | | Return From Keyword | ${client_defer_fail}
+
+| Set up LDP or VCL Nginx on DUT node
+| | [Documentation]
+| | ... | Setup for suites which uses VCL or LDP Nginx on DUT.
+| |
+| | ... | *Arguments:*
+| | ... | - dut - DUT node.
+| | ... | Type: string
+| | ... | - mode - VCL Nginx or LDP Nginx.
+| | ... | Type: string
+| | ... | - rps_cps - Test request or connect.
+| | ... | Type: string
+| | ... | - core_num - Nginx work processes number.
+| | ... | Type: int
+| | ... | - qat - Whether to use the qat engine.
+| | ... | Type: string
+| | ... | - tls_tcp - TLS or TCP.
+| |
+| | ... | *Example:*
+| |
+| | ... | \| Set up LDP or VCL NGINX on DUT node \| ${dut} |${mode}\
+| | ... | \| ${rps_cps} \| ${phy_cores} \| ${qat} \| ${tls_tcp} \|
+| |
+| | [Arguments] | ${dut} | ${mode} | ${rps_cps} | ${phy_cores} | ${qat}
+| | ... | ${tls_tcp}
+| |
+| | Set Interface State | ${dut} | ${DUT1_${int}1}[0] | up
+| | VPP Interface Set IP Address | ${dut} | ${DUT1_${int}1}[0]
+| | ... | ${dut_ip_addrs}[0] | ${dut_ip_prefix}
+| | Vpp Node Interfaces Ready Wait | ${dut}
+| | ${skip_cnt}= | Evaluate
+| | ... | ${CPU_CNT_SYSTEM} + ${CPU_CNT_MAIN} + ${vpp_hoststack_attr}[phy_cores]
+| | ${numa}= | Get interfaces numa node | ${dut} | ${DUT1_${int}1}[0]
+| | Apply Nginx configuration on DUT | ${dut} | ${phy_cores}
+| | Set To Dictionary | ${nginx_server_attr} | ip_address
+| | ... | ${dut_ip_addrs}[0]
+| | ${core_list}= | Cpu list per node str | ${dut} | ${numa}
+| | ... | skip_cnt=${skip_cnt} | cpu_cnt=${nginx_server_attr}[cpu_cnt]
+| | ${cpu_idle_list}= | Get cpu idle list | ${dut} | ${numa}
+| | ... | ${smt_used} | ${cpu_alloc_str}
+| | ${nginx_server}= | Get Nginx Command | ${nginx_server_attr}
+| | ... | ${nginx_version} | ${packages_dir}
+| | ${server_pid}= | Start Hoststack Test Program
+| | ... | ${dut} | ${nginx_server_attr}[namespace] | ${core_list}
+| | ... | ${nginx_server}
+| | Taskset Nginx PID to idle cores | ${dut} | ${cpu_idle_list}
+
+| Measure TLS requests or connections per second
+| | [Documentation]
+| | ... | Measure number of requests or connections per second using ab.
+| |
+| | ... | *Arguments:*
+| | ... | - ${ciphers} - Specify SSL/TLS cipher suite
+| | ... | - ${files} - Filename to be requested from the servers
+| | ... | - ${tls_tcp} - Test TLS or TCP.
+| | ... | - ${mode} - VCL Nginx or LDP Nginx.
+| |
+| | ... | *Example:*
+| |
+| | ... | \| Measure TLS requests or connections per second
+| | ... | \| AES128-SHA \| 64 \| tls \| rps \|
+| |
+| | [Arguments] | ${ciphers} | ${files} | ${tls_tcp} | ${mode}
+| |
+| | ${output}= | Run ab | ${tg} | ${dut_ip_addrs}[0] | ${ab_ip_addrs}[0]
+| | ... | ${tls_tcp} | ${ciphers} | ${files} | ${mode} | ${r_total} | ${c_total}
+| | ... | ${listen_port}
+| | Set test message | ${output}
+| | Log VPP Hoststack data | ${dut1}
diff --git a/resources/libraries/robot/nginx/default.robot b/resources/libraries/robot/nginx/default.robot
new file mode 100644
index 0000000000..5126da5858
--- /dev/null
+++ b/resources/libraries/robot/nginx/default.robot
@@ -0,0 +1,61 @@
+# Copyright (c) 2021 Intel and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+| Library | resources.libraries.python.InterfaceUtil
+| Library | resources.libraries.python.NginxConfigGenerator
+| Library | Collections
+
+*** Keywords ***
+| Apply Nginx configuration on DUT
+| | [Documentation]
+| | ... | Setup for suites which uses VCL or LDP Nginx on DUT.
+| |
+| | ... | *Arguments:*
+| | ... | - dut - DUT node.
+| | ... | Type: string
+| | ... | - phy_cores - vpp used phy cores number.
+| |
+| | ... | *Example:*
+| |
+| | ... | \| Apply Nginx configuration on DUT \| ${dut} | ${phy_cores}
+| |
+| | [Arguments] | ${dut} | ${phy_cores}
+| |
+| | Import Library | resources.libraries.python.NginxConfigGenerator
+| | ... | WITH NAME | nc_manager
+| | Run Keyword | nc_manager.Set Node | ${dut}
+| | Run Keyword | nc_manager.Set Nginx Path | ${packages_dir} | ${nginx_version}
+| | Run Keyword | nc_manager.Add Worker Processes | ${phy_cores} | ${smt_used}
+| | Run Keyword | nc_manager.Add Master Process
+| | Run Keyword | nc_manager.Add Daemon
+| | Run Keyword | nc_manager.Add Worker Rlimit Nofile
+| | Run Keyword | nc_manager.Add Events Use
+| | Run Keyword | nc_manager.Add Events Worker Connections
+| | Run Keyword | nc_manager.Add Events Accept Mutex
+| | Run Keyword | nc_manager.Add Events Multi Accept
+| | Run Keyword | nc_manager.Add Http Access Log
+| | Run Keyword | nc_manager.Add Http Include
+| | Run Keyword | nc_manager.Add Http Default Type
+| | Run Keyword | nc_manager.Add Http Sendfile
+| | Run Keyword | nc_manager.Add Http Keepalive Timeout | ${keep_time}
+| | Run Keyword If | ${keep_time} > 0
+| | ... | nc_manager.Add Http Keepalive Requests | ${r_total}
+| | Run Keyword | nc_manager.Add Http Server Listen | ${listen_port}
+| | Run Keyword | nc_manager.Add Http Server Root
+| | Run Keyword | nc_manager.Add Http Server Index
+| | Run Keyword | nc_manager.Add Http Server Location | ${0}
+| | Run Keyword | nc_manager.Add Http Server Location | ${64}
+| | Run Keyword | nc_manager.Add Http Server Location | ${1024}
+| | Run Keyword | nc_manager.Add Http Server Location | ${2048}
+| | Run Keyword | nc_manager.Apply Config
\ No newline at end of file
diff --git a/resources/libraries/robot/shared/suite_setup.robot b/resources/libraries/robot/shared/suite_setup.robot
index a9e57e7024..09cec67e4e 100644
--- a/resources/libraries/robot/shared/suite_setup.robot
+++ b/resources/libraries/robot/shared/suite_setup.robot
@@ -1,4 +1,4 @@
-# Copyright (c) 2020 Cisco and/or its affiliates.
+# Copyright (c) 2021 Cisco and/or its affiliates.
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:
@@ -16,6 +16,8 @@
 *** Settings ***
 | Library | resources.libraries.python.DPDK.DPDKTools
 | Library | resources.libraries.python.InterfaceUtil
+| Library | resources.libraries.python.NGINX.NGINXTools
+| Library | resources.tools.ab.ABTools
 | Library | resources.libraries.python.NodePath
 | Library | resources.libraries.python.topology.Topology
 | Library | resources.libraries.python.TrafficGenerator
@@ -198,3 +200,37 @@
 | | Configure crypto device on all DUTs | ${crypto_type} | numvfs=${numvfs}
 | | ... | force_init=${True}
 | | Configure kernel module on all DUTs | vfio_pci | force_load=${True}
+
+| Additional Suite Setup Action For nginx
+| | [Documentation]
+| | ... | Additional Setup for suites which uses Nginx.
+| |
+| | Install NGINX framework on all DUTs | ${nodes} | ${packages_dir}
+| | ... |  ${nginx_version}
+
+| Additional Suite Setup Action For ab
+| | [Documentation]
+| | ... | Additional Setup for suites which uses ab TG.
+| |
+| | Verify Program Installed | ${tg} | ab
+| | Iface update numa node | ${tg}
+| | ${running}= | Is TRex running | ${tg}
+| | Run keyword if | ${running}==${True} | Teardown traffic generator | ${tg}
+| | ${curr_driver}= | Get PCI dev driver | ${tg}
+| | ... | ${tg['interfaces']['${tg_if1}']['pci_address']}
+| | Run keyword if | '${curr_driver}'!='${None}'
+| | ... | PCI Driver Unbind | ${tg} |
+| | ... | ${tg['interfaces']['${tg_if1}']['pci_address']}
+| | ${driver}= | Get Variable Value | ${tg['interfaces']['${tg_if1}']['driver']}
+| | PCI Driver Bind | ${tg}
+| | ... | ${tg['interfaces']['${tg_if1}']['pci_address']} | ${driver}
+| | ${intf_name}= | Get Linux interface name | ${tg}
+| | ... | ${tg['interfaces']['${tg_if1}']['pci_address']}
+| | FOR | ${ip_addr} | IN | @{ab_ip_addrs}
+| | | ${ip_addr_on_intf}= | Linux interface has IP | ${tg} | ${intf_name}
+| | | ... | ${ip_addr} | ${ab_ip_prefix}
+| | | Run Keyword If | ${ip_addr_on_intf}==${False} | Set Linux interface IP
+| | | ... | ${tg} | ${intf_name} | ${ip_addr} | ${ab_ip_prefix}
+| | END
+| | Set Linux interface up | ${tg} | ${intf_name}
+| | Check ab | ${tg}
diff --git a/resources/libraries/robot/shared/suite_teardown.robot b/resources/libraries/robot/shared/suite_teardown.robot
index f164b0eeb5..20b2776eec 100644
--- a/resources/libraries/robot/shared/suite_teardown.robot
+++ b/resources/libraries/robot/shared/suite_teardown.robot
@@ -1,4 +1,4 @@
-# Copyright (c) 2020 Cisco and/or its affiliates.
+# Copyright (c) 2021 Cisco and/or its affiliates.
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:
@@ -35,6 +35,19 @@
 | | END
 | | Remove All Added VIF Ports On All DUTs From Topology | ${nodes}
 
+| Additional Suite Tear Down Action For ab
+| | [Documentation]
+| | ... | Additional teardown for suites which uses ab.
+| |
+| | ${intf_name}= | Get Linux interface name | ${tg}
+| | ... | ${tg['interfaces']['${tg_if1}']['pci_address']}
+| | FOR | ${ip_addr} | IN | @{ab_ip_addrs}
+| | | ${ip_addr_on_intf}= | Linux Interface Has IP | ${tg} | ${intf_name}
+| | | ... | ${ip_addr} | ${ab_ip_prefix}
+| | | Run Keyword If | ${ip_addr_on_intf}==${True} | Delete Linux Interface IP
+| | | ... | ${tg} | ${intf_name} | ${ip_addr} | ${ab_ip_prefix}
+| | END
+
 | Additional Suite Tear Down Action For performance
 | | [Documentation]
 | | ... | Additional teardown for suites which uses performance measurement.
diff --git a/resources/libraries/robot/shared/test_teardown.robot b/resources/libraries/robot/shared/test_teardown.robot
index 18be67cfc5..977a87d5a6 100644
--- a/resources/libraries/robot/shared/test_teardown.robot
+++ b/resources/libraries/robot/shared/test_teardown.robot
@@ -86,6 +86,14 @@
 | | | Destroy all '${container_group}' containers
 | | END
 
+| Additional Test Tear Down Action For nginx
+| | [Documentation]
+| | ... | Additional teardown for tests which uses nginx.
+| |
+| | FOR | ${dut} | IN | @{duts}
+| | | Kill Program | ${nodes['${dut}']} | nginx
+| | END
+
 | Additional Test Tear Down Action For det44
 | | [Documentation]
 | | ... | Additional teardown for tests which uses DET44 feature.