From 2b23529065c5db02ebd3aadc093397d7614e6671 Mon Sep 17 00:00:00 2001 From: Dave Wallace Date: Tue, 11 Feb 2020 00:57:10 +0000 Subject: perf: Clean up Hoststack tests - Update test names with clients/streams - Convert test results to JSON output * iperf3 results include bits_per_second * vpp_echo results include both client and server output which includes time in seconds and rx_data/tx_data in bytes which can be used to calculate the average bits per second. Tx and Rx data will always be the same: BPS = (client tx_data * 8) / ((client time + server time) / 2) - Fix WRK test results data formatting errors Change-Id: Ie2aeb665e3cc0739b16f97ba2628eebe6e041d22 Signed-off-by: Dave Wallace (cherry picked from commit 5570bf3ab49301201dd7607bb4f8de67fd8f16dc) --- ...ge2p1x710-eth-ip4tcp-ldpreload-iperf3-bps.robot | 64 ------------------- ...x710-eth-ip4tcp-nsim-ldpreload-iperf3-bps.robot | 66 ------------------- ...1x710-eth-ip4tcpbase-ldpreload-iperf3-bps.robot | 60 ++++++++++++++++++ ...-eth-ip4tcpbase-nsim-ldpreload-iperf3-bps.robot | 62 ++++++++++++++++++ ...th-ip4tcpscale1cl10s-ldpreload-iperf3-bps.robot | 62 ++++++++++++++++++ ...4tcpscale1cl10s-nsim-ldpreload-iperf3-bps.robot | 64 +++++++++++++++++++ .../10ge2p1x710-eth-ip4udpquic-vppecho-bps.robot | 74 ---------------------- ...0ge2p1x710-eth-ip4udpquicbase-vppecho-bps.robot | 61 ++++++++++++++++++ ...10-eth-ip4udpquicscale10cl10s-vppecho-bps.robot | 66 +++++++++++++++++++ ...710-eth-ip4udpquicscale10cl1s-vppecho-bps.robot | 65 +++++++++++++++++++ ...710-eth-ip4udpquicscale1cl10s-vppecho-bps.robot | 65 +++++++++++++++++++ ...ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-cps.robot | 2 +- ...ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-rps.robot | 2 +- 13 files changed, 507 insertions(+), 206 deletions(-) delete mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-ldpreload-iperf3-bps.robot delete mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-nsim-ldpreload-iperf3-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-ldpreload-iperf3-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-nsim-ldpreload-iperf3-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-ldpreload-iperf3-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-nsim-ldpreload-iperf3-bps.robot delete mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquic-vppecho-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicbase-vppecho-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl10s-vppecho-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl1s-vppecho-bps.robot create mode 100644 tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale1cl10s-vppecho-bps.robot (limited to 'tests') diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-ldpreload-iperf3-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-ldpreload-iperf3-bps.robot deleted file mode 100644 index a907c61e69..0000000000 --- a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-ldpreload-iperf3-bps.robot +++ /dev/null @@ -1,64 +0,0 @@ -# Copyright (c) 2020 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Library | resources.libraries.python.HoststackUtil -| Resource | resources/libraries/robot/shared/default.robot -| Resource | resources/libraries/robot/hoststack/hoststack.robot -| -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV -| ... | TCP | NIC_Intel-X710 | DRV_VFIO_PCI | HOSTSTACK -| ... | LDPRELOAD | IPERF3 | eth-ip4tcp-ldpreload-iperf3 -| -| Suite Setup | Setup suite single link no tg -| Suite Teardown | Tear down suite | hoststack -| Test Setup | Setup test -| Test Teardown | Tear down test -| -| Test Template | Local template -| -| Documentation | *Iperf3 client -> Iperf3 server throughput. -| -| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology -| ... | with single link between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP -| ... | *[Cfg] DUT configuration:* -| ... | *[Ref] Applicable standard specifications:* - -*** Variables *** -| @{plugins_to_enable}= | dpdk_plugin.so -| ${nic_name}= | Intel-X710 -| ${nic_driver}= | vfio-pci -| ${overhead}= | ${0} -| ${frame_size}= | ${9000} -| ${crypto_type}= | ${None} - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${clients} | ${streams} -| | -| | Set Test Variable | ${dpdk_no_tx_checksum_offload} | ${False} -| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} -| | Set Iperf3 Client Attributes | parallel=${streams} -| | ${no_results}= | Get Test Results From Hoststack Iperf3 Test -| | Run Keyword If | ${no_results}==True | FAIL -| | ... | No Test Results From Iperf3 client - -*** Test Cases *** -| tc01-9000B-1c-eth-ip4tcp-ldpreload-iperf3-bps -| | [Tags] | 1C | 1CLIENT | 1STREAM -| | phy_cores=${1} | clients=${1} | streams=${1} - -| tc02-9000B-1c-eth-ip4tcp-ldpreload-iperf3-bps -| | [Tags] | 1C | 1CLIENT | 10STREAM -| | phy_cores=${1} | clients=${1} | streams=${10} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-nsim-ldpreload-iperf3-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-nsim-ldpreload-iperf3-bps.robot deleted file mode 100644 index 0a98ee3acd..0000000000 --- a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcp-nsim-ldpreload-iperf3-bps.robot +++ /dev/null @@ -1,66 +0,0 @@ -# Copyright (c) 2020 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Library | resources.libraries.python.HoststackUtil -| Resource | resources/libraries/robot/shared/default.robot -| Resource | resources/libraries/robot/hoststack/hoststack.robot -| -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV -| ... | TCP | NIC_Intel-X710 | DRV_VFIO_PCI | HOSTSTACK -| ... | NSIM | LDPRELOAD | IPERF3 | eth-ip4tcp-nsim-ldpreload-iperf3 -| -| Suite Setup | Setup suite single link no tg -| Suite Teardown | Tear down suite -| Test Setup | Setup test -| Test Teardown | Tear down test -| -| Test Template | Local template -| -| Documentation | *Iperf3 client -> Iperf3 server throughput. -| -| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology -| ... | with single link between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP -| ... | *[Cfg] DUT configuration:* -| ... | *[Ref] Applicable standard specifications:* - -*** Variables *** -| @{plugins_to_enable}= | dpdk_plugin.so | nsim_plugin.so -| ${nic_name}= | Intel-X710 -| ${nic_driver}= | vfio-pci -| ${overhead}= | ${0} -| ${frame_size}= | ${9000} -| ${crypto_type}= | ${None} -| ${pkts_per_drop}= | ${100} - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${clients} | ${streams} -| | -| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} -| | Set Iperf3 Client Attributes | parallel=${streams} -| | Set VPP NSIM Attributes | output_feature_enable=${True} | -| | ... | packets_per_drop=${pkts_per_drop} -| | ${no_results}= | Get Test Results From Hoststack Iperf3 Test -| | Run Keyword If | ${no_results}==True | FAIL -| | ... | No Test Results From Iperf3 client - -*** Test Cases *** -| tc01-9000B-1c-eth-ip4tcp-nsim-ldpreload-iperf3-bps -| | [Tags] | 1C | 1CLIENT | 1STREAM -| | phy_cores=${1} | clients=${1} | streams=${1} - -| tc02-9000B-1c-eth-ip4tcp-nsim-ldpreload-iperf3-bps -| | [Tags] | 1C | 1CLIENT | 10STREAM -| | phy_cores=${1} | clients=${1} | streams=${10} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-ldpreload-iperf3-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-ldpreload-iperf3-bps.robot new file mode 100644 index 0000000000..f2c7165059 --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-ldpreload-iperf3-bps.robot @@ -0,0 +1,60 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | TCP | NIC_Intel-X710 | DRV_VFIO_PCI | HOSTSTACK +| ... | LDPRELOAD | IPERF3 | 1CLIENT | 1STREAM | 9000B +| ... | eth-ip4tcpbase-ldpreload-iperf3 +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite | hoststack +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *Iperf3 client -> Iperf3 server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set Test Variable | ${dpdk_no_tx_checksum_offload} | ${False} +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | ${defer_fail}= | Get Test Results From Hoststack Iperf3 Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack Iperf3 Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4tcpbase-ldpreload-iperf3-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-nsim-ldpreload-iperf3-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-nsim-ldpreload-iperf3-bps.robot new file mode 100644 index 0000000000..2eaed06173 --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpbase-nsim-ldpreload-iperf3-bps.robot @@ -0,0 +1,62 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | TCP | NIC_Intel-X710 | DRV_VFIO_PCI | HOSTSTACK +| ... | NSIM | LDPRELOAD | IPERF3 | 1CLIENT | 1STREAM | 9000B +| ... | eth-ip4tcpbase-nsim-ldpreload-iperf3 +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *Iperf3 client -> Iperf3 server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so | nsim_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} +| ${pkts_per_drop}= | ${100} + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | Set VPP NSIM Attributes | output_feature_enable=${True} | +| | ... | packets_per_drop=${pkts_per_drop} +| | ${defer_fail}= | Get Test Results From Hoststack Iperf3 Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack Iperf3 Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4tcpbase-nsim-ldpreload-iperf3-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-ldpreload-iperf3-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-ldpreload-iperf3-bps.robot new file mode 100644 index 0000000000..7d314bb1c6 --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-ldpreload-iperf3-bps.robot @@ -0,0 +1,62 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | TCP | NIC_Intel-X710 | DRV_VFIO_PCI | HOSTSTACK +| ... | LDPRELOAD | IPERF3 | 1CLIENT | 10STREAM | 9000B +| ... | eth-ip4tcpscale1cl10s-ldpreload-iperf3 +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite | hoststack +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *Iperf3 client -> Iperf3 server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} +| ${streams} | ${10} + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set Test Variable | ${dpdk_no_tx_checksum_offload} | ${False} +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | Set Iperf3 Client Attributes | parallel=${streams} +| | ${defer_fail}= | Get Test Results From Hoststack Iperf3 Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack Iperf3 Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4tcpscale1cl10s-ldpreload-iperf3-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-nsim-ldpreload-iperf3-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-nsim-ldpreload-iperf3-bps.robot new file mode 100644 index 0000000000..036414ffd7 --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4tcpscale1cl10s-nsim-ldpreload-iperf3-bps.robot @@ -0,0 +1,64 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | TCP | NIC_Intel-X710 | DRV_VFIO_PCI | HOSTSTACK +| ... | NSIM | LDPRELOAD | IPERF3 | 1CLIENT | 10STREAM | 9000B +| ... | eth-ip4tcpscale1cl10s-nsim-ldpreload-iperf3 +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *Iperf3 client -> Iperf3 server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so | nsim_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} +| ${pkts_per_drop}= | ${100} +| ${streams}= | ${10} + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | Set Iperf3 Client Attributes | parallel=${streams} +| | Set VPP NSIM Attributes | output_feature_enable=${True} | +| | ... | packets_per_drop=${pkts_per_drop} +| | ${defer_fail}= | Get Test Results From Hoststack Iperf3 Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack Iperf3 Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4tcpscale1cl10s-nsim-ldpreload-iperf3-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquic-vppecho-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquic-vppecho-bps.robot deleted file mode 100644 index a93afa974a..0000000000 --- a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquic-vppecho-bps.robot +++ /dev/null @@ -1,74 +0,0 @@ -# Copyright (c) 2020 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Library | resources.libraries.python.HoststackUtil -| Resource | resources/libraries/robot/shared/default.robot -| Resource | resources/libraries/robot/hoststack/hoststack.robot -| -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV -| ... | NIC_Intel-X710 | DRV_VFIO_PCI | UDP | QUIC | VPPECHO -| ... | HOSTSTACK | eth-ip4udpquic-vppecho -| -| Suite Setup | Setup suite single link no tg -| Suite Teardown | Tear down suite -| Test Setup | Setup test -| Test Teardown | Tear down test -| -| Test Template | Local template -| -| Documentation | *QUIC Unidirectional Echo Client -> Echo Server throughput. -| -| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology -| ... | with single link between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP-QUIC -| ... | *[Cfg] DUT configuration:* -| ... | *[Ref] Applicable standard specifications:* - -*** Variables *** -| @{plugins_to_enable}= | dpdk_plugin.so | quic_plugin.so -| ${nic_name}= | Intel-X710 -| ${nic_driver}= | vfio-pci -| ${overhead}= | ${0} -| ${frame_size}= | ${9000} -| ${crypto_type}= | ${None} - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${clients} | ${streams} | ${bytes} -| | -| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} -| | Set VPP Echo Server Attributes | cfg_vpp_feature=quic | nclients=${clients} -| | ... | quic_streams=${streams} | rx_bytes=${bytes} -| | Set VPP Echo Client Attributes | cfg_vpp_feature=quic | nclients=${clients} -| | ... | quic_streams=${streams} | tx_bytes=${bytes} -| | ${no_results}= | Get Test Results From Hoststack VPP Echo Test -| | Run Keyword If | ${no_results}==True | FAIL -| | ... | No Test Results From External Hoststack Apps - -*** Test Cases *** -| tc01-9000B-1c-eth-ip4udpquic-vppecho-bps -| | [Tags] | 1C | 1CLIENT | 1STREAM -| | phy_cores=${1} | clients=${1} | streams=${1} | bytes=10G - -| tc02-9000B-1c-eth-ip4udpquic-vppecho-bps -| | [Tags] | 1C | 1CLIENT | 10STREAM -| | phy_cores=${1} | clients=${1} | streams=${10} | bytes=1G - -| tc03-9000B-1c-eth-ip4udpquic-vppecho-bps -| | [Tags] | 1C | 10CLIENT | 1STREAM -| | phy_cores=${1} | clients=${10} | streams=${1} | bytes=1G - -| tc04-9000B-1c-eth-ip4udpquic-vppecho-bps -| | [Tags] | 1C | 10CLIENT | 10STREAM -| | phy_cores=${1} | clients=${10} | streams=${10} | bytes=100M diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicbase-vppecho-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicbase-vppecho-bps.robot new file mode 100644 index 0000000000..564844e623 --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicbase-vppecho-bps.robot @@ -0,0 +1,61 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | NIC_Intel-X710 | DRV_VFIO_PCI | UDP | QUIC | VPPECHO +| ... | 1CLIENT | 1STREAM | HOSTSTACK | 9000B | eth-ip4udpquicbase-vppecho +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *QUIC Unidirectional Echo Client -> Echo Server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP-QUIC +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so | quic_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} +| ${bytes}= | 10G + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | Set VPP Echo Server Attributes | cfg_vpp_feature=quic | rx_bytes=${bytes} +| | Set VPP Echo Client Attributes | cfg_vpp_feature=quic | tx_bytes=${bytes} +| | ${defer_fail}= | Get Test Results From Hoststack VPP Echo Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack VPP Echo Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4udpquicbase-vppecho-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl10s-vppecho-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl10s-vppecho-bps.robot new file mode 100644 index 0000000000..f4e5c146fb --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl10s-vppecho-bps.robot @@ -0,0 +1,66 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | NIC_Intel-X710 | DRV_VFIO_PCI | UDP | QUIC | VPPECHO +| ... | HOSTSTACK | 10CLIENT | 10STREAM | 9000B +| ... | eth-ip4udpquicscale10cl10s-vppecho +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *QUIC Unidirectional Echo Client -> Echo Server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP-QUIC +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so | quic_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} +| ${clients}= | ${10} +| ${streams}= | ${10} +| ${bytes}= | 100M + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | Set VPP Echo Server Attributes | cfg_vpp_feature=quic | nclients=${clients} +| | ... | quic_streams=${streams} | rx_bytes=${bytes} +| | Set VPP Echo Client Attributes | cfg_vpp_feature=quic | nclients=${clients} +| | ... | quic_streams=${streams} | tx_bytes=${bytes} +| | ${defer_fail}= | Get Test Results From Hoststack VPP Echo Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack VPP Echo Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4udpquicscale10cl10s-vppecho-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl1s-vppecho-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl1s-vppecho-bps.robot new file mode 100644 index 0000000000..e160f0562f --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale10cl1s-vppecho-bps.robot @@ -0,0 +1,65 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | NIC_Intel-X710 | DRV_VFIO_PCI | UDP | QUIC | VPPECHO +| ... | HOSTSTACK | 10CLIENT | 1STREAM | 9000B +| ... | eth-ip4udpquicscale10cl1s-vppecho +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *QUIC Unidirectional Echo Client -> Echo Server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP-QUIC +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so | quic_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} +| ${clients}= | ${10} +| ${bytes}= | 1G + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | Set VPP Echo Server Attributes | cfg_vpp_feature=quic | nclients=${clients} +| | ... | rx_bytes=${bytes} +| | Set VPP Echo Client Attributes | cfg_vpp_feature=quic | nclients=${clients} +| | ... | tx_bytes=${bytes} +| | ${defer_fail}= | Get Test Results From Hoststack VPP Echo Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack VPP Echo Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4udpquicscale10cl1s-vppecho-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale1cl10s-vppecho-bps.robot b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale1cl10s-vppecho-bps.robot new file mode 100644 index 0000000000..d03e247df2 --- /dev/null +++ b/tests/vpp/perf/hoststack/10ge2p1x710-eth-ip4udpquicscale1cl10s-vppecho-bps.robot @@ -0,0 +1,65 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.HoststackUtil +| Resource | resources/libraries/robot/shared/default.robot +| Resource | resources/libraries/robot/hoststack/hoststack.robot +| +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV +| ... | NIC_Intel-X710 | DRV_VFIO_PCI | UDP | QUIC | VPPECHO +| ... | HOSTSTACK | 1CLIENT | 10STREAM | 9000B +| ... | eth-ip4udpquicscale1cl10s-vppecho +| +| Suite Setup | Setup suite single link no tg +| Suite Teardown | Tear down suite +| Test Setup | Setup test +| Test Teardown | Tear down test +| +| Test Template | Local template +| +| Documentation | *QUIC Unidirectional Echo Client -> Echo Server throughput. +| +| ... | *[Top] Network Topologies:* DUT-DUT 2-node topology +| ... | with single link between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP-QUIC +| ... | *[Cfg] DUT configuration:* +| ... | *[Ref] Applicable standard specifications:* + +*** Variables *** +| @{plugins_to_enable}= | dpdk_plugin.so | quic_plugin.so +| ${nic_name}= | Intel-X710 +| ${nic_driver}= | vfio-pci +| ${overhead}= | ${0} +| ${frame_size}= | ${9000} +| ${crypto_type}= | ${None} +| ${streams}= | ${10} +| ${bytes}= | 1G + +*** Keywords *** +| Local template +| | [Arguments] | ${phy_cores} +| | +| | Set VPP Hoststack Attributes | phy_cores=${phy_cores} +| | Set VPP Echo Server Attributes | cfg_vpp_feature=quic +| | ... | quic_streams=${streams} | rx_bytes=${bytes} +| | Set VPP Echo Client Attributes | cfg_vpp_feature=quic +| | ... | quic_streams=${streams} | tx_bytes=${bytes} +| | ${defer_fail}= | Get Test Results From Hoststack VPP Echo Test +| | Run Keyword If | ${defer_fail}==True | FAIL +| | ... | Defered Failure From Hoststack VPP Echo Test Program + +*** Test Cases *** +| tc01-9000B-1c-eth-ip4udpquicscale1cl10s-vppecho-bps +| | [Tags] | 1C +| | phy_cores=${1} diff --git a/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-cps.robot b/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-cps.robot index d7a7e6cf88..6b4f3c30ab 100644 --- a/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-cps.robot +++ b/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-cps.robot @@ -19,7 +19,7 @@ | Resource | resources/libraries/robot/hoststack/tcp_setup.robot | | Force Tags | 2_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV -| ... | HTTP | TCP | TCP_CPS | NIC_Intel-X710 | DRV_VFIO_PCI +| ... | HTTP | TCP | TCP_CPS | NIC_Intel-X710 | DRV_VFIO_PCI | IMIX | ... | eth-ip4tcphttp-wrk8u8c50con | | Suite Setup | Setup suite single link | wrk diff --git a/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-rps.robot b/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-rps.robot index 27469bb189..b9b2226855 100644 --- a/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-rps.robot +++ b/tests/vpp/perf/hoststack/2n1l-10ge2p1x710-eth-ip4tcphttp-wrk8u8c50con-rps.robot @@ -19,7 +19,7 @@ | Resource | resources/libraries/robot/hoststack/tcp_setup.robot | | Force Tags | 2_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV -| ... | HTTP | TCP | TCP_RPS | NIC_Intel-X710 | DRV_VFIO_PCI +| ... | HTTP | TCP | TCP_RPS | NIC_Intel-X710 | DRV_VFIO_PCI | IMIX | ... | eth-ip4tcphttp-wrk8u8c50con | | Suite Setup | Setup suite single link | wrk -- cgit 1.2.3-korg