aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report
diff options
context:
space:
mode:
Diffstat (limited to 'docs/report')
-rw-r--r--docs/report/introduction/test_environment_intro.rst5
-rw-r--r--docs/report/introduction/test_environment_sut_calib_clx.rst224
-rw-r--r--docs/report/introduction/test_environment_sut_meltspec_clx.rst154
3 files changed, 381 insertions, 2 deletions
diff --git a/docs/report/introduction/test_environment_intro.rst b/docs/report/introduction/test_environment_intro.rst
index 721b4142e5..e0df3b64ff 100644
--- a/docs/report/introduction/test_environment_intro.rst
+++ b/docs/report/introduction/test_environment_intro.rst
@@ -15,8 +15,8 @@ topology types are used:
server as TG both connected in ring topology.
Tested SUT servers are based on a range of processors including Intel
-Xeon Haswell-SP, Intel Xeon Skylake-SP, Arm, Intel Atom. More detailed
-description is provided in
+Xeon Haswell-SP, Intel Xeon Skylake-SP, Intel Xeon Cascadelake-SP, Arm, Intel
+Atom. More detailed description is provided in
:ref:`tested_physical_topologies`. Tested logical topologies are
described in :ref:`tested_logical_topologies`.
@@ -25,6 +25,7 @@ Server Specifications
Complete technical specifications of compute servers used in CSIT
physical testbeds are maintained in FD.io CSIT repository:
+`FD.io CSIT testbeds - Xeon Cascadelake`_,
`FD.io CSIT testbeds - Xeon Skylake, Arm, Atom`_ and
`FD.io CSIT Testbeds - Xeon Haswell`_.
diff --git a/docs/report/introduction/test_environment_sut_calib_clx.rst b/docs/report/introduction/test_environment_sut_calib_clx.rst
new file mode 100644
index 0000000000..43e2f599ea
--- /dev/null
+++ b/docs/report/introduction/test_environment_sut_calib_clx.rst
@@ -0,0 +1,224 @@
+Calibration Data - Cascadelake
+------------------------------
+
+Following sections include sample calibration data measured on
+s32-t27-sut1 server running in one of the Intel Xeon Skylake testbeds as
+specified in `FD.io CSIT testbeds - Xeon Cascadelake`_.
+
+Calibration data obtained from all other servers in Cascadelake testbeds
+shows the same or similar values.
+
+
+Linux cmdline
+~~~~~~~~~~~~~
+
+::
+
+ $ cat /proc/cmdline
+ BOOT_IMAGE=/boot/vmlinuz-4.15.0-60-generic root=UUID=1d03969e-a2a0-41b2-a97e-1cc171b07e88 ro isolcpus=1-23,25-47,49-71,73-95 nohz_full=1-23,25-47,49-71,73-95 rcu_nocbs=1-23,25-47,49-71,73-95 numa_balancing=disable intel_pstate=disable intel_iommu=on iommu=pt nmi_watchdog=0 audit=0 nosoftlockup processor.max_cstate=1 intel_idle.max_cstate=1 hpet=disable tsc=reliable mce=off console=tty0 console=ttyS0,115200n8
+
+Linux uname
+~~~~~~~~~~~
+
+::
+
+ $ uname -a
+ Linux s32-t27-sut1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
+
+
+System-level Core Jitter
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ $ sudo taskset -c 3 /home/testuser/pma_tools/jitter/jitter -i 30
+ Linux Jitter testing program version 1.9
+ Iterations=30
+ The pragram will execute a dummy function 80000 times
+ Display is updated every 20000 displayUpdate intervals
+ Thread affinity will be set to core_id:7
+ Timings are in CPU Core cycles
+ Inst_Min: Minimum Excution time during the display update interval(default is ~1 second)
+ Inst_Max: Maximum Excution time during the display update interval(default is ~1 second)
+ Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest
+ last_Exec: The Excution time of last iteration just before the display update
+ Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset
+ Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset
+ tmp: Cumulative value calcualted by the dummy function
+ Interval: Time interval between the display updates in Core Cycles
+ Sample No: Sample number
+
+ Inst_Min,Inst_Max,Inst_jitter,last_Exec,Abs_min,Abs_max,tmp,Interval,Sample No
+ 160022,167590,7568,160026,160022,167590,2057568256,3203711852,1
+ 160022,170628,10606,160024,160022,170628,4079222784,3204010824,2
+ 160022,169824,9802,160024,160022,170628,1805910016,3203812064,3
+ 160022,168832,8810,160030,160022,170628,3827564544,3203792594,4
+ 160022,168248,8226,160026,160022,170628,1554251776,3203765920,5
+ 160022,167834,7812,160028,160022,170628,3575906304,3203761114,6
+ 160022,167442,7420,160024,160022,170628,1302593536,3203769250,7
+ 160022,169120,9098,160028,160022,170628,3324248064,3203853340,8
+ 160022,170710,10688,160024,160022,170710,1050935296,3203985878,9
+ 160022,167952,7930,160024,160022,170710,3072589824,3203733756,10
+ 160022,168314,8292,160030,160022,170710,799277056,3203741152,11
+ 160022,169672,9650,160024,160022,170710,2820931584,3203739910,12
+ 160022,168684,8662,160024,160022,170710,547618816,3203727336,13
+ 160022,168246,8224,160024,160022,170710,2569273344,3203739052,14
+ 160022,168134,8112,160030,160022,170710,295960576,3203735874,15
+ 160022,170230,10208,160024,160022,170710,2317615104,3203996356,16
+ 160022,167190,7168,160024,160022,170710,44302336,3203713628,17
+ 160022,167304,7282,160024,160022,170710,2065956864,3203717954,18
+ 160022,167500,7478,160024,160022,170710,4087611392,3203706674,19
+ 160022,167302,7280,160024,160022,170710,1814298624,3203726452,20
+ 160022,167266,7244,160024,160022,170710,3835953152,3203702804,21
+ 160022,167820,7798,160022,160022,170710,1562640384,3203719138,22
+ 160022,168100,8078,160024,160022,170710,3584294912,3203716636,23
+ 160022,170408,10386,160024,160022,170710,1310982144,3203946958,24
+ 160022,167276,7254,160024,160022,170710,3332636672,3203706236,25
+ 160022,167052,7030,160024,160022,170710,1059323904,3203696444,26
+ 160022,170322,10300,160024,160022,170710,3080978432,3203747514,27
+ 160022,167332,7310,160024,160022,170710,807665664,3203716210,28
+ 160022,167426,7404,160026,160022,170710,2829320192,3203700630,29
+ 160022,168840,8818,160024,160022,170710,556007424,3203727658,30
+
+
+Memory Bandwidth
+~~~~~~~~~~~~~~~~
+
+::
+
+ $ sudo /home/testuser/mlc --bandwidth_matrix
+ Intel(R) Memory Latency Checker - v3.7
+ Command line parameters: --bandwidth_matrix
+
+ Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
+ Measuring Memory Bandwidths between nodes within system
+ Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
+ Using all the threads from each core if Hyper-threading is enabled
+ Using Read-only traffic type
+ Numa node
+ Numa node 0 1
+ 0 122097.7 51327.9
+ 1 51309.2 122005.5
+
+::
+
+ $ sudo /home/testuser/mlc --peak_injection_bandwidth
+ Intel(R) Memory Latency Checker - v3.7
+ Command line parameters: --peak_injection_bandwidth
+
+ Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
+
+ Measuring Peak Injection Memory Bandwidths for the system
+ Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
+ Using all the threads from each core if Hyper-threading is enabled
+ Using traffic with the following read-write ratios
+ ALL Reads : 243159.4
+ 3:1 Reads-Writes : 219132.5
+ 2:1 Reads-Writes : 216603.1
+ 1:1 Reads-Writes : 203713.0
+ Stream-triad like: 193790.8
+
+::
+
+ $ sudo /home/testuser/mlc --max_bandwidth
+ Intel(R) Memory Latency Checker - v3.7
+ Command line parameters: --max_bandwidth
+
+ Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
+
+ Measuring Maximum Memory Bandwidths for the system
+ Will take several minutes to complete as multiple injection rates will be tried to get the best bandwidth
+ Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
+ Using all the threads from each core if Hyper-threading is enabled
+ Using traffic with the following read-write ratios
+ ALL Reads : 244114.27
+ 3:1 Reads-Writes : 219441.97
+ 2:1 Reads-Writes : 216603.72
+ 1:1 Reads-Writes : 203679.09
+ Stream-triad like: 214902.80
+
+
+Memory Latency
+~~~~~~~~~~~~~~
+
+::
+
+ $ sudo /home/testuser/mlc --latency_matrix
+ Intel(R) Memory Latency Checker - v3.7
+ Command line parameters: --latency_matrix
+
+ Using buffer size of 2000.000MiB
+ Measuring idle latencies (in ns)...
+ Numa node
+ Numa node 0 1
+ 0 81.2 130.2
+ 1 130.2 81.1
+
+::
+
+ $ sudo /home/testuser/mlc --idle_latency
+ Intel(R) Memory Latency Checker - v3.7
+ Command line parameters: --idle_latency
+
+ Using buffer size of 2000.000MiB
+ Each iteration took 186.1 core clocks ( 80.9 ns)
+
+::
+
+ $ sudo /home/testuser/mlc --loaded_latency
+ Intel(R) Memory Latency Checker - v3.7
+ Command line parameters: --loaded_latency
+
+ Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
+
+ Measuring Loaded Latencies for the system
+ Using all the threads from each core if Hyper-threading is enabled
+ Using Read-only traffic type
+ Inject Latency Bandwidth
+ Delay (ns) MB/sec
+ ==========================
+ 00000 233.86 243421.9
+ 00002 230.61 243544.1
+ 00008 232.56 243394.5
+ 00015 229.52 244076.6
+ 00050 225.82 244290.6
+ 00100 161.65 236744.8
+ 00200 100.63 133844.0
+ 00300 96.84 90548.2
+ 00400 95.71 68504.3
+ 00500 95.68 55139.0
+ 00700 88.77 39798.4
+ 01000 84.74 28200.1
+ 01300 83.08 21915.5
+ 01700 82.27 16969.3
+ 02500 81.66 11810.6
+ 03500 81.98 8662.9
+ 05000 81.48 6306.8
+ 09000 81.17 3857.8
+ 20000 80.19 2179.9
+
+
+L1/L2/LLC Latency
+~~~~~~~~~~~~~~~~~
+
+::
+
+ $ sudo /home/testuser/mlc --c2c_latency
+ Intel(R) Memory Latency Checker - v3.7
+ Command line parameters: --c2c_latency
+
+ Measuring cache-to-cache transfer latency (in ns)...
+ Local Socket L2->L2 HIT latency 55.5
+ Local Socket L2->L2 HITM latency 55.6
+ Remote Socket L2->L2 HITM latency (data address homed in writer socket)
+ Reader Numa Node
+ Writer Numa Node 0 1
+ 0 - 115.6
+ 1 115.6 -
+ Remote Socket L2->L2 HITM latency (data address homed in reader socket)
+ Reader Numa Node
+ Writer Numa Node 0 1
+ 0 - 178.2
+ 1 178.4 -
+
+.. include:: ../introduction/test_environment_sut_meltspec_clx.rst
diff --git a/docs/report/introduction/test_environment_sut_meltspec_clx.rst b/docs/report/introduction/test_environment_sut_meltspec_clx.rst
new file mode 100644
index 0000000000..9056be839b
--- /dev/null
+++ b/docs/report/introduction/test_environment_sut_meltspec_clx.rst
@@ -0,0 +1,154 @@
+Spectre and Meltdown Checks
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Following section displays the output of a running shell script to tell if
+system is vulnerable against the several speculative execution CVEs that were
+made public in 2018. Script is available on `Spectre & Meltdown Checker Github
+<https://github.com/speed47/spectre-meltdown-checker>`_.
+
+::
+
+ Spectre and Meltdown mitigation detection tool v0.42
+
+ Checking for vulnerabilities on current system
+ Kernel is Linux 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64
+ CPU is Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz
+
+ Hardware check
+ * Hardware support (CPU microcode) for mitigation techniques
+ * Indirect Branch Restricted Speculation (IBRS)
+ * SPEC_CTRL MSR is available: YES
+ * CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
+ * Indirect Branch Prediction Barrier (IBPB)
+ * PRED_CMD MSR is available: YES
+ * CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
+ * Single Thread Indirect Branch Predictors (STIBP)
+ * SPEC_CTRL MSR is available: YES
+ * CPU indicates STIBP capability: YES (Intel STIBP feature bit)
+ * Speculative Store Bypass Disable (SSBD)
+ * CPU indicates SSBD capability: YES (Intel SSBD)
+ * L1 data cache invalidation
+ * FLUSH_CMD MSR is available: YES
+ * CPU indicates L1D flush capability: YES (L1D flush feature bit)
+ * Microarchitecture Data Sampling
+ * VERW instruction is available: YES (MD_CLEAR feature bit)
+ * Enhanced IBRS (IBRS_ALL)
+ * CPU indicates ARCH_CAPABILITIES MSR availability: YES
+ * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: YES
+ * CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): YES
+ * CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
+ * CPU/Hypervisor indicates L1D flushing is not necessary on this system: YES
+ * Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
+ * CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): YES
+ * CPU supports Software Guard Extensions (SGX): NO
+ * CPU microcode is known to cause stability problems: NO (model 0x55 family 0x6 stepping 0x7 ucode 0x5000021 cpuid 0x50657)
+ * CPU microcode is the latest known available version: awk: fatal: cannot open file `bash' for reading (No file or directory)
+ UNKNOWN (latest microcode version for your CPU model is unknown)
+ * CPU vulnerability to the speculative execution attack variants
+ * Vulnerable to CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
+ * Vulnerable to CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
+ * Vulnerable to CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
+ * Vulnerable to CVE-2018-3640 (Variant 3a, rogue system register read): YES
+ * Vulnerable to CVE-2018-3639 (Variant 4, speculative store bypass): YES
+ * Vulnerable to CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
+ * Vulnerable to CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): NO
+ * Vulnerable to CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): NO
+ * Vulnerable to CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
+ * Vulnerable to CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
+ * Vulnerable to CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
+ * Vulnerable to CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO
+
+ CVE-2017-5753 aka 'Spectre Variant 1, bounds check bypass'
+ * Mitigated according to the /sys interface: YES (Mitigation: usercopy/swapgs barriers and __user pointer saniation)
+ * Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
+ * Kernel has the Red Hat/Ubuntu patch: NO
+ * Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
+ * Kernel has the Red Hat/Ubuntu patch: NO
+ * Kernel has mask_nospec64 (arm64): NO
+ > STATUS: NOT VULNERABLE (Mitigation: usercopy/swapgs barriers and __user pointer sanitization)
+
+ CVE-2017-5715 aka 'Spectre Variant 2, branch target injection'
+ * Mitigated according to the /sys interface: YES (Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling)
+ * Mitigation 1
+ * Kernel is compiled with IBRS support: YES
+ * IBRS enabled and active: YES (Enhanced flavor, performance impact will be greatly reduced)
+ * Kernel is compiled with IBPB support: YES
+ * IBPB enabled and active: YES
+ * Mitigation 2
+ * Kernel has branch predictor hardening (arm): NO
+ * Kernel compiled with retpoline option: YES
+ * Kernel supports RSB filling: YES
+ > STATUS: NOT VULNERABLE (Enhanced IBRS + IBPB are mitigating the vulnerability)
+
+ CVE-2017-5754 aka 'Variant 3, Meltdown, rogue data cache load'
+ * Mitigated according to the /sys interface: YES (Not affected)
+ * Kernel supports Page Table Isolation (PTI): YES
+ * PTI enabled and active: NO
+ * Reduced performance impact of PTI: YES (CPU supports INVPCID, performance impact of PTI will be greatly reduced)
+ * Running as a Xen PV DomU: NO
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ CVE-2018-3640 aka 'Variant 3a, rogue system register read'
+ * CPU microcode mitigates the vulnerability: YES
+ > STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
+
+ CVE-2018-3639 aka 'Variant 4, speculative store bypass'
+ * Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
+ * Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
+ * SSB mitigation is enabled and active: YES (per-thread through prctl)
+ * SSB mitigation currently active for selected processes: YES ((deleted) systemd-journald systemd-logind systemd-networkd systemd-resolved systemd-timesyncd systemd-udevd)
+ > STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
+
+ CVE-2018-3615 aka 'Foreshadow (SGX), L1 terminal fault'
+ * CPU microcode mitigates the vulnerability: N/A
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ CVE-2018-3620 aka 'Foreshadow-NG (OS), L1 terminal fault'
+ * Mitigated according to the /sys interface: YES (Not affected)
+ * Kernel supports PTE inversion: YES (found in kernel image)
+ * PTE inversion enabled and active: NO
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ CVE-2018-3646 aka 'Foreshadow-NG (VMM), L1 terminal fault'
+ * Information from the /sys interface: Not affected
+ * This system is a host running a hypervisor: NO
+ * Mitigation 1 (KVM)
+ * EPT is disabled: NO
+ * Mitigation 2
+ * L1D flush is supported by kernel: YES (found flush_l1d in /proc/cpuinfo)
+ * L1D flush enabled: NO
+ * Hardware-backed L1D flush supported: YES (performance impact of the mitigation will be greatly reduced)
+
+ * Hyper-Threading (SMT) is enabled: YES
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ CVE-2018-12126 aka 'Fallout, microarchitectural store buffer data sampling (MSBDS)'
+ * Mitigated according to the /sys interface: YES (Not affected)
+ * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
+ * Kernel mitigation is enabled and active: NO
+ * SMT is either mitigated or disabled: NO
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ CVE-2018-12130 aka 'ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)'
+ * Mitigated according to the /sys interface: YES (Not affected)
+ * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
+ * Kernel mitigation is enabled and active: NO
+ * SMT is either mitigated or disabled: NO
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ CVE-2018-12127 aka 'RIDL, microarchitectural load port data sampling (MLPDS)'
+ * Mitigated according to the /sys interface: YES (Not affected)
+ * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
+ * Kernel mitigation is enabled and active: NO
+ * SMT is either mitigated or disabled: NO
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ CVE-2019-11091 aka 'RIDL, microarchitectural data sampling uncacheable memory (MDSUM)'
+ * Mitigated according to the /sys interface: YES (Not affected)
+ * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
+ * Kernel mitigation is enabled and active: NO
+ * SMT is either mitigated or disabled: NO
+ > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)
+
+ > SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:OK CVE-2018-3615:OK CVE-2018-3620:OK CVE-2018-3646:OK CVE-2018-12126:OK CVE-2018-12130:OK CVE-2018-12127:OK CVE-2019-11091:OK
+