####1. Description of setup#1 ---> 2520-debug_user
userspace: qemu-kvm-0.12.1.2-2.355.el6.x86_64
host kernel: 2.6.32-358.el6.x86_64
guest kernel: 62.64.104.5700
####2. Description of setup#2 ---> 2522-debug_user
userspace: qemu-kvm-0.12.1.2-2.355.el6.x86_64
host kernel: 2.6.32-358.el6.x86_64
guest kernel: 62.64.104.5700
The tests are * 60* seconds sessions of 'Netperf'. 'throughput' was taken from netperf's report.
Other measurements were taken on the host.
How to read the results:
- The Throughput is measured in Mbit/sec.
- Every Avg line represents the average value based on *5* repetitions of the same test,
and the following SD line represents the Standard Deviation between the *5* repetitions.
- The Standard deviation is displayed as a percentage of the average.
- The significance of the differences between the two averages is calculated using unpaired T-test that
takes into account the SD of the averages.
- The paired t-test is computed for the averages of same category.
== Regression Testing: Win2012.64.netperf_win.netperf_exe.default_setting ==
Category:TCP_STREAM (RX) |
# | Tile | size | sessions | throughput | CPU | thr_per_CPU |
1 | Avg | 64 | 1 | 1039.29 | 19.89 | 52.25 |
| %SD | 2% | 19% | 21% |
2 | Avg | 1022.11 | 13.50 | 75.71 |
| %SD | 4% | 3% | 5% |
- | %Diff between Avg | -1.653% | -32.127% | +44.900% |
- | Significance | -0.546 | -0.994 | +0.997 |
1 | Avg | 64 | 2 | 2203.11 | 27.78 | 79.31 |
| %SD | 8% | 7% | 15% |
2 | Avg | 2065.49 | 22.69 | 91.03 |
| %SD | 2% | 9% | 11% |
- | %Diff between Avg | -6.247% | -18.323% | +14.777% |
- | Significance | -0.876 | -0.995 | +0.874 |
1 | Avg | 64 | 4 | 2907.98 | 25.83 | 112.58 |
| %SD | 3% | 6% | 6% |
2 | Avg | 3952.73 | 29.85 | 132.42 |
| %SD | 2% | 2% | 3% |
- | %Diff between Avg | +35.927% | +15.563% | +17.623% |
- | Significance | +1.000 | +0.999 | +1.000 |
1 | Avg | 64 | 8 | 2983.58 | 29.75 | 100.29 |
| %SD | 2% | 8% | 10% |
2 | Avg | 4079.26 | 29.84 | 136.70 |
| %SD | 0.89% | 3% | 3% |
- | %Diff between Avg | +36.724% | +0.303% | +36.305% |
- | Significance | +1.000 | +0.064 | +1.000 |
1 | Avg | 256 | 1 | 2448.56 | 26.57 | 92.16 |
| %SD | 8% | 12% | 6% |
2 | Avg | 3918.88 | 14.70 | 266.59 |
| %SD | 4% | 15% | 11% |
- | %Diff between Avg | +60.048% | -44.674% | +189.269% |
- | Significance | +1.000 | -1.000 | +1.000 |
1 | Avg | 256 | 2 | 2734.69 | 25.53 | 107.12 |
| %SD | 8% | 7% | 7% |
2 | Avg | 7305.93 | 27.45 | 266.15 |
| %SD | 3% | 4% | 7% |
- | %Diff between Avg | +167.158% | +7.521% | +148.460% |
- | Significance | +1.000 | +0.930 | +1.000 |
1 | Avg | 256 | 4 | 2952.89 | 27.11 | 108.92 |
| %SD | 2% | 9% | 8% |
2 | Avg | 9206.17 | 32.51 | 283.18 |
| %SD | 2% | 4% | 2% |
- | %Diff between Avg | +211.768% | +19.919% | +159.989% |
- | Significance | +1.000 | +0.997 | +1.000 |
1 | Avg | 256 | 8 | 2761.32 | 29.03 | 95.12 |
| %SD | 2% | 7% | 7% |
2 | Avg | 9446.91 | 31.25 | 302.30 |
| %SD | 0.43% | 3% | 3% |
- | %Diff between Avg | +242.116% | +7.647% | +217.809% |
- | Significance | +1.000 | +0.956 | +1.000 |
1 | Avg | 1024 | 1 | 2393.31 | 24.95 | 95.92 |
| %SD | 6% | 7% | 5% |
2 | Avg | 4121.83 | 15.48 | 266.27 |
| %SD | 6% | 18% | 17% |
- | %Diff between Avg | +72.223% | -37.956% | +177.596% |
- | Significance | +1.000 | -1.000 | +1.000 |
1 | Avg | 1024 | 2 | 2583.00 | 25.90 | 99.73 |
| %SD | 9% | 16% | 8% |
2 | Avg | 7125.25 | 26.59 | 267.97 |
| %SD | 10% | 5% | 7% |
- | %Diff between Avg | +175.852% | +2.664% | +168.695% |
- | Significance | +1.000 | +0.262 | +1.000 |
1 | Avg | 1024 | 4 | 2958.47 | 25.77 | 114.80 |
| %SD | 3% | 4% | 3% |
2 | Avg | 9065.03 | 31.21 | 290.45 |
| %SD | 3% | 4% | 2% |
- | %Diff between Avg | +206.409% | +21.110% | +153.005% |
- | Significance | +1.000 | +1.000 | +1.000 |
1 | Avg | 1024 | 8 | 2861.34 | 29.57 | 96.76 |
| %SD | 8% | 1% | 7% |
2 | Avg | 9298.97 | 32.09 | 289.78 |
| %SD | 3% | 6% | 8% |
- | %Diff between Avg | +224.987% | +8.522% | +199.483% |
- | Significance | +1.000 | +0.982 | +1.000 |
1 | Avg | 4096 | 1 | 2390.13 | 24.80 | 96.38 |
| %SD | 7% | 7% | 6% |
2 | Avg | 4107.17 | 15.66 | 262.27 |
| %SD | 6% | 24% | 30% |
- | %Diff between Avg | +71.839% | -36.855% | +172.121% |
- | Significance | +1.000 | -0.999 | +0.999 |
1 | Avg | 4096 | 2 | 2553.56 | 23.78 | 107.38 |
| %SD | 7% | 13% | 7% |
2 | Avg | 7387.60 | 28.01 | 263.75 |
| %SD | 6% | 4% | 5% |
- | %Diff between Avg | +189.306% | +17.788% | +145.623% |
- | Significance | +1.000 | +0.979 | +1.000 |
1 | Avg | 4096 | 4 | 3003.88 | 27.60 | 108.84 |
| %SD | 6% | 6% | 9% |
2 | Avg | 8988.08 | 31.43 | 285.97 |
| %SD | 1% | 3% | 2% |
- | %Diff between Avg | +199.216% | +13.877% | +162.743% |
- | Significance | +1.000 | +0.998 | +1.000 |
1 | Avg | 4096 | 8 | 2953.22 | 27.45 | 107.59 |
| %SD | 3% | 7% | 9% |
2 | Avg | 9438.89 | 31.39 | 300.70 |
| %SD | 0.48% | 0.40% | 0.55% |
- | %Diff between Avg | +219.614% | +14.353% | +179.487% |
- | Significance | +1.000 | +0.999 | +1.000 |
1 | Avg | 16384 | 1 | 2393.40 | 25.80 | 92.77 |
| %SD | 8% | 8% | 5% |
2 | Avg | 4217.23 | 16.10 | 261.94 |
| %SD | 0.52% | 15% | 17% |
- | %Diff between Avg | +76.202% | -37.597% | +182.354% |
- | Significance | +1.000 | -1.000 | +1.000 |
1 | Avg | 16384 | 2 | 2626.88 | 25.80 | 101.82 |
| %SD | 10% | 15% | 5% |
2 | Avg | 7394.05 | 28.83 | 256.47 |
| %SD | 8% | 8% | 3% |
- | %Diff between Avg | +181.477% | +11.744% | +151.886% |
- | Significance | +1.000 | +0.840 | +1.000 |
1 | Avg | 16384 | 4 | 3063.17 | 26.74 | 114.55 |
| %SD | 5% | 11% | 9% |
2 | Avg | 8976.40 | 31.67 | 283.44 |
| %SD | 5% | 4% | 4% |
- | %Diff between Avg | +193.043% | +18.437% | +147.438% |
- | Significance | +1.000 | +0.991 | +1.000 |
1 | Avg | 16384 | 8 | 2892.91 | 28.78 | 100.52 |
| %SD | 9% | 6% | 14% |
2 | Avg | 9438.44 | 31.10 | 303.49 |
| %SD | 0.89% | 5% | 5% |
- | %Diff between Avg | +226.261% | +8.061% | +201.920% |
- | Significance | +1.000 | +0.953 | +1.000 |
1 | Avg | 65535 | 1 | 2301.90 | 25.34 | 90.84 |
| %SD | 8% | 7% | 5% |
2 | Avg | 4177.28 | 15.74 | 265.39 |
| %SD | 5% | 9% | 8% |
- | %Diff between Avg | +81.471% | -37.885% | +192.151% |
- | Significance | +1.000 | -1.000 | +1.000 |
1 | Avg | 65535 | 2 | 2751.21 | 25.89 | 106.27 |
| %SD | 6% | 12% | 9% |
2 | Avg | 7073.56 | 27.25 | 259.58 |
| %SD | 2% | 5% | 5% |
- | %Diff between Avg | +157.107% | +5.253% | +144.265% |
- | Significance | +1.000 | +0.602 | +1.000 |
1 | Avg | 65535 | 4 | 2991.08 | 27.04 | 110.62 |
| %SD | 5% | 7% | 4% |
2 | Avg | 9248.90 | 32.73 | 282.58 |
| %SD | 2% | 2% | 1% |
- | %Diff between Avg | +209.216% | +21.043% | +155.451% |
- | Significance | +1.000 | +1.000 | +1.000 |
1 | Avg | 65535 | 8 | 2750.07 | 28.31 | 97.14 |
| %SD | 13% | 6% | 13% |
2 | Avg | 9398.68 | 31.51 | 298.28 |
| %SD | 2% | 4% | 5% |
- | %Diff between Avg | +241.761% | +11.303% | +207.062% |
- | Significance | +1.000 | +0.993 | +1.000 |
- | Total Significance | | | +1.000 | -0.176 | +1.000 |