ポイント
①遅延(RTT)が小さければTCPの1セッションで帯域をフル近くまで使える。
②遅延(RTT)が大きいとTCPの1セッションあたりでは帯域をフル近くまで使えない。
③遅延(RTT)が大きいとTCPの1セッションあたりでは帯域をフル近くまで使えないが、複数セッションを使えば帯域をフル近くまで使える。
ポイントの検証結果
検証環境
直結のパスでiperf3を動かし測定
RTTはtcコマンドで設定
RTT、パケロス、帯域を設定するツールを作ったので、それで設定した。※最後部にコード掲載。
サマリ
| ポイント | RTT(ms) | 並列実行数 | 最大スループット(Mb/s) |
|---|---|---|---|
| ① | 0.183 | 1 | 944 |
| ② | 100 | 1 | 247 |
| ③ | 100 | 4 | 931 |
結果ログ
①の確認
root@sv1:~# echo "#SV1"; tc qdisc show; echo "#SV2"; ssh sv2-r "tc qdisc show" #SV1 qdisc noqueue 0: dev lo root refcnt 2 qdisc tbf 1: dev enp0s25 root refcnt 2 rate 1Gbit burst 500000b lat 36.0ms qdisc fq_codel 0: dev enp2s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn qdisc fq_codel 0: dev enp3s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn #SV2 qdisc noqueue 0: dev lo root refcnt 2 qdisc tbf 1: dev enp0s25 root refcnt 2 rate 1Gbit burst 500000b lat 36.0ms qdisc fq_codel 0: dev enp2s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn qdisc fq_codel 0: dev enp3s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn root@sv1:~# ping sv2 -c 1 > /dev/null && ping sv2 -c 5 | tail -3 --- sv2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4074ms rtt min/avg/max/mdev = 0.145/0.183/0.223/0.029 ms root@sv1:~# iperf3 -c sv2 -t 30 Connecting to host sv2, port 5201 [ 4] local 192.168.0.250 port 49056 connected to 192.168.0.251 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 113 MBytes 944 Mbits/sec 0 366 KBytes [ 4] 1.00-2.00 sec 111 MBytes 935 Mbits/sec 0 366 KBytes [ 4] 2.00-3.00 sec 111 MBytes 935 Mbits/sec 0 366 KBytes [ 4] 3.00-4.00 sec 111 MBytes 934 Mbits/sec 0 366 KBytes [ 4] 4.00-5.00 sec 111 MBytes 935 Mbits/sec 0 366 KBytes [ 4] 5.00-6.00 sec 111 MBytes 934 Mbits/sec 0 366 KBytes [ 4] 6.00-7.00 sec 111 MBytes 935 Mbits/sec 0 366 KBytes [ 4] 7.00-8.00 sec 111 MBytes 934 Mbits/sec 0 366 KBytes [ 4] 8.00-9.00 sec 111 MBytes 934 Mbits/sec 0 366 KBytes [ 4] 9.00-10.00 sec 111 MBytes 935 Mbits/sec 0 366 KBytes [ 4] 10.00-11.00 sec 111 MBytes 935 Mbits/sec 0 366 KBytes [ 4] 11.00-12.00 sec 112 MBytes 943 Mbits/sec 0 553 KBytes [ 4] 12.00-13.00 sec 111 MBytes 934 Mbits/sec 0 553 KBytes [ 4] 13.00-14.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 14.00-15.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 15.00-16.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 16.00-17.00 sec 111 MBytes 934 Mbits/sec 0 553 KBytes [ 4] 17.00-18.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 18.00-19.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 19.00-20.00 sec 111 MBytes 934 Mbits/sec 0 553 KBytes [ 4] 20.00-21.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 21.00-22.00 sec 111 MBytes 934 Mbits/sec 0 553 KBytes [ 4] 22.00-23.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 23.00-24.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 24.00-25.00 sec 111 MBytes 934 Mbits/sec 0 553 KBytes [ 4] 25.00-26.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 26.00-27.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 27.00-28.00 sec 111 MBytes 934 Mbits/sec 0 553 KBytes [ 4] 28.00-29.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes [ 4] 29.00-30.00 sec 111 MBytes 935 Mbits/sec 0 553 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-30.00 sec 3.27 GBytes 935 Mbits/sec 0 sender [ 4] 0.00-30.00 sec 3.26 GBytes 934 Mbits/sec receiver iperf Done.
②の確認
root@sv1:~# echo "#SV1"; tc qdisc show; echo "#SV2"; ssh sv2-r "tc qdisc show" #SV1 qdisc noqueue 0: dev lo root refcnt 2 qdisc tbf 1: dev enp0s25 root refcnt 2 rate 1Gbit burst 500000b lat 36.0ms qdisc netem 10: dev enp0s25 parent 1: limit 1000 delay 50.0ms qdisc fq_codel 0: dev enp2s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn qdisc fq_codel 0: dev enp3s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn #SV2 qdisc noqueue 0: dev lo root refcnt 2 qdisc tbf 1: dev enp0s25 root refcnt 2 rate 1Gbit burst 500000b lat 36.0ms qdisc netem 10: dev enp0s25 parent 1: limit 1000 delay 50.0ms qdisc fq_codel 0: dev enp2s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn qdisc fq_codel 0: dev enp3s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn root@sv1:~# ping sv2 -c 1 > /dev/null && ping sv2 -c 5 | tail -3 --- sv2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4005ms rtt min/avg/max/mdev = 100.204/100.234/100.301/0.034 ms root@sv1:~# iperf3 -c sv2 -t 30 Connecting to host sv2, port 5201 [ 4] local 192.168.0.250 port 49248 connected to 192.168.0.251 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 239 KBytes 1.96 Mbits/sec 0 19.8 KBytes [ 4] 1.00-2.00 sec 369 KBytes 3.02 Mbits/sec 0 32.5 KBytes [ 4] 2.00-3.00 sec 509 KBytes 4.17 Mbits/sec 0 52.3 KBytes [ 4] 3.00-4.00 sec 827 KBytes 6.78 Mbits/sec 0 86.3 KBytes [ 4] 4.00-5.00 sec 1.37 MBytes 11.5 Mbits/sec 0 140 KBytes [ 4] 5.00-6.00 sec 2.17 MBytes 18.2 Mbits/sec 0 228 KBytes [ 4] 6.00-7.00 sec 3.54 MBytes 29.7 Mbits/sec 0 369 KBytes [ 4] 7.00-8.00 sec 5.84 MBytes 49.0 Mbits/sec 0 600 KBytes [ 4] 8.00-9.00 sec 7.83 MBytes 65.7 Mbits/sec 0 976 KBytes [ 4] 9.00-10.00 sec 11.2 MBytes 94.3 Mbits/sec 0 1.52 MBytes [ 4] 10.00-11.00 sec 18.3 MBytes 154 Mbits/sec 0 2.43 MBytes [ 4] 11.00-12.00 sec 27.6 MBytes 232 Mbits/sec 0 3.02 MBytes [ 4] 12.00-13.00 sec 29.4 MBytes 246 Mbits/sec 0 3.02 MBytes [ 4] 13.00-14.00 sec 28.4 MBytes 238 Mbits/sec 0 3.02 MBytes [ 4] 14.00-15.00 sec 28.9 MBytes 242 Mbits/sec 0 3.02 MBytes [ 4] 15.00-16.00 sec 29.4 MBytes 247 Mbits/sec 0 3.02 MBytes [ 4] 16.00-17.00 sec 28.6 MBytes 239 Mbits/sec 0 3.02 MBytes [ 4] 17.00-18.00 sec 28.2 MBytes 237 Mbits/sec 0 3.02 MBytes [ 4] 18.00-19.00 sec 28.1 MBytes 236 Mbits/sec 0 3.02 MBytes [ 4] 19.00-20.00 sec 29.4 MBytes 247 Mbits/sec 0 3.02 MBytes [ 4] 20.00-21.00 sec 29.4 MBytes 247 Mbits/sec 0 3.02 MBytes [ 4] 21.00-22.00 sec 29.4 MBytes 246 Mbits/sec 0 3.02 MBytes [ 4] 22.00-23.00 sec 28.7 MBytes 241 Mbits/sec 0 3.02 MBytes [ 4] 23.00-24.00 sec 28.1 MBytes 236 Mbits/sec 0 3.02 MBytes [ 4] 24.00-25.00 sec 28.2 MBytes 236 Mbits/sec 0 3.02 MBytes [ 4] 25.00-26.00 sec 29.4 MBytes 246 Mbits/sec 0 3.02 MBytes [ 4] 26.00-27.00 sec 29.4 MBytes 246 Mbits/sec 0 3.02 MBytes [ 4] 27.00-28.00 sec 29.4 MBytes 247 Mbits/sec 0 3.02 MBytes [ 4] 28.00-29.00 sec 28.9 MBytes 243 Mbits/sec 0 3.02 MBytes [ 4] 29.00-30.00 sec 27.6 MBytes 232 Mbits/sec 0 3.02 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-30.00 sec 599 MBytes 167 Mbits/sec 0 sender [ 4] 0.00-30.00 sec 598 MBytes 167 Mbits/sec receiver iperf Done. root@sv1:~#
③の確認
root@sv1:~# echo "#SV1"; tc qdisc show; echo "#SV2"; ssh sv2-r "tc qdisc show" #SV1 qdisc noqueue 0: dev lo root refcnt 2 qdisc tbf 1: dev enp0s25 root refcnt 2 rate 1Gbit burst 500000b lat 36.0ms qdisc netem 10: dev enp0s25 parent 1: limit 1000 delay 50.0ms qdisc fq_codel 0: dev enp2s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn qdisc fq_codel 0: dev enp3s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn #SV2 qdisc noqueue 0: dev lo root refcnt 2 qdisc tbf 1: dev enp0s25 root refcnt 2 rate 1Gbit burst 500000b lat 36.0ms qdisc netem 10: dev enp0s25 parent 1: limit 1000 delay 50.0ms qdisc fq_codel 0: dev enp2s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn qdisc fq_codel 0: dev enp3s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn root@sv1:~# ping sv2 -c 1 > /dev/null && ping sv2 -c 5 | tail -3 --- sv2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4005ms rtt min/avg/max/mdev = 100.212/100.237/100.252/0.400 ms root@sv1:~# iperf3 -c sv2 -t 30 -P 4 Connecting to host sv2, port 5201 [ 4] local 192.168.0.250 port 49190 connected to 192.168.0.251 port 5201 [ 6] local 192.168.0.250 port 49192 connected to 192.168.0.251 port 5201 [ 8] local 192.168.0.250 port 49194 connected to 192.168.0.251 port 5201 [ 10] local 192.168.0.250 port 49196 connected to 192.168.0.251 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 291 KBytes 2.39 Mbits/sec 0 21.2 KBytes [ 6] 0.00-1.00 sec 267 KBytes 2.19 Mbits/sec 0 19.8 KBytes [ 8] 0.00-1.00 sec 284 KBytes 2.33 Mbits/sec 0 21.2 KBytes [ 10] 0.00-1.00 sec 257 KBytes 2.11 Mbits/sec 0 19.8 KBytes [SUM] 0.00-1.00 sec 1.07 MBytes 9.01 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 1.00-2.00 sec 321 KBytes 2.63 Mbits/sec 0 35.4 KBytes [ 6] 1.00-2.00 sec 325 KBytes 2.66 Mbits/sec 0 32.5 KBytes [ 8] 1.00-2.00 sec 320 KBytes 2.62 Mbits/sec 0 35.4 KBytes [ 10] 1.00-2.00 sec 321 KBytes 2.63 Mbits/sec 0 32.5 KBytes [SUM] 1.00-2.00 sec 1.26 MBytes 10.5 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 2.00-3.00 sec 573 KBytes 4.69 Mbits/sec 0 58.0 KBytes [ 6] 2.00-3.00 sec 509 KBytes 4.17 Mbits/sec 0 53.7 KBytes [ 8] 2.00-3.00 sec 573 KBytes 4.69 Mbits/sec 0 58.0 KBytes [ 10] 2.00-3.00 sec 509 KBytes 4.17 Mbits/sec 0 52.3 KBytes [SUM] 2.00-3.00 sec 2.11 MBytes 17.7 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 3.00-4.00 sec 954 KBytes 7.82 Mbits/sec 0 93.3 KBytes [ 6] 3.00-4.00 sec 827 KBytes 6.78 Mbits/sec 0 86.3 KBytes [ 8] 3.00-4.00 sec 954 KBytes 7.82 Mbits/sec 0 93.3 KBytes [ 10] 3.00-4.00 sec 827 KBytes 6.78 Mbits/sec 0 86.3 KBytes [SUM] 3.00-4.00 sec 3.48 MBytes 29.2 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 4.00-5.00 sec 1.43 MBytes 12.0 Mbits/sec 0 153 KBytes [ 6] 4.00-5.00 sec 1.37 MBytes 11.5 Mbits/sec 0 139 KBytes [ 8] 4.00-5.00 sec 1.49 MBytes 12.5 Mbits/sec 0 153 KBytes [ 10] 4.00-5.00 sec 1.37 MBytes 11.5 Mbits/sec 0 140 KBytes [SUM] 4.00-5.00 sec 5.65 MBytes 47.4 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 5.00-6.00 sec 2.36 MBytes 19.8 Mbits/sec 0 249 KBytes [ 6] 5.00-6.00 sec 2.11 MBytes 17.7 Mbits/sec 0 226 KBytes [ 8] 5.00-6.00 sec 2.36 MBytes 19.8 Mbits/sec 0 249 KBytes [ 10] 5.00-6.00 sec 2.17 MBytes 18.2 Mbits/sec 0 228 KBytes [SUM] 5.00-6.00 sec 9.01 MBytes 75.6 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 6.00-7.00 sec 3.98 MBytes 33.4 Mbits/sec 0 404 KBytes [ 6] 6.00-7.00 sec 3.54 MBytes 29.7 Mbits/sec 0 368 KBytes [ 8] 6.00-7.00 sec 3.98 MBytes 33.4 Mbits/sec 0 406 KBytes [ 10] 6.00-7.00 sec 3.60 MBytes 30.2 Mbits/sec 0 372 KBytes [SUM] 6.00-7.00 sec 15.1 MBytes 127 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 7.00-8.00 sec 6.46 MBytes 54.2 Mbits/sec 0 659 KBytes [ 6] 7.00-8.00 sec 5.84 MBytes 49.0 Mbits/sec 0 597 KBytes [ 8] 7.00-8.00 sec 6.40 MBytes 53.7 Mbits/sec 0 659 KBytes [ 10] 7.00-8.00 sec 5.90 MBytes 49.5 Mbits/sec 0 604 KBytes [SUM] 7.00-8.00 sec 24.6 MBytes 206 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 8.00-9.00 sec 8.20 MBytes 68.8 Mbits/sec 0 1.05 MBytes [ 6] 8.00-9.00 sec 7.77 MBytes 65.2 Mbits/sec 0 970 KBytes [ 8] 8.00-9.00 sec 8.20 MBytes 68.8 Mbits/sec 0 1.05 MBytes [ 10] 8.00-9.00 sec 7.83 MBytes 65.7 Mbits/sec 0 981 KBytes [SUM] 8.00-9.00 sec 32.0 MBytes 268 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 9.00-10.00 sec 13.0 MBytes 110 Mbits/sec 0 1.70 MBytes [ 6] 9.00-10.00 sec 11.5 MBytes 96.5 Mbits/sec 0 1.52 MBytes [ 8] 9.00-10.00 sec 13.0 MBytes 109 Mbits/sec 0 1.70 MBytes [ 10] 9.00-10.00 sec 11.7 MBytes 98.0 Mbits/sec 0 1.54 MBytes [SUM] 9.00-10.00 sec 49.2 MBytes 413 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 10.00-11.00 sec 21.0 MBytes 177 Mbits/sec 0 2.74 MBytes [ 6] 10.00-11.00 sec 18.4 MBytes 154 Mbits/sec 0 2.44 MBytes [ 8] 10.00-11.00 sec 20.4 MBytes 171 Mbits/sec 0 2.71 MBytes [ 10] 10.00-11.00 sec 17.6 MBytes 148 Mbits/sec 0 2.42 MBytes [SUM] 10.00-11.00 sec 77.4 MBytes 650 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 11.00-12.00 sec 27.1 MBytes 227 Mbits/sec 0 3.15 MBytes [ 6] 11.00-12.00 sec 26.4 MBytes 221 Mbits/sec 0 3.15 MBytes [ 8] 11.00-12.00 sec 27.5 MBytes 230 Mbits/sec 0 3.15 MBytes [ 10] 11.00-12.00 sec 27.3 MBytes 229 Mbits/sec 0 3.01 MBytes [SUM] 11.00-12.00 sec 108 MBytes 908 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 12.00-13.00 sec 27.7 MBytes 233 Mbits/sec 0 3.15 MBytes [ 6] 12.00-13.00 sec 27.8 MBytes 233 Mbits/sec 0 3.15 MBytes [ 8] 12.00-13.00 sec 27.1 MBytes 227 Mbits/sec 0 3.15 MBytes [ 10] 12.00-13.00 sec 27.7 MBytes 232 Mbits/sec 0 3.01 MBytes [SUM] 12.00-13.00 sec 110 MBytes 925 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 13.00-14.00 sec 27.8 MBytes 233 Mbits/sec 0 3.15 MBytes [ 6] 13.00-14.00 sec 27.8 MBytes 233 Mbits/sec 0 3.15 MBytes [ 8] 13.00-14.00 sec 27.7 MBytes 233 Mbits/sec 0 3.15 MBytes [ 10] 13.00-14.00 sec 27.2 MBytes 228 Mbits/sec 0 3.01 MBytes [SUM] 13.00-14.00 sec 111 MBytes 927 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 14.00-15.00 sec 27.8 MBytes 233 Mbits/sec 0 3.15 MBytes [ 6] 14.00-15.00 sec 27.5 MBytes 230 Mbits/sec 0 3.15 MBytes [ 8] 14.00-15.00 sec 27.8 MBytes 233 Mbits/sec 0 3.15 MBytes [ 10] 14.00-15.00 sec 27.7 MBytes 232 Mbits/sec 0 3.01 MBytes [SUM] 14.00-15.00 sec 111 MBytes 929 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 15.00-16.00 sec 27.4 MBytes 230 Mbits/sec 0 3.15 MBytes [ 6] 15.00-16.00 sec 27.8 MBytes 233 Mbits/sec 0 3.15 MBytes [ 8] 15.00-16.00 sec 28.1 MBytes 235 Mbits/sec 0 3.15 MBytes [ 10] 15.00-16.00 sec 27.7 MBytes 232 Mbits/sec 0 3.01 MBytes [SUM] 15.00-16.00 sec 111 MBytes 931 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 16.00-17.00 sec 27.8 MBytes 233 Mbits/sec 0 3.15 MBytes [ 6] 16.00-17.00 sec 27.7 MBytes 233 Mbits/sec 0 3.15 MBytes [ 8] 16.00-17.00 sec 27.3 MBytes 230 Mbits/sec 0 3.15 MBytes [ 10] 16.00-17.00 sec 27.8 MBytes 233 Mbits/sec 0 3.01 MBytes [SUM] 16.00-17.00 sec 111 MBytes 928 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 17.00-18.00 sec 27.2 MBytes 228 Mbits/sec 0 3.15 MBytes [ 6] 17.00-18.00 sec 27.0 MBytes 227 Mbits/sec 0 3.15 MBytes [ 8] 17.00-18.00 sec 27.2 MBytes 228 Mbits/sec 0 3.15 MBytes [ 10] 17.00-18.00 sec 27.0 MBytes 227 Mbits/sec 0 3.01 MBytes [SUM] 17.00-18.00 sec 108 MBytes 910 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 18.00-19.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 18.00-19.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 8] 18.00-19.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 18.00-19.00 sec 26.5 MBytes 222 Mbits/sec 0 3.01 MBytes [SUM] 18.00-19.00 sec 106 MBytes 891 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 19.00-20.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 19.00-20.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 8] 19.00-20.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 19.00-20.00 sec 26.5 MBytes 222 Mbits/sec 0 3.01 MBytes [SUM] 19.00-20.00 sec 106 MBytes 891 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 20.00-21.00 sec 26.4 MBytes 221 Mbits/sec 0 3.15 MBytes [ 6] 20.00-21.00 sec 26.4 MBytes 221 Mbits/sec 0 3.15 MBytes [ 8] 20.00-21.00 sec 26.4 MBytes 221 Mbits/sec 0 3.15 MBytes [ 10] 20.00-21.00 sec 26.3 MBytes 220 Mbits/sec 0 3.01 MBytes [SUM] 20.00-21.00 sec 105 MBytes 885 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 21.00-22.00 sec 26.7 MBytes 224 Mbits/sec 0 3.15 MBytes [ 6] 21.00-22.00 sec 26.7 MBytes 224 Mbits/sec 0 3.15 MBytes [ 8] 21.00-22.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 21.00-22.00 sec 26.4 MBytes 221 Mbits/sec 0 3.01 MBytes [SUM] 21.00-22.00 sec 106 MBytes 892 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 22.00-23.00 sec 26.5 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 22.00-23.00 sec 26.4 MBytes 221 Mbits/sec 0 3.15 MBytes [ 8] 22.00-23.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 22.00-23.00 sec 26.1 MBytes 219 Mbits/sec 0 3.01 MBytes [SUM] 22.00-23.00 sec 106 MBytes 885 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 23.00-24.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 23.00-24.00 sec 26.5 MBytes 223 Mbits/sec 0 3.15 MBytes [ 8] 23.00-24.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 23.00-24.00 sec 26.3 MBytes 221 Mbits/sec 0 3.01 MBytes [SUM] 23.00-24.00 sec 106 MBytes 888 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 24.00-25.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 24.00-25.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 8] 24.00-25.00 sec 26.4 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 24.00-25.00 sec 26.3 MBytes 220 Mbits/sec 0 3.01 MBytes [SUM] 24.00-25.00 sec 106 MBytes 888 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 25.00-26.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 25.00-26.00 sec 26.5 MBytes 223 Mbits/sec 0 3.15 MBytes [ 8] 25.00-26.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 25.00-26.00 sec 26.3 MBytes 220 Mbits/sec 0 3.01 MBytes [SUM] 25.00-26.00 sec 106 MBytes 888 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 26.00-27.00 sec 26.7 MBytes 224 Mbits/sec 0 3.15 MBytes [ 6] 26.00-27.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 8] 26.00-27.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 26.00-27.00 sec 26.3 MBytes 221 Mbits/sec 0 3.01 MBytes [SUM] 26.00-27.00 sec 106 MBytes 890 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 27.00-28.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 27.00-28.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 8] 27.00-28.00 sec 26.4 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 27.00-28.00 sec 26.4 MBytes 222 Mbits/sec 0 3.01 MBytes [SUM] 27.00-28.00 sec 106 MBytes 889 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 28.00-29.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 28.00-29.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 8] 28.00-29.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 28.00-29.00 sec 26.4 MBytes 222 Mbits/sec 0 3.01 MBytes [SUM] 28.00-29.00 sec 106 MBytes 890 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 29.00-30.00 sec 26.6 MBytes 223 Mbits/sec 0 3.15 MBytes [ 6] 29.00-30.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 8] 29.00-30.00 sec 26.5 MBytes 222 Mbits/sec 0 3.15 MBytes [ 10] 29.00-30.00 sec 26.3 MBytes 221 Mbits/sec 0 3.01 MBytes [SUM] 29.00-30.00 sec 106 MBytes 888 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-30.00 sec 571 MBytes 160 Mbits/sec 0 sender [ 4] 0.00-30.00 sec 570 MBytes 159 Mbits/sec receiver [ 6] 0.00-30.00 sec 563 MBytes 157 Mbits/sec 0 sender [ 6] 0.00-30.00 sec 562 MBytes 157 Mbits/sec receiver [ 8] 0.00-30.00 sec 568 MBytes 159 Mbits/sec 0 sender [ 8] 0.00-30.00 sec 567 MBytes 159 Mbits/sec receiver [ 10] 0.00-30.00 sec 561 MBytes 157 Mbits/sec 0 sender [ 10] 0.00-30.00 sec 560 MBytes 156 Mbits/sec receiver [SUM] 0.00-30.00 sec 2.21 GBytes 633 Mbits/sec 0 sender [SUM] 0.00-30.00 sec 2.21 GBytes 632 Mbits/sec receiver iperf Done. root@sv1:~#
おまけ。設定ツール
使い方
root@sv1:~# python2 ./set.py -h usage: set.py [-h] [--rtt RTT] [--loss LOSS] --tp TP nw analyzer optional arguments: -h, --help show this help message and exit --rtt RTT rtt(ms) --loss LOSS packet loss(percent) --tp TP through put(Mb/s) root@sv1:~#
コード
# -*- coding: utf-8 -*- import argparse import sys from datetime import datetime as dt import subprocess import time import json import syslog import copy import os #os.environ['LANG'] = 'C' DST_HOST_CNT = "sv2-r" DST_HOST = "sv2" DST_NIC = "enp0s25" SRC_NIC = "enp0s25" TMP_LOG1 = "/tmp/ana_http_tmp_iperf.log" TMP_LOG2 = "/tmp/ana_http_tmp_wget.log" #L_ENV = { 'LANG':'C' } PATH="/test/" parser = argparse.ArgumentParser(description='nw analyzer') parser.add_argument('--rtt', help='rtt(ms)', type=float) parser.add_argument('--loss', help='packet loss(percent)', type=float) parser.add_argument('--tp', help='through put(Mb/s)', type=float, required=True) args = parser.parse_args() ARG = copy.deepcopy(sys.argv) LOG_LABEL = dt.now().strftime('%H:%M:%S') + " " + " ".join(ARG[1:]) def get_hz(): src_kv, dst_kv = exec_cmd_both("uname -r", True) src_hz = exec_cmd('grep CONFIG_HZ= /boot/config-' + src_kv, True) src_hz = src_hz.split("=")[1] if not src_hz.isdigit(): sub_error("src_hz invalid data : " + src_hz) dst_hz = exec_cmd('ssh ' + DST_HOST_CNT + ' grep CONFIG_HZ= /boot/config-' + dst_kv, True) dst_hz = dst_hz.split("=")[1] if not dst_hz.isdigit(): sub_error("dst_hz invalid data : " + dst_hz) return src_hz, dst_hz def chk_env(): # NIC確認 exec_cmd("ip link show " + SRC_NIC, True) exec_cmd("ssh " + DST_HOST_CNT + " ip link show " + DST_NIC, True) def setup_tp(src_hz, dst_hz): def set_param(hz): rate = args.tp burst = rate * 1000 * 1000 / 8 / int(hz) limit = burst * 10 return [ rate, burst, limit ] src_rate, src_burst, src_limit = set_param(src_hz) dst_rate, dst_burst, dst_limit = set_param(dst_hz) cmd = "tc qdisc delete dev " + SRC_NIC + " root" exec_cmd(cmd, False) cmd = "tc qdisc delete dev " + DST_NIC + " root" exec_rmt_cmd(cmd, False) cmd = "tc qdisc add dev " + SRC_NIC + " root handle 1:0 tbf limit " + str(src_limit) + " burst " + str(src_burst) + " rate " + str(src_rate) + "mbit" exec_cmd(cmd, True) cmd = "tc qdisc add dev " + DST_NIC + " root handle 1:0 tbf limit " + str(dst_limit) + " burst " + str(dst_burst) + " rate " + str(dst_rate) + "mbit" exec_rmt_cmd(cmd, True) def test_tp(): exec_rmt_cmd("pkill -x iperf3", False) # exec_rmt_cmd("iperf3 -d -D -s", True) exec_rmt_cmd("iperf3 -J -D -s", True) time.sleep(1) exec_cmd("rm -f " + TMP_LOG1, True) # o = exec_cmd("iperf3 -d --get-server-output -c " + DST_HOST, True) exec_cmd("iperf3 -J --get-server-output -c " + DST_HOST + " --logfile " + TMP_LOG1, True) with open(TMP_LOG1) as f: j = json.load(f) exec_rmt_cmd("pkill -x iperf3", False) l = str() s = j['end']['streams'][0]['sender'] for k,v in s.items(): l = l + " s_" + str(k) + "=" + str(v) r = j['server_output_json']['end']['streams'][0]['receiver'] for k,v in r.items(): l = l + " r_" + str(k) + "=" + str(v) op_log("Test: " + l[1:]) def setup_delay_and_loss(): def exec_tc_cmd(p): opt = list() if "rtt" in p: opt.append("delay") opt.append(str(p["rtt"] / 2) + "ms") if "loss" in p: opt.append("loss") opt.append(str(p["loss"] / 2) + "%") opt = " ".join(opt) if opt: cmd = "tc qdisc add dev " + SRC_NIC + " parent 1:0 handle 10: netem " + opt exec_cmd(cmd, True) cmd = "tc qdisc add dev " + DST_NIC + " parent 1:0 handle 10: netem " + opt exec_rmt_cmd(cmd, True) if args.rtt: if args.loss: exec_tc_cmd({"rtt":args.rtt, "loss":args.loss}) else: exec_tc_cmd({"rtt":args.rtt}) else: if args.loss: exec_tc_cmd({"loss":args.loss}) def get_setting(): src_res, dst_res = exec_cmd_both("tc qdisc show", True) for i in [src_res, dst_res]: for l in i.split("\n"): op_log("Setting: " + l.strip()) def setup_ol(): cmd = "ethtool -K " + SRC_NIC + " tso off gso off gro off rx off tx off" exec_cmd(cmd, True) cmd = "ethtool -K " + DST_NIC + " tso off gso off gro off rx off tx off" exec_rmt_cmd(cmd, True) def setup_env(): # 各種NICオフロード機能無効化 setup_ol() # CONFIG_HZ取得 src_hz, dst_hz = get_hz() # 帯域設定 setup_tp(src_hz, dst_hz) # 遅延 パケロス設定 setup_delay_and_loss() # 設定確認 get_setting() def sub_error(m): m = "Error: " + m print(m) op_log(m) exit() def exec_cmd_both(cmd, cmd_err): src_out = exec_cmd(cmd, cmd_err) dst_out = exec_cmd("ssh " + DST_HOST_CNT + " " + cmd, cmd_err) return src_out, dst_out def exec_rmt_cmd(cmd, cmd_err): cmd = "ssh " + DST_HOST_CNT + " " + cmd d = exec_cmd(cmd, cmd_err) return d def exec_cmd(cmd, cmd_err): op_log("Exec_cmd: " + cmd) try: d = subprocess.check_output(cmd.split(), stderr=subprocess.STDOUT) d = d.strip() return d except subprocess.CalledProcessError as e: rc = str(e.returncode).strip() op = e.output.strip() if cmd_err: sub_error("RC: " + rc + " CMD: " + cmd + " OUTPUT: " + op) exit() except OSError as e: sub_error(cmd + ": " + e.strerror) exit() def op_log(mes): syslog.syslog(LOG_LABEL + " " + mes) def main(): chk_env() setup_env() if __name__ == '__main__': main()
