-
iperf3 performance WAN interface, outbound great / inbound terrible
Greetings,
First a brief summary of my ClearOS system:
ClearOS 7.5.0, 3.10.0-862.11.6.v7.x86_64 #1 SMP Wed Aug 15 20:03:47 MDT 2018
8 CPU(core) Xeon E3-1260L v5 2.90Ghz, 32GB RAM, 8 Ethernet Intel I210 ports, igb driver ver 5.4.0-k
Network topology:
WAN network and ISP gw----[ClearOS FW]---Internal network
Should scream, right? Yeah not so much....
I've been seeing bad inbound "download" performance so I started using iperf3 on different network segments to find the bad and debug it. Running iperf3 from the ClearOS box to machines on the internal network runs great, ~930-960Mbit in both directions.
Running the same test on the ClearOS box to a machine on my WAN network the outbound performance is great, inbound is terribly broken.
iperf3 send from ClearOS firewall to WAN system (my server on local WAN network)
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 114 MBytes 956 Mbits/sec 0 542 KBytes
[ 4] 1.00-2.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
[ 4] 2.00-3.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
[ 4] 3.00-4.00 sec 113 MBytes 948 Mbits/sec 0 542 KBytes
[ 4] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
[ 4] 5.00-6.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
[ 4] 6.00-7.00 sec 113 MBytes 948 Mbits/sec 0 542 KBytes
[ 4] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
[ 4] 8.00-9.00 sec 112 MBytes 939 Mbits/sec 0 542 KBytes
[ 4] 9.00-10.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
iperf3 send from WAN system to ClearOS firewall
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 10.4 MBytes 87.4 Mbits/sec
[ 4] 1.00-2.00 sec 7.39 MBytes 62.0 Mbits/sec
[ 4] 2.00-3.00 sec 90.4 MBytes 758 Mbits/sec
[ 4] 3.00-4.00 sec 107 MBytes 901 Mbits/sec
[ 4] 4.00-5.00 sec 1.93 MBytes 16.2 Mbits/sec
[ 4] 5.00-6.00 sec 2.36 MBytes 19.8 Mbits/sec
[ 4] 6.00-7.00 sec 3.91 MBytes 32.8 Mbits/sec
[ 4] 7.00-8.00 sec 1.99 MBytes 16.7 Mbits/sec
[ 4] 8.00-9.00 sec 2.17 MBytes 18.2 Mbits/sec
[ 4] 9.00-10.00 sec 2.24 MBytes 18.8 Mbits/sec
This is awful. I should see near same performance each direction. How do I resolve this? <100MBit? It's a gigabit interface connected to a gigabit switch connected to another local machine on the WAN network with a gigabit interface.
ClearOS is doing something wrong/bad/dumb. I looked at Bandwidth and QoS Manager and disabled the engine but there is no change in the terrible inbound performance. The WAN interface is Automatic/Automatic for Rate-to-Quantum.
Running top on the ClearOS box all I see creating a load is snort and that hovers in the 25-35% range during the iperf3 tests. No iowait, no swap, nothing obviously wrong.
tuned-adm is set to balanced, I assume this is a choice made by ClearOS developers. Latency-performance seems a better fit but I'm not going to start turning knobs in the OS since it is supposed to be an appliance-like product.
What can I do to get the inbound traffic on the WAN interface to perform at the same levels as outbound?
Thanks,
--Jeff -