My Community Dashboard

  • iperf3 performance WAN interface, outbound great / inbound terrible

    Greetings,

    First a brief summary of my ClearOS system:
    ClearOS 7.5.0, 3.10.0-862.11.6.v7.x86_64 #1 SMP Wed Aug 15 20:03:47 MDT 2018
    8 CPU(core) Xeon E3-1260L v5 2.90Ghz, 32GB RAM, 8 Ethernet Intel I210 ports, igb driver ver 5.4.0-k

    Network topology:
    WAN network and ISP gw----[ClearOS FW]---Internal network

    Should scream, right? Yeah not so much....

    I've been seeing bad inbound "download" performance so I started using iperf3 on different network segments to find the bad and debug it. Running iperf3 from the ClearOS box to machines on the internal network runs great, ~930-960Mbit in both directions.

    Running the same test on the ClearOS box to a machine on my WAN network the outbound performance is great, inbound is terribly broken.

    iperf3 send from ClearOS firewall to WAN system (my server on local WAN network)
    [ ID] Interval Transfer Bandwidth Retr Cwnd
    [ 4] 0.00-1.00 sec 114 MBytes 956 Mbits/sec 0 542 KBytes
    [ 4] 1.00-2.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
    [ 4] 2.00-3.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
    [ 4] 3.00-4.00 sec 113 MBytes 948 Mbits/sec 0 542 KBytes
    [ 4] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
    [ 4] 5.00-6.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
    [ 4] 6.00-7.00 sec 113 MBytes 948 Mbits/sec 0 542 KBytes
    [ 4] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes
    [ 4] 8.00-9.00 sec 112 MBytes 939 Mbits/sec 0 542 KBytes
    [ 4] 9.00-10.00 sec 112 MBytes 938 Mbits/sec 0 542 KBytes

    iperf3 send from WAN system to ClearOS firewall
    [ ID] Interval Transfer Bandwidth
    [ 4] 0.00-1.00 sec 10.4 MBytes 87.4 Mbits/sec
    [ 4] 1.00-2.00 sec 7.39 MBytes 62.0 Mbits/sec
    [ 4] 2.00-3.00 sec 90.4 MBytes 758 Mbits/sec
    [ 4] 3.00-4.00 sec 107 MBytes 901 Mbits/sec
    [ 4] 4.00-5.00 sec 1.93 MBytes 16.2 Mbits/sec
    [ 4] 5.00-6.00 sec 2.36 MBytes 19.8 Mbits/sec
    [ 4] 6.00-7.00 sec 3.91 MBytes 32.8 Mbits/sec
    [ 4] 7.00-8.00 sec 1.99 MBytes 16.7 Mbits/sec
    [ 4] 8.00-9.00 sec 2.17 MBytes 18.2 Mbits/sec
    [ 4] 9.00-10.00 sec 2.24 MBytes 18.8 Mbits/sec

    This is awful. I should see near same performance each direction. How do I resolve this? <100MBit? It's a gigabit interface connected to a gigabit switch connected to another local machine on the WAN network with a gigabit interface.

    ClearOS is doing something wrong/bad/dumb. I looked at Bandwidth and QoS Manager and disabled the engine but there is no change in the terrible inbound performance. The WAN interface is Automatic/Automatic for Rate-to-Quantum.

    Running top on the ClearOS box all I see creating a load is snort and that hovers in the 25-35% range during the iperf3 tests. No iowait, no swap, nothing obviously wrong.

    tuned-adm is set to balanced, I assume this is a choice made by ClearOS developers. Latency-performance seems a better fit but I'm not going to start turning knobs in the OS since it is supposed to be an appliance-like product.

    What can I do to get the inbound traffic on the WAN interface to perform at the same levels as outbound?

    Thanks,

    --Jeff