Forums

Resolved
0 votes
Two 1gig ports Ethernet ought to equal 2gig right?

Problem is it only pings and networks if I pull one of the two cables out. As soon as it establishes a bonded connection again it fails, then works again when I unplug either cable.

Following these instructions https://documentation.clearos.com/content:en_us:kb_howtos_network_bonding
I am able to create a bonded LAN that the ClearOS WebGUI reports as 2000mbps.
The instructions actually specify MODE=5 which does seem to work. However MODE=4 is the true 802.3ad connection requiring a Switch with LAG ports enabled.
The TP-Link TL-SG108E actually has this feature and ClearOS knows if I'm plugged into those ports or non-bonded ports which causes it to complain on the screen connected to the server.

[root@cleare3 ~]# lspci -k | grep Eth -A 3
01:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)
Subsystem: Intel Corporation PRO/1000 PT Dual Port Server Adapter
Kernel driver in use: e1000e
Kernel modules: e1000e
01:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)
Subsystem: Intel Corporation PRO/1000 PT Dual Port Server Adapter
Kernel driver in use: e1000e
Kernel modules: e1000e
--
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
Subsystem: Micro-Star International Co., Ltd. [MSI] Device 7788
Kernel driver in use: r8169
Kernel modules: r8169



[root@cleare3 ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 00:15:17:77:d5:e0
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 1
Actor Key: 9
Partner Key: 1
Partner Mac Address: 00:00:00:00:00:00

Slave Interface: enp1s0f0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:15:17:77:d5:e0
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: churned
Actor Churned Count: 0
Partner Churned Count: 1
details actor lacp pdu:
system priority: 65535
system mac address: 00:15:17:77:d5:e0
port key: 9
port priority: 255
port number: 1
port state: 77
details partner lacp pdu:
system priority: 65535
system mac address: 00:00:00:00:00:00
oper key: 1
port priority: 255
port number: 1
port state: 1

Slave Interface: enp1s0f1
MII Status: down
Speed: Unknown
Duplex: Unknown
Link Failure Count: 4
Permanent HW addr: 00:15:17:77:d5:e1
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: monitoring
Partner Churn State: monitoring
Actor Churned Count: 2
Partner Churned Count: 2
details actor lacp pdu:
system priority: 65535
system mac address: 00:15:17:77:d5:e0
port key: 0
port priority: 255
port number: 2
port state: 69
details partner lacp pdu:
system priority: 65535
system mac address: 00:00:00:00:00:00
oper key: 1
port priority: 255
port number: 1
port state: 1


Note one cable unplugged so it works. You can see it's attempting IEEE 802.3ad Dynamic link aggregation.


Here are my config files;
[root@cleare3 ~]# cat /etc/clearos/network.conf
# Network mode
MODE="trustedstandalone"

# Network interface roles
EXTIF="enp3s0"
LANIF="bond0 enp3s0"
DMZIF=""
HOTIF=""

# Domain and Internet Hostname
DEFAULT_DOMAIN="sothcott.co.uk"
INTERNET_HOSTNAME="cleare3.sothcott.co.uk"

# Extra LANS
EXTRALANS=""

# ISP Maximum Speeds

############
[root@cleare3 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT="yes"
USERCTL="no"
BOOTPROTO="none"
IPADDR="192.168.32.32"
NETMASK="255.255.255.0"
BONDING_MASTER="yes"
BONDING_OPTS="mode=4 miimon=100"

############
[root@cleare3 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp1s0f0
DEVICE=enp1s0f0
TYPE="Bonded"
ONBOOT="yes"
USERCTL="no"
BOOTPROTO="none"
MASTER="bond0"
SLAVE="yes"

############
[root@cleare3 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp1s0f1
DEVICE=enp1s0f1
TYPE="Bonded"
ONBOOT="yes"
USERCTL="no"
BOOTPROTO="none"
MASTER="bond0"
SLAVE="yes"





It would be excellent to get this working. I had got this set up in Proxmod which is debian based. It was weird because I could talk to Proxmox over the bonded connection but the ClearOS VM running inside Proxmox on the bonded link had the same problem in needing one cable to be pulled out. This seems to be a ClearOS problem not a hardware problem.
Tuesday, June 30 2020, 05:43 PM
Share this post:
Responses (2)
  • Accepted Answer

    Tuesday, June 30 2020, 08:53 PM - #Permalink
    Resolved
    0 votes
    Thanks Nick,

    I know about the LANIF and EXTIF thing, that was me messing about trying to get it to work, it's not the problem.
    Yes your BONDING_OPTS line is the same as mode=4
    I don't think the quotes are required except when there are spaces. When you look at what ClearOS itself puts in there, they don't use quotes.

    I've not actually had any problems with the default realtec driver but now you remind me we normally change it. Maybe some sort of cross interference so it might help.

    My next test with this hardware will be Mint Linux because it seemed OK in Proxmox which is Debian based.

    This whole bonding thing looks easy enough but it's hard. To be able to build a 2gbps network using a couple of cheap cards and a cheap TP-Link router would be fantastic.

    Here is another cheap switch that can do 2gbps
    TP-Link TL-SG1016D 16-Port Gigabit Switch
    I might be better off with one of these as the link aggregation has more options.

    If I figure out how to make this work then I will post instructions here.
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, June 30 2020, 07:12 PM - #Permalink
    Resolved
    0 votes
    The instructions also have a "mode=4" example. I've also seen working:
    BONDING_OPTS="mode=802.3ad miimon=10 lacp_rate=1"
    But it is beyond my knowledge. I've only ever been able to play around with a bonded connection at one end, I think and that was a long time ago.

    You do have one configuration issue as enp3s0 is defined both as a LANIF and EXTIF.

    Also, with enp3s0, can I suggest you install the kmod-r8168 and kmod-r8169 drvers then reboot. The r8168 driver works much better than the r8169 with this NIC.
    The reply is currently minimized Show
Your Reply