You can always try adding back in the quotes and see if it breaks anything. I don't fancy doing it on my system just in case ....

Nick Howitt wrote:

If that is an unedited old file, NETMASK is missing its opening quotes.

Thanks, I corrected a typo. Had a screendump to type

If that is an unedited old file, NETMASK is missing its opening quotes.

Just to let you all know that I solved my syswatch problems by going to: Network>IP Settings and editing eth0 (WAN) interface and just "Update" the settings.

I have heartbeat again and I noticed a slightly different /etc/sysconfig/network-scripts/ifcfg-eth0 file:

old one:

# Automatically generated

DEVICE="eth0"

ONBOOT="yes"

BOOTPROTO="static"

NM_CONTROLLED="no"

IPADDR="xxx.xxx.xxx.xxx"

NETMASK="255.255.254.0"

NETWORK="xxx.xxx.xxx.xxx"

BROADCAST="xxx.xxx.xxx.xxx"

GATEWAY="xxx.xxx.xxx.xxx"

and new one:

DEVICE=eth0

TYPE="Ethernet"

ONBOOT="yes"

USERCTL="no"

BOOTPROTO="static"

IPADDR="xxx.xxx.xxx.xxx"

NETMASK="255.255.254.0"

GATEWAY="xxx.xxx.xxx.xxx"

Could the (missing) quotes around eth0 have anything to do with it?

Ya, I really hope this gets fixed soon. I love ClearOS, and I don't have time to learn PFSense at the moment.

I am having same issue with new 6.5. I am currently running 5.2 on XenServer with no issues using 6 visualized physical interfaces supplied to 5.2 using 4 as wans (set as DHCP from cable/SOHO router) in gateway mode with no issues. When I duplicate the setup or try to utilize vlans within 6.5 I'm faced with zero stability. Once I add the second external interface (DHCP) the interfaces are no longer stable. I have tried following the forums regarding Multi WAN not working properly with more than 2 WAN connections? COS6.4t with no luck. 6.5 would be extremely valuable but without stability in the current state I'm completely out of luck.

This is the output on a VPS with only 1 WAN interface and no LAN:

Wed Mar 12 18:08:45 2014 info: system - syswatch started

Wed Mar 12 18:08:45 2014 info: config - IP referrer tool is not installed

Wed Mar 12 18:08:45 2014 info: config - debug level - 0

Wed Mar 12 18:08:45 2014 info: config - retries - 5

Wed Mar 12 18:08:45 2014 info: config - heartbeat - 10

Wed Mar 12 18:08:45 2014 info: config - interval - 60 seconds

Wed Mar 12 18:08:45 2014 info: config - offline interval - 10 seconds

Wed Mar 12 18:08:45 2014 info: config - referrer IP detection - disabled

Wed Mar 12 18:08:45 2014 info: config - ping server auto-detect - enabled

Wed Mar 12 18:08:45 2014 info: config - try pinging gateway - yes

Wed Mar 12 18:08:45 2014 info: config - number of external networks - 1

Wed Mar 12 18:08:45 2014 info: config - monitoring external network - eth0

Wed Mar 12 18:08:45 2014 info: info - loading network configuration

Wed Mar 12 18:08:45 2014 info: info - network configuration for eth0 - config:

Wed Mar 12 18:08:45 2014 info: info - network configuration for eth0 - onboot: disabled

Wed Mar 12 18:08:45 2014 info: info - network configuration for eth0 - type: static

Wed Mar 12 18:08:45 2014 info: info - network configuration for eth0 - wifi: disabled

Wed Mar 12 18:08:45 2014 info: eth0 - network - IP address - xxx.xxx.xxx.xxx

Wed Mar 12 18:08:45 2014 info: eth0 - network - gateway - unknown

Wed Mar 12 18:08:45 2014 info: eth0 - network - type - public IP range

Wed Mar 12 18:08:45 2014 warn: eth0 - no default gateway found

Wed Mar 12 18:08:45 2014 info: eth0 - ping check - no gateway found

Wed Mar 12 18:08:45 2014 info: eth0 - waiting for static IP reconnect

Wed Mar 12 18:08:45 2014 info: system - WAN network is not up

Wed Mar 12 18:08:55 2014 info: eth0 - ping check - no gateway found

Wed Mar 12 18:08:55 2014 info: eth0 - waiting for static IP reconnect

Wed Mar 12 18:08:55 2014 info: system - WAN network is not up

I discovered these problems today after a failure and the messages started after the update of syswatch.

I don't know if this has anything to do with the NIC driver. Server is working fine. LSPCI does not give output because the VPS runs on a XEN server at a hosting location.

Is there an update on the way?

Should have added that a reboot is necessary to effect any change to the kernel boot line, but you probably know that already...

I wonder if I could just install a dual gigabit PCIe card with a better chipset and be done with it. Though it really upsets me because I bought this board specifically for the onboard NICs, since my other choice was Realtek.

I haven't been able to get everyone off the server long enough to upgrade the driver, but I'll mess with that soon and report back.

82574L :-( It seems that even the kmod driver on it's own doesn't fix all this NIC's problems. You should also add "pcie_aspm=off" to the kernel line in /etv/grub.conf according to a current thread on the CentOS mailing list. Checking the web there is a lot of discussion relating to 82574L and pcie_aspm.

Of course, Maxxarcade, no idea whether this tip will finally fix your problems or you have something else going on, but this is simple to implement as a test...

82574L Are you using the stock driver? If so, please replace it with this one. The older drivers (including the stock ones) are known to have problems with this card.

I forgot to mention that I only have one WAN. Just positing in MultiWAN because that's where this thread is.

I figure that the WAN being marked as offline is normal, and the system would try to access internet through another port, but it seems like it gives up on Eth0 permanently and downs the link, instead of trying to access it again at a later time.

Oddly enough, the system stayed up this weekend when I was having some cable lines replaced and the modem had to go offline a few times. So the problem with ping failures might not be related to the link going down randomly. I guess all I can do is wait and see when it happens again, and then post my logs here.

EDIT: Just tried connecting to my VPN from work, and it appears my system is down again. Won't know for sure till I get home.

EDIT 2: Just came back up. Here is lspci and uname info.


[root@asrock ~]# lspci -k | grep Eth -A 4
03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Subsystem: ASRock Incorporation Device 10d3
Kernel driver in use: e1000e
Kernel modules: e1000e
04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Subsystem: ASRock Incorporation Device 10d3
Kernel driver in use: e1000e
Kernel modules: e1000e


[root@asrock ~]# uname -r
2.6.32-431.3.1.v6.x86_64
[root@asrock ~]#

If you think you have NIC driver problems, please post the output to:

lspci -k | grep Eth -A 4

uname -r

Peter Baldwin wrote:

George Miller, III wrote:

I THINK SOMETHING ELSE IS POSSIBLY GOING ON

Hopefully the issue can be resolved with the NIC driver update (if that applies here). The system/syswatch logs show the same behavior as other logs that I have seen - the ping tests to the Internet fail and WAN is taken out of service by the firewall. That's the intended behavior!

Here's a timeline summary. At 9:59:10, syswatch has failed to ping the upstream gateway (1 hop) and a ClearCenter cluster IP (69.90.141.72) for the 3rd (or is it 5th?) consecutive time. It marks the WAN as offline:

[quote]Thu Mar 6 09:59:01 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:03 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:08 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:08 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:10 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 09:59:10 2014 info: system - changing active WAN list - none (was eth2) ...

The firewall now reloads now that eth2/WAN is considered offline:

Mar 6 09:59:10 server firewall: Starting firewall... ...
Mar 6 09:59:10 server firewall: Loading environment ...
Mar 6 09:59:10 server firewall: Detected LAN role for interface: eth0 ...
Mar 6 09:59:10 server firewall: Detected LAN role for interface: eth1 ...
Mar 6 09:59:10 server firewall: WARNING: No configured WAN interfaces, continuing anyway... ...
...

Syswatch continues to ping both the upstream gateway and Internet host. The ping tests fail 3 more rounds (9 ping tests in all) before one succeeds:

Thu Mar 6 09:59:53 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 10:00:05 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 10:00:07 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 10:00:12 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 10:00:12 2014 warn: eth2 - connection is down ...
Thu Mar 6 10:00:22 2014 info: eth2 - ping check on server #1 passed - 63.162.206.129 ...
Thu Mar 6 10:00:22 2014 info: system - changing active WAN list - eth2 (was none) ...
...

And the firewall restarts with the WAN activated. The syswatch software is behaving the way it is supposed to be behaving. The big question is this: why are those ping tests failing? In the syswatch logs, do you see the ping failures last a similar amount of time (a couple of minutes) or are some outages longer?[/quote]


This is the exact thing that's happening to me. But the big problem is that it eventually downs the link on Eth0 and kills all access to the modem. Only a reboot of ClearOS fixes it.

I'm also still wondering if anyone running 32 bit has had this issue, or if it's limited to x64 systems. I ran the same version of ClearOS on my old 32 bit Proliant server for a few weeks before upgrading to the new system, and I don't remember the Proliant having any issues. Though it did have Broadcom NICs if I remember right.

George Miller, III wrote:

I THINK SOMETHING ELSE IS POSSIBLY GOING ON

Hopefully the issue can be resolved with the NIC driver update (if that applies here). The system/syswatch logs show the same behavior as other logs that I have seen - the ping tests to the Internet fail and WAN is taken out of service by the firewall. That's the intended behavior!

Here's a timeline summary. At 9:59:10, syswatch has failed to ping the upstream gateway (1 hop) and a ClearCenter cluster IP (69.90.141.72) for the 3rd (or is it 5th?) consecutive time. It marks the WAN as offline:

Thu Mar 6 09:59:01 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:03 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:08 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:08 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:10 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 09:59:10 2014 info: system - changing active WAN list - none (was eth2) ...

The firewall now reloads now that eth2/WAN is considered offline:

Mar 6 09:59:10 server firewall: Starting firewall... ...
Mar 6 09:59:10 server firewall: Loading environment ...
Mar 6 09:59:10 server firewall: Detected LAN role for interface: eth0 ...
Mar 6 09:59:10 server firewall: Detected LAN role for interface: eth1 ...
Mar 6 09:59:10 server firewall: WARNING: No configured WAN interfaces, continuing anyway... ...
...

Syswatch continues to ping both the upstream gateway and Internet host. The ping tests fail 3 more rounds (9 ping tests in all) before one succeeds:

Thu Mar 6 09:59:53 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 10:00:05 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 10:00:07 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 10:00:12 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 10:00:12 2014 warn: eth2 - connection is down ...
Thu Mar 6 10:00:22 2014 info: eth2 - ping check on server #1 passed - 63.162.206.129 ...
Thu Mar 6 10:00:22 2014 info: system - changing active WAN list - eth2 (was none) ...
...

And the firewall restarts with the WAN activated. The syswatch software is behaving the way it is supposed to be behaving. The big question is this: why are those ping tests failing? In the syswatch logs, do you see the ping failures last a similar amount of time (a couple of minutes) or are some outages longer?

FYI Version 3.0.4 is also sitting on my FTP site / repo for ClearOS 6
ftp://timburgess.net/repo/clearos/6/testing/x86_64/kmod-e1000e-3.0.4-1.clearos.x86_64.rpm

@ Maxxarcade,
The kmod one is essentially the same as Intel's only with the ability to survive kernel updates. You appear to have the latest one Tim compiled. There was a version available by yum as well, but I can't find it for the moment. I use the following to compile it:

#!/bin/bash



Version=3.0.4

# TARGET=i686

TARGET=x86_64



cd ~/rpmbuild/SOURCES



wget http://elrepo.reloumirrors.net/elrepo/el6/SRPMS/e1000e-kmod-$Version-1.el6.elrepo.src.rpm

rpmbuild --rebuild --define "kversion `uname -r`" --define 'packager Nick Howitt' e1000e-kmod-$Version-1.el6.elrepo.src.rpm

Normally I compile with a user "build".

I have not compiled the latest one as I no longer have the e1000e card.. It was only released in the last week or so but the version you have should be OK.

ChangeLog for 2.5.4 - http://downloadmirror.intel.com/15817/eng/readmefirst.txt

ChangeLog for 3.0.4 - http://downloadmirror.intel.com/15817/eng/releasenotes_e1000e.txt

The problem with xxxxxxxL has been known for some time, so I would expect the version you have is OK

I downloaded the driver, now I just need to remember how to transfer it to the ClearOS box and get it installed. I'm not too experienced in the Linux command line LOL.

Any idea what the differences are between the Kmod driver and the latest Intel driver? Does the Intel one fix the stability issues as well as the Kmod?

This is the version running here - locally compiled - released by Intel January 13, 2014

version:        3.0.4-NAPI

license:        GPL

description:    Intel(R) PRO/1000 Network Driver

author:         Intel Corporation, <linux.nics@intel.com>

srcversion:     B86187D504664D50DF59C36

https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=15817&keyword=%22e1000-%22&lang=eng

Here's what I get:

[root@asrock ~]# modinfo e1000e
filename: /lib/modules/2.6.32-431.3.1.v6.x86_64/weak-updates/e1000e/e1000e.ko
version: 2.5.4-NAPI
license: GPL
description: Intel(R) PRO/1000 Network Driver
author: Intel Corporation, <linux.nics@intel.com>
srcversion: 14FC0D45EE1DAA1B5E0DBBA

It won't say kmod. The best you can do is a "modinfo e1000e".

I successfully ran the code to install the kmod driver and rebooted, but lspci still says the same info, with the e1000e driver being used. Is that normal, or should it say kmod?

Thanks, I'll look into that later tonight.

Ironically, I bought this board instead of a consumer grade one that was 1/2 the price and twice as powerful, because the other board had Realtek NIC's. I figured the Intel server ones would be trouble free.

That's probably another reason why my old server worked so well. It had Broadcom NIC's. I ran that system for over 6 years 24/7. Only stopped using it because it draws 175 watts at idle LOL.

The 82574L has problems - you need to be using the latest driver...

see http://www.clearfoundation.com/component/option,com_kunena/Itemid,232/catid,6/func,view/id,59100/#59128 and similar posts in these forums...

My board has Intel NIC's.

http://www.newegg.com/Product/Product.aspx?Item=N82E16813157417

[root@asrock ~]# lspci -k | grep Eth -A 4
03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Subsystem: ASRock Incorporation Device 10d3
Kernel driver in use: e1000e
Kernel modules: e1000e
04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Subsystem: ASRock Incorporation Device 10d3
Kernel driver in use: e1000e
Kernel modules: e1000e


It's running fine at the moment, since I rebooted it about 19 hours ago. The problem occurs randomly, sometimes 3 or 4 days later. A while back it happened 2 or 3 times within 24 hours.

Maxxarcade are you by any chance using an RTL8111/8168 with the 8169 driver - if so, that would probably explain your problems..

What is the output of

lspci -k | grep Eth -A 4

Ah, George just beat me to it while I was typing my last post. Pretty much the same thing for me!

I was just about to send back my new server board, thinking it had a flaky LAN controller. Has anyone noticed if this issue is limited to the x64 version?

Also, when the problem occurs, the Ethernet light on my modem turns orange, meaning 10/100 link. Normally it is blue which means it is linked in Gigabit. It's almost like something is crashing the LAN driver for Eth0. The ClearOS dashboard shows the link as down, and I can't access the modem admin page.

Charter Cable, DHCP. Using a Motorola SB6120 modem if I remember right.

I've been having some signal issues lately as well, which probably explains the ping failures. However that never caused my previous ClearOS server to drop the WAN port completely. On the old server, I could still see the modem admin page during a failure. With the new server, it puts the link completely down on the WAN. Rebooting the server fixes it, and there was no need to reboot the modem.

I wonder if the 32 bit version behaves differently for some reason? My new server uses x64. Both were running the latest version of ClearOS Community. It's almost as if it's trying to failover to another WAN port that's not there. The system log says "No configured WAN interfaces" whenever the problem occurs. Normally Eth0 is WAN, and Eth1 is LAN.

I'll see if I can post a clip of my logs later tonight if that would help any. I just need to edit out my IP address from them first.

I THINK SOMETHING ELSE IS POSSIBLY GOING ON

I too have a similar situation as Maxxarcade. I applied the patch a couple of days ago on one system, but it still looses direction with regard to the WAN port in the firewall according to the system logs. The WAN side is a static IP:

Mar 6 09:59:10 server firewall: Starting firewall... ...
Mar 6 09:59:10 server firewall: Loading environment ...
Mar 6 09:59:10 server firewall: Detected LAN role for interface: eth0 ...
Mar 6 09:59:10 server firewall: Detected LAN role for interface: eth1 ...
Mar 6 09:59:10 server firewall: WARNING: No configured WAN interfaces, continuing anyway... ...
Mar 6 09:59:10 server firewall: Setting kernel parameters ...
Mar 6 09:59:10 server firewall: Detected LAN info - eth0 10.0.1.1 on network 10.0.1.0/24 ...
Mar 6 09:59:10 server firewall: Detected LAN info - eth1 10.0.0.1 on network 10.0.0.0/24 ...
Mar 6 09:59:10 server firewall: Using gateway mode ...
Mar 6 09:59:10 server firewall: Loading kernel modules ...
Mar 6 09:59:10 server firewall: Loading kernel modules for NAT ...
Mar 6 09:59:10 server firewall: Setting default policy to DROP ...
Mar 6 09:59:10 server firewall: Defining custom chains ...
Mar 6 09:59:10 server firewall: Running blocked external rules ...
Mar 6 09:59:10 server firewall: Running custom rules ...
Mar 6 09:59:10 server firewall: Running common rules ...
Mar 6 09:59:10 server firewall: Running incoming denied rules ...
Mar 6 09:59:10 server firewall: Running user-defined incoming rules ...
Mar 6 09:59:10 server firewall: Allowing incoming tcp port/range 22 ...
Mar 6 09:59:10 server firewall: Allowing incoming tcp port/range 81 ...
Mar 6 09:59:10 server firewall: Disabling PPTP pass-through ...
Mar 6 09:59:10 server firewall: Allowing incoming GRE (47) for PPTP ...
Mar 6 09:59:10 server firewall: Allowing incoming TCP (6) port 1723 for PPTP ...
Mar 6 09:59:10 server firewall: Running default incoming allowed rules ...
Mar 6 09:59:10 server firewall: Running user-defined port forward rules ...
Mar 6 09:59:10 server firewall: No WAN interfaces up or configured, not starting bandwidth manager ...
Mar 6 09:59:10 server firewall: Skipping 1-to-1 NAT rules - no active WAN interfaces ...
Mar 6 09:59:10 server firewall: Running user-defined proxy rules ...
Mar 6 09:59:10 server firewall: Content filter is online ...
Mar 6 09:59:10 server firewall: Web proxy is online ...
Mar 6 09:59:10 server firewall: Enabled proxy+filter transparent mode for filter port: 8080 ...
Mar 6 09:59:10 server firewall: Blocking proxy port 3128 to force users through content filter ...
Mar 6 09:59:10 server firewall: Running multipath ...
Mar 6 09:59:10 server firewall: Enabling standby NAT on WAN interface eth2 ...
Mar 6 09:59:10 server firewall: Running user-defined outgoing block rules ...
Mar 6 09:59:10 server firewall: Running default forwarding rules ...
Mar 6 09:59:10 server firewall: Execution time: 0.070s ...
Mar 6 09:59:10 server firewall: Running post-firewall 10650 ...
Mar 6 09:59:10 server firewall: Running /etc/clearos/firewall.d/local ...
Mar 6 09:59:10 server firewall: # This script is run after every firewall restart. Add custom rules here. ...
Mar 6 09:59:10 server firewall: Running /etc/clearos/firewall.d/10-ibvpn ...
Mar 6 09:59:10 server firewall: Running /etc/clearos/firewall.d/10-intrusion-prevention ...
Mar 6 09:59:10 server firewall: Running /etc/clearos/firewall.d/10-ntp ...

Mar 6 10:00:22 server firewall: Starting firewall... ...
Mar 6 10:00:22 server firewall: Loading environment ...
Mar 6 10:00:22 server firewall: Detected WAN role for interface: eth2 ...
Mar 6 10:00:22 server firewall: Detected LAN role for interface: eth0 ...
Mar 6 10:00:22 server firewall: Detected LAN role for interface: eth1 ...
Mar 6 10:00:22 server firewall: Setting kernel parameters ...
Mar 6 10:00:22 server firewall: Detected WAN info - eth2 63.162.206.169 on network 63.162.206.128/26 ...
Mar 6 10:00:22 server firewall: Detected LAN info - eth0 10.0.1.1 on network 10.0.1.0/24 ...
Mar 6 10:00:22 server firewall: Detected LAN info - eth1 10.0.0.1 on network 10.0.0.0/24 ...
Mar 6 10:00:22 server firewall: Using gateway mode ...
Mar 6 10:00:22 server firewall: Loading kernel modules ...
Mar 6 10:00:22 server firewall: Loading kernel modules for NAT ...
Mar 6 10:00:22 server firewall: Setting default policy to DROP ...
Mar 6 10:00:22 server firewall: Defining custom chains ...
Mar 6 10:00:22 server firewall: Running blocked external rules ...
Mar 6 10:00:22 server firewall: Running custom rules ...
Mar 6 10:00:22 server firewall: Running common rules ...
Mar 6 10:00:22 server firewall: Running incoming denied rules ...
Mar 6 10:00:22 server firewall: Running user-defined incoming rules ...
Mar 6 10:00:22 server firewall: Allowing incoming tcp port/range 22 ...
Mar 6 10:00:22 server firewall: Allowing incoming tcp port/range 81 ...
Mar 6 10:00:22 server firewall: Disabling PPTP pass-through ...
Mar 6 10:00:22 server firewall: Allowing incoming GRE (47) for PPTP ...
Mar 6 10:00:22 server firewall: Allowing incoming TCP (6) port 1723 for PPTP ...
Mar 6 10:00:22 server firewall: Running default incoming allowed rules ...
Mar 6 10:00:22 server firewall: Running user-defined port forward rules ...
Mar 6 10:00:22 server firewall: Initializing bandwidth manager ...
Mar 6 10:00:22 server firewall: Bandwidth manager is enabled but no WAN interfaces configured! ...
Mar 6 10:00:22 server firewall: Running 1-to-1 NAT rules ...
Mar 6 10:00:22 server firewall: Running user-defined proxy rules ...
Mar 6 10:00:22 server firewall: Content filter is online ...
Mar 6 10:00:22 server firewall: Web proxy is online ...
Mar 6 10:00:22 server firewall: Enabled proxy+filter transparent mode for filter port: 8080 ...
Mar 6 10:00:22 server firewall: Blocking proxy port 3128 to force users through content filter ...
Mar 6 10:00:22 server firewall: Running multipath ...
Mar 6 10:00:22 server firewall: Enabling NAT on WAN interface eth2 ...
Mar 6 10:00:22 server firewall: Running user-defined outgoing block rules ...
Mar 6 10:00:22 server firewall: Running default forwarding rules ...
Mar 6 10:00:22 server firewall: Execution time: 0.068s ...
Mar 6 10:00:22 server firewall: Running post-firewall 10650 ...
Mar 6 10:00:22 server firewall: Running /etc/clearos/firewall.d/local ...
Mar 6 10:00:22 server firewall: # This script is run after every firewall restart. Add custom rules here. ...
Mar 6 10:00:22 server firewall: Running /etc/clearos/firewall.d/10-ibvpn ...
Mar 6 10:00:22 server firewall: Running /etc/clearos/firewall.d/10-intrusion-prevention ...
Mar 6 10:00:22 server firewall: Running /etc/clearos/firewall.d/10-ntp ...
Mar 6 10:28:30 server firewall: Starting firewall... ...


Notice that it didn't detect the WAN port in the firs group, but the "Starting firewall..." that follows detected the WAN PORT.
We called out the ISP and they are saying the internet connection is not failing at all.

I THINK SOMETHING ELSE IS GOING ON POSSIBLY????

Thu Mar 6 09:59:01 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:03 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:08 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:08 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:10 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 09:59:10 2014 info: system - changing active WAN list - none (was eth2) ...
Thu Mar 6 09:59:10 2014 info: system - current WANs in use - none ...
Thu Mar 6 09:59:10 2014 info: system - restarting firewall ...
Thu Mar 6 09:59:10 2014 info: system - updating intrusion prevention whitelist ...
Thu Mar 6 09:59:10 2014 info: system - adding ping server 69.90.141.72 ...
Thu Mar 6 09:59:10 2014 info: system - adding ping server 63.162.206.129 ...
Thu Mar 6 09:59:10 2014 info: system - adding DNS server 8.8.8.8 ...
Thu Mar 6 09:59:10 2014 info: system - adding DNS server 8.8.4.4 ...
Thu Mar 6 09:59:22 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:24 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:29 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:29 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:31 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 09:59:41 2014 info: system - heartbeat... ...
Thu Mar 6 09:59:44 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:46 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:51 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:51 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:53 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 10:00:05 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 10:00:07 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 10:00:12 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 10:00:12 2014 warn: eth2 - connection is down ...
Thu Mar 6 10:00:22 2014 info: eth2 - ping check on server #1 passed - 63.162.206.129 ...
Thu Mar 6 10:00:22 2014 info: system - changing active WAN list - eth2 (was none) ...
Thu Mar 6 10:00:22 2014 info: system - current WANs in use - eth2 ...
Thu Mar 6 10:00:22 2014 info: system - restarting firewall ...
Thu Mar 6 10:00:22 2014 info: system - updating intrusion prevention whitelist ...
Thu Mar 6 10:00:22 2014 info: system - adding ping server 69.90.141.72 ...
Thu Mar 6 10:00:22 2014 info: system - adding ping server 63.162.206.129 ...
Thu Mar 6 10:00:22 2014 info: system - adding DNS server 8.8.8.8 ...
Thu Mar 6 10:00:22 2014 info: system - adding DNS server 8.8.4.4 ...
Thu Mar 6 10:00:22 2014 info: system - dynamic DNS updated ...



Here is the syswatch log from the same time frame:

Thu Mar 6 09:59:01 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:03 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:08 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:08 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:10 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 09:59:10 2014 info: system - changing active WAN list - none (was eth2) ...
Thu Mar 6 09:59:10 2014 info: system - current WANs in use - none ...
Thu Mar 6 09:59:10 2014 info: system - restarting firewall ...
Thu Mar 6 09:59:10 2014 info: system - updating intrusion prevention whitelist ...
Thu Mar 6 09:59:10 2014 info: system - adding ping server 69.90.141.72 ...
Thu Mar 6 09:59:10 2014 info: system - adding ping server 63.162.206.129 ...
Thu Mar 6 09:59:10 2014 info: system - adding DNS server 8.8.8.8 ...
Thu Mar 6 09:59:10 2014 info: system - adding DNS server 8.8.4.4 ...
Thu Mar 6 09:59:22 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:24 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:29 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:29 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:31 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 09:59:41 2014 info: system - heartbeat... ...
Thu Mar 6 09:59:44 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 09:59:46 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 09:59:51 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 09:59:51 2014 warn: eth2 - connection is down ...
Thu Mar 6 09:59:53 2014 info: eth2 - waiting for static IP reconnect ...
Thu Mar 6 10:00:05 2014 info: eth2 - ping check on gateway failed - 63.162.206.129 ...
Thu Mar 6 10:00:07 2014 info: eth2 - ping check on server #1 failed - 63.162.206.129 ...
Thu Mar 6 10:00:12 2014 info: eth2 - ping check on server #2 failed - 69.90.141.72 ...
Thu Mar 6 10:00:12 2014 warn: eth2 - connection is down ...
Thu Mar 6 10:00:22 2014 info: eth2 - ping check on server #1 passed - 63.162.206.129 ...
Thu Mar 6 10:00:22 2014 info: system - changing active WAN list - eth2 (was none) ...
Thu Mar 6 10:00:22 2014 info: system - current WANs in use - eth2 ...
Thu Mar 6 10:00:22 2014 info: system - restarting firewall ...
Thu Mar 6 10:00:22 2014 info: system - updating intrusion prevention whitelist ...
Thu Mar 6 10:00:22 2014 info: system - adding ping server 69.90.141.72 ...
Thu Mar 6 10:00:22 2014 info: system - adding ping server 63.162.206.129 ...
Thu Mar 6 10:00:22 2014 info: system - adding DNS server 8.8.8.8 ...
Thu Mar 6 10:00:22 2014 info: system - adding DNS server 8.8.4.4 ...
Thu Mar 6 10:00:22 2014 info: system - dynamic DNS updated ...

@ Maxxarcade - do you have a PPPoE connection?

Maxxarcade wrote:

... saying ping check failed, changing active WAN list, stuff like that.

A ping failure is a ping failure. Any idea why the ping is failing? Syswatch is reporting the ping problem, not creating it!

I've been having frequent issues with my system as well. At first I thought it was my new hardware, but I think it's ClearOS related. I installed the syswatch update, which took care of the log getting flooded, but it still drops my WAN port after a couple days, saying ping check failed, changing active WAN list, stuff like that.

Hopefully this gets resolved soon, because I need this system to work. I've been a ClearOS user for about 7 years and never had an issue like this before.

I have a couple of log files in hand, but I haven't found any smoking guns (just flaky PPPoE connections).

The syswatch change was nothing earth shattering. There is a case where the interface name used in the network configuration filename won't match the name inside the file. For example:

/etc/sysconfig/network-scripts/ifcfg-eth0



DEVICE=eth33

IPADDR="192.168.55.6"

NETMASK="255.255.255.0"

... snip ...

The ifcfg-eth0 filename uses eth33. The use case is unusual and certainly something that can only be accomplished via the command line. Syswatch will now honor the DEVICE parameter inside the configuration file.

So what was the bug? The syswatch network monitor only cares about the configuration of external/WAN interfaces. In the buggy 6.5.2 release, syswatch started monitoring all network interfaces! The algorithm for determining network up/down etc. didn't change at all.

Just in case, we're still investigating, but have come up empty so far.

I didn't realise it was a clearos issue until I was going to make a post about it and saw this it did seem weird it started to happen recently but this makes sense as my internet on two of my clients servers drop out frequently and the server could always connect to the internet again but has to restart networking or restart the server to work again.

Thanks for the update Peter I'll monitor this and update.

Once installed, you don't have to do anything. The yum process looks after it.

It does not quite work like you are imagining. In updates-testing there is the next release, not a patch on the current release, so yum will never roll it back to the previous version.

Auto update may update the file again if they release a higher version. That is part of the normal update process. If you wait a few days or weeks, the Clear team will move the patch, possibly to clearos-updates and the auto-update process will then install the new release. You are just speeding up the process by picking the new release out of the updates-testing repo.

Once we apply the patch by running this:
yum --enablerepo=clearos-test,clearos-updates-testing upgrade syswatch
and answere "y" to the question and it installs....

What's next? I am not knowledgeable with the update process - so forgive me.

Is there an auto update that will overwrite this patch down the road, or do we need to run something else in a few days once the bug is fixed?

What is to prevent the auto updates from overwriting the patch and putting us back where we were?

Thanks for your help,

George

Could you send the /var/log/system and /var/log/syswatch log files to developer@clearfoundation.com? We might be able to spot the problem.

Hi

I am still having issues with this even after the update. (yum --enablerepo=clearos-test,clearos-updates-testing upgrade syswatch)
When I downgrade back to syswatch-6.5.0-1.v6.noarch the connection stays active.
On syswatch-6.5.2-1.v6.noarch and on syswatch-6.5.3-1.v6.noarch the PPPoE connection seems to freez up.
The connection looks active but no responce.

I dont have a Multiwan setup and the issue is on all our ClearOS 6 servers that downloaded the syswatch-6.5.2-1.v6.noarch update

Older ClearOS 5 Server seems unaffected.

Alexander Petin wrote:

In my case it was not harmless because LAN is DHCP enabled, so the interface continue restarts every 10 secs.

Point well taken. It's fairly unusual to use DHCP on a LAN interface, but it's certainly possible. In fact, I do that on test VMs all the time!