Thanks for all the help, so I worked around it by using yum to install the latest v7 Kernel that was available since that one was skipped. With that said as @Nick pointed out there's a couple of rogue kernels out there in the repository that needs clean up. Here's the yum command in case someone else runs into this yum install kernel-3.10.0-514.2.2.v7 this will automatically put this kernel on top of the Grub list and therefore no more editing is required, since it seems like 514.6.1 & 514.2.2 where released simultaneously to the repo, I doubt that 2.2 is installed on your system.


Installed Packages
kernel.x86_64 3.10.0-327.3.1.el7 @anaconda/7.2.0
kernel.x86_64 3.10.0-327.36.3.v7 @clearos-updates
kernel.x86_64 3.10.0-514.2.2.v7 @clearos
kernel.x86_64 3.10.0-514.6.1.el7 @private-clearcenter-verified-updates
Available Packages
kernel.x86_64 3.10.0-327.36.3.v7 private-verified-updates
kernel.x86_64 3.10.0-514.el7 private-verified-updates
kernel.x86_64 3.10.0-514.2.2.el7 private-verified-updates
kernel.x86_64 3.10.0-514.2.2.v7 clearos
kernel.x86_64 3.10.0-514.2.2.v7 clearos-verified
kernel.x86_64 3.10.0-514.2.2.v7 private-verified-updates
kernel.x86_64 3.10.0-514.6.1.el7 private-verified-updates

it looks like I can, since yum is keeping 3 Kernels, (right now we are back to prod but I can test later). From what you posted and what I have read I assume booting temporarly from the v7 kernel should get me back up and running?

[root@gateway ~]# rpm -q kernel
kernel-3.10.0-327.3.1.el7.x86_64
kernel-3.10.0-327.36.3.v7.x86_64
kernel-3.10.0-514.6.1.el7.x86_64
[root@gateway ~]# uname -a
Linux removed.name.local 3.10.0-514.6.1.el7.x86_64

Firewall Crash When enabling QoS

After a reboot my firewall would not start, the issue has been isolated to QoS manager, when disabled the firewall starts fine, this config has been solid for months with nothing but automatic upgrades,

We have had a couple of kernel upgrades since the install , below is the output of uname

3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Here's the ouput of the firewall start, it looks like imq is not loading? I have no clue how to resolve this, thanks for the help.

modprobe: FATAL: Module imq not found.
firewall: /sbin/modprobe imq numdevs=1 = 256
Cannot find device "imq0"
firewall: /sbin/ip link set imq0 up = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 root handle 1: htb default 16 r2q 296 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1: classid 1:1 htb rate 25400kbit = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:10 htb rate 3810kbit ceil 25400kbit prio 0 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:10 handle 10: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:11 htb rate 3810kbit ceil 25400kbit prio 1 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:11 handle 11: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 11 fw flowid 1:11 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:12 htb rate 3556kbit ceil 25400kbit prio 2 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:12 handle 12: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 12 fw flowid 1:12 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:13 htb rate 3556kbit ceil 25400kbit prio 3 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:13 handle 13: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 13 fw flowid 1:13 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:14 htb rate 3556kbit ceil 25400kbit prio 4 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:14 handle 14: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 14 fw flowid 1:14 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:15 htb rate 3556kbit ceil 25400kbit prio 5 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:15 handle 15: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 15 fw flowid 1:15 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:16 htb rate 3556kbit ceil 25400kbit prio 6 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:16 handle 16: sfq perturb 10 = 256
Cannot find device "imq0"