After a reboot my firewall would not start, the issue has been isolated to QoS manager, when disabled the firewall starts fine, this config has been solid for months with nothing but automatic upgrades,
We have had a couple of kernel upgrades since the install , below is the output of uname
3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Here's the ouput of the firewall start, it looks like imq is not loading? I have no clue how to resolve this, thanks for the help.
modprobe: FATAL: Module imq not found.
firewall: /sbin/modprobe imq numdevs=1 = 256
Cannot find device "imq0"
firewall: /sbin/ip link set imq0 up = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 root handle 1: htb default 16 r2q 296 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1: classid 1:1 htb rate 25400kbit = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:10 htb rate 3810kbit ceil 25400kbit prio 0 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:10 handle 10: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:11 htb rate 3810kbit ceil 25400kbit prio 1 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:11 handle 11: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 11 fw flowid 1:11 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:12 htb rate 3556kbit ceil 25400kbit prio 2 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:12 handle 12: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 12 fw flowid 1:12 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:13 htb rate 3556kbit ceil 25400kbit prio 3 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:13 handle 13: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 13 fw flowid 1:13 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:14 htb rate 3556kbit ceil 25400kbit prio 4 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:14 handle 14: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 14 fw flowid 1:14 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:15 htb rate 3556kbit ceil 25400kbit prio 5 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:15 handle 15: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 15 fw flowid 1:15 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:16 htb rate 3556kbit ceil 25400kbit prio 6 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:16 handle 16: sfq perturb 10 = 256
Cannot find device "imq0"
We have had a couple of kernel upgrades since the install , below is the output of uname
3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Here's the ouput of the firewall start, it looks like imq is not loading? I have no clue how to resolve this, thanks for the help.
modprobe: FATAL: Module imq not found.
firewall: /sbin/modprobe imq numdevs=1 = 256
Cannot find device "imq0"
firewall: /sbin/ip link set imq0 up = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 root handle 1: htb default 16 r2q 296 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1: classid 1:1 htb rate 25400kbit = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:10 htb rate 3810kbit ceil 25400kbit prio 0 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:10 handle 10: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:11 htb rate 3810kbit ceil 25400kbit prio 1 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:11 handle 11: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 11 fw flowid 1:11 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:12 htb rate 3556kbit ceil 25400kbit prio 2 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:12 handle 12: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 12 fw flowid 1:12 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:13 htb rate 3556kbit ceil 25400kbit prio 3 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:13 handle 13: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 13 fw flowid 1:13 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:14 htb rate 3556kbit ceil 25400kbit prio 4 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:14 handle 14: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 14 fw flowid 1:14 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:15 htb rate 3556kbit ceil 25400kbit prio 5 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:15 handle 15: sfq perturb 10 = 256
Cannot find device "imq0"
firewall: /sbin/tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 15 fw flowid 1:15 = 256
Cannot find device "imq0"
firewall: /sbin/tc class add dev imq0 parent 1:1 classid 1:16 htb rate 3556kbit ceil 25400kbit prio 6 = 256
Cannot find device "imq0"
firewall: /sbin/tc qdisc add dev imq0 parent 1:16 handle 16: sfq perturb 10 = 256
Cannot find device "imq0"
Share this post:
Responses (10)
-
Accepted Answer
-
Accepted Answer
On bug 13621 Peter has said he will push through a kernel update. This should supersede the el7 kernel and fix the issue.
As a thought, it may be able to permanently avoid this issue from ever happening again by adding "exclude=kernel*el7*" to yum.conf. I've removed the el7 kernel, added the line to yum.conf and done a "yum update" and it has not tried updating the kernel again. Remove the line from yum.conf causes yum to try to update the kernel again. I don't want to implement this fix yet until the kernel has been updated to a v7 kernel as I don't know what would happen to the support packages (devel, headers, tools and tools-libs) which should match version numbers. -
Accepted Answer
-
Accepted Answer
Thanks for all the help, so I worked around it by using yum to install the latest v7 Kernel that was available since that one was skipped. With that said as @Nick pointed out there's a couple of rogue kernels out there in the repository that needs clean up. Here's the yum command in case someone else runs into this yum install kernel-3.10.0-514.2.2.v7 this will automatically put this kernel on top of the Grub list and therefore no more editing is required, since it seems like 514.6.1 & 514.2.2 where released simultaneously to the repo, I doubt that 2.2 is installed on your system.
Installed Packages
kernel.x86_64 3.10.0-327.3.1.el7 @anaconda/7.2.0
kernel.x86_64 3.10.0-327.36.3.v7 @clearos-updates
kernel.x86_64 3.10.0-514.2.2.v7 @clearos
kernel.x86_64 3.10.0-514.6.1.el7 @private-clearcenter-verified-updates
Available Packages
kernel.x86_64 3.10.0-327.36.3.v7 private-verified-updates
kernel.x86_64 3.10.0-514.el7 private-verified-updates
kernel.x86_64 3.10.0-514.2.2.el7 private-verified-updates
kernel.x86_64 3.10.0-514.2.2.v7 clearos
kernel.x86_64 3.10.0-514.2.2.v7 clearos-verified
kernel.x86_64 3.10.0-514.2.2.v7 private-verified-updates
kernel.x86_64 3.10.0-514.6.1.el7 private-verified-updates -
Accepted Answer
Peter Baldwin wrote:
@Peter,
Yes, you definitely want to use the v7 kernels. I'll add a tracker item to make sure the QoS/firewall system can detect this situation.
Your tracker could help but it is not the root cause. The O/P has not chosen to install an el7 kernel and I have not either. As I said in my e-mail, my kernel arrived by the overnight yum update on 23/02. If you query the repo's you get:
This is from the Business repos. The el7 kernel is the last one pushed through and it is still there. This means everyone on Business using automatic updates will have been installng this kernel. Anyone of them who reboots will have the same issue with QoS as the O/P (and Bandwidth). You can see that 2 el7 kernels have made it through to the repos, but this should never happen.[root@server ~]# yum list kernel-3* --show-duplicates
Loaded plugins: clearcenter-marketplace, fastestmirror
ClearCenter Marketplace: fetching repositories...
Loading mirror speeds from cached hostfile
<ship>
Installed Packages
kernel.x86_64 3.10.0-327.36.3.v7 @clearos-updates
kernel.x86_64 3.10.0-514.2.2.v7 @clearos-qa
kernel.x86_64 3.10.0-514.6.1.el7 @private-verified-updates
Available Packages
kernel.x86_64 3.10.0-327.36.3.v7 private-verified-updates
kernel.x86_64 3.10.0-514.el7 private-verified-updates
kernel.x86_64 3.10.0-514.2.2.el7 private-verified-updates
kernel.x86_64 3.10.0-514.2.2.v7 clearos
kernel.x86_64 3.10.0-514.2.2.v7 clearos-updates
kernel.x86_64 3.10.0-514.2.2.v7 clearos-verified
kernel.x86_64 3.10.0-514.2.2.v7 private-verified-updates
kernel.x86_64 3.10.0-514.6.1.el7 private-verified-updates
As a minimum I believe ClearOS should be pulling the 3.10.0-514.6.1.el7 kernel from the repo to prevent any more damage. At the same time ClearOS should be pushing through a v7 kernel a.s.a.p. to reverse the damage caused by the previous update.
Note this is not an issue with Community which is still at 3.10.0-514.2.2.v7. Bearing in mind Business should not be ahead of Community in the update cycle, it looks like the 3.10.0-514.6.1.el7 kernel is a complete rogue. -
Accepted Answer
-
Accepted Answer
it looks like I can, since yum is keeping 3 Kernels, (right now we are back to prod but I can test later). From what you posted and what I have read I assume booting temporarly from the v7 kernel should get me back up and running?
[root@gateway ~]# rpm -q kernel
kernel-3.10.0-327.3.1.el7.x86_64
kernel-3.10.0-327.36.3.v7.x86_64
kernel-3.10.0-514.6.1.el7.x86_64
[root@gateway ~]# uname -a
Linux removed.name.local 3.10.0-514.6.1.el7.x86_64 -
Accepted Answer
-
Accepted Answer
Please login to post a reply
You will need to be logged in to be able to post a reply. Login using the form on the right or register an account if you are new here.
Register Here »