A little bit of background. I just revved two ClearOS 6.7 boxes up to ClearOS 7.2 with a clean install. They've both been running for a few years without issue doing IPSEC point to point. After the upgrade, I'm having strange behavior where one side seems *mostly* invisible to the other. I say mostly because I can actually do DNS lookups on the remote LAN and can access hosts on port 80/443 that have something running. I cannot ping or ssh to those same hosts. Interestingly enough, I have port forwarding set up for those two ports as well on the firewall. I do have 500/4500 UDP open on both sides as well as 22 and a few other ports I use for various services. Nothing else seems to be awry here, but OpenSwan or the firewall are doing something strange.
Both systems are set up with a near identical software load and both have the exact same firewall rules minus the two ports being forwarded. Any help would be appreciated. Pulling my hair out a bit here digging through logs and posts to try and see what is going on.
Both systems are set up with a near identical software load and both have the exact same firewall rules minus the two ports being forwarded. Any help would be appreciated. Pulling my hair out a bit here digging through logs and posts to try and see what is going on.
Share this post:
Responses (17)
-
Accepted Answer
Are you sure that in ClearOS 6 you did not have an extra firewall rule which SNAT'd any packets emerging from the tunnel?
For tcpdump I google "tcpdump examples" and use that. First determine your interface (use the ClearOS LAN interface and monitor that) then you probably just want to pick out packets containint both the source and destination IP's of your test. I always build up my queries bit by bit as I never remember where the brackets and quotes go. I start big then narrow down.
For the firewall, a couple of rules in your remote machine will do:
You may want to add narrow down by interface as well but I am not sure. Messages go to /var/log/messages.$IPTABLES -s your_source_ip -d your_remote_ip -j LOG --log-prefix "outbound packets"
$IPTABLES -s your_remote_ip -d your_source_ip -j LOG --log-prefix prefix "return packets" -
Accepted Answer
Nick Howitt wrote:
What sort of devices are you trying to connect to? If they are Windows devices, can you please try dropping their firewalls. Also you could try sniffing the packets on the ClearOS LAN interface, either with tcpdump of a firewall rule, but I doubt if you have a problem as the pings are getting through.
It isn't a client firewall issue. The majority of systems running on the other side are Macs or linux boxes (mostly CentOS 7) with firewall disabled since they are hosting private internal services on a variety of ports for a ton of things.
If we go back to the start of this problem as well - there were no issues with client connectivity between sites when using ClearOS 6. It was the move to ClearOS 7 that started to create this issue - so it is safe to rule out anything on the client side as none of that has changed between when I had a perfectly working tunnel under COS6 and now. Also, it is important to note that when using OpenVPN to do remote client connectivity everything works as expected to all clients at either site - so client configuration isn't an issue.
Nick Howitt wrote:
What may be interesting is seeing the source IP's on the packets as the exit the remote firewall.
I'd be very interested in some example tests that I could run with TCPDump to determine what might be happening. I'm admittedly a bit of a neophyte with the tool and the documentation listed in this thread is open enough that it is hard to determine where I should even start. Internal IP to external IP? All traffic? From where? To where? I'm happy to do any tests recommended to get this sorted. -
Accepted Answer
What sort of devices are you trying to connect to? If they are Windows devices, can you please try dropping their firewalls. Also you could try sniffing the packets on the ClearOS LAN interface, either with tcpdump of a firewall rule, but I doubt if you have a problem as the pings are getting through. What may be interesting is seeing the source IP's on the packets as the exit the remote firewall. -
Accepted Answer
Nick Howitt wrote:
Your traceroutes are completely wrong as nothing is going through the tunnel. You should not see any hops between the IPsec endpoints. If you want to check the tunnel working you should be doing a traceroute to the remote ClearOS LAN IP directly, From A do a traceroute to rightsourceip=192.168.10.1 or any non-Windows device on the remote LAN. The problem with Windows is its firewall, where it may reject pings and other connections from outside its own LAN. You would need to check its settings.
That's likely because I did external interface to external interface. Doing internal to internal looks like this:
Firewall up and down are identical in this case (site A to site B, from 192.168.3.1 internal LAN):
traceroute to 192.168.10.1 (192.168.10.1), 30 hops max, 60 byte packets
1 192.168.10.1 (192.168.10.1) 36.335 ms 41.213 ms 46.864 ms
Doing the reverse also works identical with firewall up or down (site B to site A, from 192.168.10.1 internal LAN)
traceroute to 192.168.3.1 (192.168.3.1), 30 hops max, 60 byte packets
1 router.copperbell.lan (192.168.3.1) 0.051 ms 0.030 ms 0.028 ms
These are both router to router trace routes, so the tunnel is up and the two end points are responding. Going onto a client at site A and doing a trace route to a known good running machine at site B gets me this:
300dpi:~ szumlins$ traceroute 192.168.10.5
traceroute to 192.168.10.5 (192.168.10.5), 64 hops max, 52 byte packets
1 router (192.168.3.1) 112.262 ms 0.903 ms 2.851 ms
2 192.168.10.1 (192.168.10.1) 34.109 ms 156.985 ms 30.984 ms
3 192.168.10.5 (192.168.10.5) 38.667 ms 35.952 ms 31.828 ms
However I cannot access any services on that machine (192.168.10.5) through the tunnel - the firewall appears to be blocking all of them.
If I do the inverse and trace route a machine on site A from a machine on site B I get the following:
[szumlins@trashcan:~]: traceroute 192.168.3.20
traceroute to 192.168.3.20 (192.168.3.20), 64 hops max, 52 byte packets
1 router (192.168.10.1) 0.573 ms 0.209 ms 0.196 ms
2 192.168.3.1 (192.168.3.1) 35.062 ms 33.166 ms 30.841 ms
3 192.168.3.20 (192.168.3.20) 32.994 ms 32.749 ms 33.170 ms
In both cases, the firewall being on/off has no impact to the trace route results. But I can't actually access any services on any machines on the opposite side of the tunnel.
As soon as I turn off the firewall at both sites, my tunnel works as expected, but now my NAT/forwarding doesn't work at either site (which is expected). So there is a config in iptables that is taking precedent over the tunnel and blocking communications. At this point that is pretty clear because when it is removed from the equation everything works. What I can't figure out is what that setting is or what it is happening the way it is.
Nick Howitt wrote:
Similarly if you want to access a web server at B from A, the FQDN at A should resolve to a LAN IP at B and vice versa. This is achieved through the DNS Server settings (or the hosts file)
It isn't a DNS issue. As I said earlier, completely eliminating DNS 100% from all configs and using only IPs results in the exact same behavior under the same tests. That said, I have full working forward/reverse DNS at both sites for the LANs. -
Accepted Answer
Your traceroutes are completely wrong as nothing is going through the tunnel. You should not see any hops between the IPsec endpoints. If you want to check the tunnel working you should be doing a traceroute to the remote ClearOS LAN IP directly, From A do a traceroute to rightsourceip=192.168.10.1 or any non-Windows device on the remote LAN. The problem with Windows is its firewall, where it may reject pings and other connections from outside its own LAN. You would need to check its settings.
Similarly if you want to access a web server at B from A, the FQDN at A should resolve to a LAN IP at B and vice versa. This is achieved through the DNS Server settings (or the hosts file) -
Accepted Answer
Nick Howitt wrote:
Can we start again with the troubleshooting as things may have changed over 2 years. Please can you give the conn definitions from either end from /etc/ipsec.d.*.conf and the connection log from both ends.
Site A
[root@router ~]# cat /etc/ipsec.d/ipsec.unmanaged.pvt.conf
conn pvt
type=tunnel
authby=secret
auto=start
left=site.a.public.ip
leftnexthop=site.a.gateway
leftsourceip=192.168.3.1
leftsubnet=192.168.3.0/24
right=site.b.public.ip
rightsubnet=192.168.10.0/24
rightnexthop=site.b.gateway
rightsourceip=192.168.10.1
Site B
[root@router ~]# cat /etc/ipsec.d/ipsec.unmanaged.pvt.conf
conn pvt
type=tunnel
authby=secret
auto=start
left=site.b.pubic.ip
leftnexthop=site.b.gateway
leftsourceip=192.168.10.1
leftsubnet=192.168.10.0/24
right=site.a.public.ip
rightsubnet=192.168.3.0/24
rightnexthop=site.a.gateway
rightsourceip=192.168.3.1
Nick Howitt wrote:
With the firewall up and down at B, can you give a traceroute from A to B.
[quote]
Firewall Up
traceroute to site.b.public.dns (site.b.public.ip), 30 hops max, 60 byte packets
1 96.120.41.241 (96.120.41.241) 7.892 ms 13.053 ms 13.961 ms
2 68.85.218.57 (68.85.218.57) 14.123 ms 14.120 ms 14.263 ms
3 96.110.18.6 (96.110.18.6) 13.697 ms 13.654 ms 13.626 ms
4 96.110.18.21 (96.110.18.21) 13.599 ms * 13.732 ms
5 be-33668-cr02.350ecermak.il.ibone.comcast.net (68.86.90.45) 22.149 ms 21.079 ms 22.072 ms
6 be-10563-pe01.350ecermak.il.ibone.comcast.net (68.86.82.158) 18.847 ms 15.289 ms 15.782 ms
7 as7843.350ecermak.il.ibone.comcast.net (75.149.230.70) 21.440 ms 21.439 ms 20.210 ms
8 bu-ether39.chcgildt87w-bcr00.tbone.rr.com (66.109.1.67) 26.457 ms 23.694 ms 107.14.17.197 (107.14.17.197) 21.925 ms
9 24.27.236.1 (24.27.236.1) 30.761 ms 31.669 ms 31.735 ms
10 72-31-205-12.net.bhntampa.com (72.31.205.12) 27.067 ms bun103.detr01-car2.bhn.net (72.31.205.106) 27.138 ms 72-31-205-12.net.bhntampa.com (72.31.205.12) 27.009 ms
11 97-69-182-11.res.bhn.net (97.69.182.11) 27.815 ms 97-69-182-9.res.bhn.net (97.69.182.9) 27.880 ms ten0-0-0-2.detr01-ser3.bhn.net (72.31.205.171) 27.605 ms
12 72-31-205-61.net.bhntampa.com (72.31.205.61) 27.543 ms 72-31-205-77.net.bhntampa.com (72.31.205.77) 25.516 ms 72-31-205-79.net.bhntampa.com (72.31.205.79) 26.551 ms
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
Firewall Down
traceroute to site.b.public.dns (site.b.public.ip), 30 hops max, 60 byte packets
1 96.120.41.241 (96.120.41.241) 15.845 ms 21.566 ms 22.463 ms
2 68.85.218.57 (68.85.218.57) 22.533 ms 22.480 ms 22.690 ms
3 96.110.18.6 (96.110.18.6) 22.161 ms 22.419 ms 22.384 ms
4 96.110.18.21 (96.110.18.21) 22.099 ms 22.045 ms 22.012 ms
5 be-33668-cr02.350ecermak.il.ibone.comcast.net (68.86.90.45) 28.147 ms 29.160 ms 29.880 ms
6 be-10563-pe01.350ecermak.il.ibone.comcast.net (68.86.82.158) 26.674 ms 15.276 ms 18.168 ms
7 as7843.350ecermak.il.ibone.comcast.net (75.149.230.70) 17.074 ms 17.842 ms 43.411 ms
8 bu-ether29.chcgildt87w-bcr00.tbone.rr.com (107.14.17.195) 23.538 ms 107.14.17.197 (107.14.17.197) 26.502 ms bu-ether29.chcgildt87w-bcr00.tbone.rr.com (107.14.17.195) 25.418 ms
9 24.27.236.1 (24.27.236.1) 28.771 ms 27.593 ms 28.479 ms
10 bun103.detr01-car2.bhn.net (72.31.205.106) 28.617 ms * 28.393 ms
11 ten0-0-0-0.detr02-ser4.bhn.net (72.31.204.175) 28.387 ms ten0-0-0-2.detr02-ser4.bhn.net (72.31.205.175) 28.350 ms ten0-0-0-0.detr02-ser4.bhn.net (72.31.204.175) 28.302 ms
12 72-31-205-63.net.bhntampa.com (72.31.205.63) 22.259 ms 72-31-204-215.net.bhntampa.com (72.31.204.215) 27.634 ms 72-31-204-241.net.bhntampa.com (72.31.204.241) 26.501 ms
13 site.b.isp.dns (site.b.public.ip) 40.153 ms 32.505 ms 32.426 ms
So the firewall is successfully stopping the traceroute it looks to me.
[quote]Nick Howitt wrote:
Thinking about this some more, is this a DNS issue? Are you trying to access machines by FQDN when you get the issue? Can you try pinging the remote host by IP and FQDN? Is pinging by FQDN pinging your remote external IP?
If this is the case, in Site A you need a DNS entry for the machines in Site B to go directly to the LAN IP. I am not sure how this work with port forwards where you only forward certain IP's to LAN machines and it can be different IP's. I have a feeling it will fail. Try setting the DNS entry for machines you for which have port forwarding rules to point to the remote ClearOS LAN IP.
Does not seem to be DNS related. If I remove DNS from the equation completely (both on the public side and internally) I still get the same results. -
Accepted Answer
Thinking about this some more, is this a DNS issue? Are you trying to access machines by FQDN when you get the issue? Can you try pinging the remote host by IP and FQDN? Is pinging by FQDN pinging your remote external IP?
If this is the case, in Site A you need a DNS entry for the machines in Site B to go directly to the LAN IP. I am not sure how this work with port forwards where you only forward certain IP's to LAN machines and it can be different IP's. I have a feeling it will fail. Try setting the DNS entry for machines you for which have port forwarding rules to point to the remote ClearOS LAN IP. -
Accepted Answer
Can we start again with the troubleshooting as things may have changed over 2 years. Please can you give the conn definitions from either end from /etc/ipsec.d.*.conf and the connection log from both ends.
With the firewall up and down at B, can you give a traceroute from A to B.
BTW, from earlier in your post a tcpdump should not see any pings on your external interfaces if they should be going through the tunnel. You will see traffic but not unencrypted. -
Accepted Answer
After 2 years (!?) I've finally had some time to sit down and diagnose this some more and am still having the same issues. That said, I HAVE nailed this down to being a rule in the side B firewall. If I stop the firewall service with
I instantly get access to everything on both sides. Obviously this also brings down my router's function as an internet gateway at the same time and leaves a pretty big security hole, so not a good viable option.systemctl stop firewall
I'm ready to take a look deeper now that I at least have an idea of where the problem is. I've spent quite some time today trying to compare iptables outputs between both sides and do not see anything obvious, but turning off the firewall on one side makes it all work so I know it is hiding in there somewhere (maybe even a rule order)? If there are specific tests that can help nail this down with tcpdump, happy to run them as well. Any help is appreciated. Below is the output of the Site B firewall config as of today:
[root@router ~]# iptables -L -nv
Chain INPUT (policy DROP 42 packets, 2189 bytes)
pkts bytes target prot opt in out source destination
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 match-set f2b-sshd-ddos src reject-with icmp-port-unreachable
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 match-set f2b-sshd src reject-with icmp-port-unreachable
0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 25,465,587,220,993,110,995 match-set f2b-postfix-sasl src reject-with icmp-port-unreachable
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_INGRESS src
7 532 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:123 state RELATED,ESTABLISHED
7 352 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID
0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:0x12/0x12 state NEW reject-with tcp-reset
0 0 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:!0x17/0x02 state NEW
0 0 DROP all -- eno1 * 127.0.0.0/8 0.0.0.0/0
0 0 DROP all -- eno1 * 169.254.0.0/16 0.0.0.0/0
576 66060 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
505 67867 ACCEPT all -- eno2 * 0.0.0.0/0 0.0.0.0/0
27 1553 ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 0
0 0 ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 3
2 168 ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 8
0 0 ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 11
44 16705 ACCEPT udp -- eno1 * 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
0 0 ACCEPT tcp -- eno1 * 0.0.0.0/0 0.0.0.0/0 tcp spt:67 dpt:68
0 0 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpt:4500
0 0 ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:1194
0 0 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpts:5060:5061
0 0 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpt:1194
252 18159 ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:22
0 0 ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:81
1 120 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp spt:500 dpt:500
356 53168 ACCEPT esp -- * * 0.0.0.0/0 my.public.ip
0 0 ACCEPT ah -- * * 0.0.0.0/0 my.public.ip
0 0 ACCEPT all -- * * 0.0.0.0/0 my.public.ip mark match 0x64
7 501 ACCEPT all -- * * 0.0.0.0/0 192.168.10.1 mark match 0x64
362 51306 ACCEPT udp -- eno1 * 0.0.0.0/0 0.0.0.0/0 udp dpts:1024:65535 state RELATED,ESTABLISHED
1127 945K ACCEPT tcp -- eno1 * 0.0.0.0/0 0.0.0.0/0 tcp dpts:1024:65535 state RELATED,ESTABLISHED
Chain FORWARD (policy DROP 304 packets, 24468 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_SELF src,dst,dst
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_EGRESS dst
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_INGRESS src
6 523 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x64
613 120K ACCEPT tcp -- * eno2 0.0.0.0/0 192.168.10.73 tcp dpt:443
0 0 ACCEPT tcp -- * eno2 0.0.0.0/0 192.168.10.54 tcp dpt:8800
1102 226K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
107 7144 ACCEPT all -- eno2 * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_SELF src,dst,dst
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_EGRESS dst
576 66060 ACCEPT all -- * lo 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * pptp+ 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
432 53262 ACCEPT all -- * eno2 0.0.0.0/0 0.0.0.0/0
31 2009 ACCEPT icmp -- * eno1 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT udp -- * eno1 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67
0 0 ACCEPT tcp -- * eno1 0.0.0.0/0 0.0.0.0/0 tcp spt:68 dpt:67
0 0 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spt:4500
0 0 ACCEPT tcp -- * eno1 my.public.ip 0.0.0.0/0 tcp spt:1194
0 0 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spts:5060:5061
0 0 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spt:1194
148 24577 ACCEPT tcp -- * eno1 my.public.ip 0.0.0.0/0 tcp spt:22
0 0 ACCEPT tcp -- * eno1 my.public.ip 0.0.0.0/0 tcp spt:81
1 120 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spt:500 dpt:500
10 1568 ACCEPT esp -- * eno1 my.public.ip 0.0.0.0/0
0 0 ACCEPT ah -- * eno1 my.public.ip 0.0.0.0/0
1582 121K ACCEPT all -- * eno1 0.0.0.0/0 0.0.0.0/0
Chain DROP-lan (0 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
-
Accepted Answer
I am puzzled. Can you reinstate your leftsourceip (local LAN interface IP) at both ends then post a connection log of the same connection from both sides (but nothing before "loading secrets from "/etc/ipsec.d/ipsec.unmanaged.pvt.secrets")? Then try pinging from server to server (use the remote server LAN IP). If that works, then from server to remote LAN, local LAN to remote server's LAN IP, then LAN <-> LAN. Does anything work?
I am also curious about your next hops. What do you mean by my.public.router? Are you behind a router? In general you don't need to set the nexthops, but if your connection negotiates then it should not matter either way. -
Accepted Answer
Nick Howitt wrote:
From the logs I am not interested in anything down to "loading ipsec.secrets". That is just Libreswan (not Openswan) start up stuff. It is the tunnel negotiation after that where the interesting (or not) stuff happens. Your logs don't match as they are a few minutes out and both logs show Libreswan initiating and not responding. I could do with the response log, say from site B, at around the time you restarted IPsec at site A. I only have a responding set up so I can't necessarily compare, but your logs are missing what I would expect but that could only be a responding thing.
Side A
Jul 7 15:58:06 router pluto[9567]: "pvt" #1: initiating Main Mode
Jul 7 15:58:06 router pluto[9567]: "pvt" #1: ERROR: asynchronous network error report on enp2s0 (sport=500) for message to my.public.ip port 500, complainant my.public.ip : Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated)]
Jul 7 15:58:07 router pluto[9567]: "pvt" #1: ERROR: asynchronous network error report on enp2s0 (sport=500) for message to my.public.ip port 500, complainant my.public.ip : Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated)]
Jul 7 15:58:07 router pluto[9567]: "pvt" #1: ERROR: asynchronous network error report on enp2s0 (sport=500) for message to my.public.ip port 500, complainant my.public.ip : Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated)]
Jul 7 15:58:08 router pluto[9567]: "pvt" #1: ERROR: asynchronous network error report on enp2s0 (sport=500) for message to my.public.ip port 500, complainant my.public.ip : Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated)]
Jul 7 15:58:09 router pluto[9567]: assign_holdpass() delete_bare_shunt() failed
Jul 7 15:58:09 router pluto[9567]: initiate_ondemand_body() failed to install negotiation_shunt,
Jul 7 15:58:09 router pluto[9567]: initiate on demand from 192.168.3.20:64926 to 192.168.10.15:3283 proto=6 state: fos_start because: acquire
Jul 7 15:58:10 router pluto[9567]: "pvt" #1: ERROR: asynchronous network error report on enp2s0 (sport=500) for message to my.public.ip port 500, complainant my.public.ip : Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated)]
Jul 7 15:58:11 router pluto[9567]: assign_holdpass() delete_bare_shunt() failed
Jul 7 15:58:11 router pluto[9567]: initiate_ondemand_body() failed to install negotiation_shunt,
Jul 7 15:58:11 router pluto[9567]: initiate on demand from 192.168.3.20:64989 to 192.168.10.16:3283 proto=6 state: fos_start because: acquire
Jul 7 15:58:14 router pluto[9567]: "pvt" #1: ERROR: asynchronous network error report on enp2s0 (sport=500) for message to my.public.ip port 500, complainant my.public.ip : Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated)]
Jul 7 15:58:17 router pluto[9567]: packet from my.public.ip :500: received Vendor ID payload [Dead Peer Detection]
Jul 7 15:58:17 router pluto[9567]: packet from my.public.ip :500: received Vendor ID payload [FRAGMENTATION]
Jul 7 15:58:17 router pluto[9567]: packet from my.public.ip :500: received Vendor ID payload [RFC 3947]
Jul 7 15:58:17 router pluto[9567]: packet from my.public.ip :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-03]
Jul 7 15:58:17 router pluto[9567]: packet from my.public.ip :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n]
Jul 7 15:58:17 router pluto[9567]: packet from my.public.ip :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02]
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: responding to Main Mode
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: transition from state STATE_MAIN_R0 to state STATE_MAIN_R1
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: STATE_MAIN_R1: sent MR1, expecting MI2
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: transition from state STATE_MAIN_R1 to state STATE_MAIN_R2
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: STATE_MAIN_R2: sent MR2, expecting MI3
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: Main mode peer ID is ID_IPV4_ADDR: 'my.public.ip '
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: transition from state STATE_MAIN_R2 to state STATE_MAIN_R3
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: STATE_MAIN_R3: sent MR3, ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
Jul 7 15:58:17 router pluto[9567]: "pvt" #2: the peer proposed: 192.168.3.0/24:0/0 -> 192.168.10.0/24:0/0
Jul 7 15:58:17 router pluto[9567]: "pvt" #3: responding to Quick Mode proposal {msgid:108bc663}
Jul 7 15:58:17 router pluto[9567]: "pvt" #3: us: 192.168.3.0/24===my.remote.ip <my.remote.ip >
Jul 7 15:58:17 router pluto[9567]: "pvt" #3: them: my.public.ip <my.remote.dns.name >===192.168.10.0/24
Jul 7 15:58:17 router pluto[9567]: "pvt" #3: transition from state STATE_QUICK_R0 to state STATE_QUICK_R1
Jul 7 15:58:17 router pluto[9567]: "pvt" #3: STATE_QUICK_R1: sent QR1, inbound IPsec SA installed, expecting QI2 tunnel mode {ESP=>0x75618966 <0x4da2377d xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
Jul 7 15:58:17 router pluto[9567]: "pvt" #3: transition from state STATE_QUICK_R1 to state STATE_QUICK_R2
Jul 7 15:58:17 router pluto[9567]: "pvt" #3: STATE_QUICK_R2: IPsec SA established tunnel mode {ESP=>0x75618966 <0x4da2377d xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: received Vendor ID payload [Dead Peer Detection]
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: received Vendor ID payload [FRAGMENTATION]
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: received Vendor ID payload [RFC 3947]
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: STATE_MAIN_I2: sent MI2, expecting MR2
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: STATE_MAIN_I3: sent MI3, expecting MR3
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: received Vendor ID payload [CAN-IKEv2]
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: Main mode peer ID is ID_IPV4_ADDR: 'my.public.ip '
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
Jul 7 15:58:22 router pluto[9567]: "pvt" #1: STATE_MAIN_I4: ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
Jul 7 15:58:22 router pluto[9567]: "pvt" #4: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEV1_ALLOW+IKEV2_ALLOW+SAREF_TRACK+IKE_FRAG_ALLOW {using isakmp#1 msgid:d4c5292b proposal=defaults pfsgroup=OAKLEY_GROUP_MODP2048}
Jul 7 15:58:22 router pluto[9567]: "pvt" #4: transition from state STATE_QUICK_I1 to state STATE_QUICK_I2
Jul 7 15:58:22 router pluto[9567]: "pvt" #4: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP=>0x366a736a <0x298c262e xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
Side B
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: initiating Main Mode
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: received Vendor ID payload [Dead Peer Detection]
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: received Vendor ID payload [FRAGMENTATION]
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: received Vendor ID payload [RFC 3947]
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: STATE_MAIN_I2: sent MI2, expecting MR2
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: STATE_MAIN_I3: sent MI3, expecting MR3
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: received Vendor ID payload [CAN-IKEv2]
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: Main mode peer ID is ID_IPV4_ADDR: 'my.remote.ip '
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
Jul 7 15:58:17 router pluto[32163]: "pvt" #1: STATE_MAIN_I4: ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
Jul 7 15:58:17 router pluto[32163]: "pvt" #2: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEV1_ALLOW+IKEV2_ALLOW+SAREF_TRACK+IKE_FRAG_ALLOW {using isakmp#1 msgid:108bc663 proposal=defaults pfsgroup=OAKLEY_GROUP_MODP2048}
Jul 7 15:58:17 router pluto[32163]: "pvt" #2: transition from state STATE_QUICK_I1 to state STATE_QUICK_I2
Jul 7 15:58:17 router pluto[32163]: "pvt" #2: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP=>0x4da2377d <0x75618966 xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
Jul 7 15:58:22 router pluto[32163]: packet from my.remote.ip :500: received Vendor ID payload [Dead Peer Detection]
Jul 7 15:58:22 router pluto[32163]: packet from my.remote.ip :500: received Vendor ID payload [FRAGMENTATION]
Jul 7 15:58:22 router pluto[32163]: packet from my.remote.ip :500: received Vendor ID payload [RFC 3947]
Jul 7 15:58:22 router pluto[32163]: packet from my.remote.ip :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-03]
Jul 7 15:58:22 router pluto[32163]: packet from my.remote.ip :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n]
Jul 7 15:58:22 router pluto[32163]: packet from my.remote.ip :500: ignoring Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02]
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: responding to Main Mode
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: transition from state STATE_MAIN_R0 to state STATE_MAIN_R1
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: STATE_MAIN_R1: sent MR1, expecting MI2
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: transition from state STATE_MAIN_R1 to state STATE_MAIN_R2
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: STATE_MAIN_R2: sent MR2, expecting MI3
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: Main mode peer ID is ID_IPV4_ADDR: 'my.remote.ip '
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: transition from state STATE_MAIN_R2 to state STATE_MAIN_R3
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: STATE_MAIN_R3: sent MR3, ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
Jul 7 15:58:22 router pluto[32163]: "pvt" #3: the peer proposed: 192.168.10.0/24:0/0 -> 192.168.3.0/24:0/0
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: responding to Quick Mode proposal {msgid:d4c5292b}
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: us: 192.168.10.0/24===my.public.ip <my.public.ip >
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: them: my.remote.ip <my.remote.dns.name >===192.168.3.0/24
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: keeping refhim=4294901761 during rekey
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: transition from state STATE_QUICK_R0 to state STATE_QUICK_R1
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: STATE_QUICK_R1: sent QR1, inbound IPsec SA installed, expecting QI2 tunnel mode {ESP=>0x298c262e <0x366a736a xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: transition from state STATE_QUICK_R1 to state STATE_QUICK_R2
Jul 7 15:58:22 router pluto[32163]: "pvt" #4: STATE_QUICK_R2: IPsec SA established tunnel mode {ESP=>0x298c262e <0x366a736a xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
In your conn, have you defined the subnets you want to link? I suspect not. Can you post the contents of any conf file in /etc/ipsec.d.
Yes, I have defined this. Here are the configs from both sides:
Side A
conn pvt
type=tunnel
authby=secret
auto=start
left=my.public.ip
leftnexthop=my.public.router
leftsourceip=192.168.3.1
leftsubnet=192.168.3.0/24
right=my.remote.public.ip
rightsubnet=192.168.10.0/24
Side B
conn pvt
type=tunnel
authby=secret
auto=start
left=my.public.ip
leftnexthop=my.public.router
leftsourceip=192.168.10.1
leftsubnet=192.168.10.0/24
right=my.remote.public.ip
rightsubnet=192.168.3.0/24
Nick Howitt wrote:
I think you are using one of the ClearOS packages because your log file is being redirected. Can you clarify which one? Is it the free Static IPsec VPN for home?
I'm using the ClearOS Static IPSEC VPN for Home package.
Nick Howitt wrote:
[edit]
BTW, if you do a tcpdump on the external interface you should only see encrypted packets.
[/edit]
[edit2]
Also, if not set, please set your Local LAN IP to your ClearOS LAN IP. It helps server to server communication. There is no need to set the Remote LAN IP. This does nothing.
[/edit2]
I've removed the Local LAN IP directives on both sides and still don't have a working two way tunnel. -
Accepted Answer
From the logs I am not interested in anything down to "loading ipsec.secrets". That is just Libreswan (not Openswan) start up stuff. It is the tunnel negotiation after that where the interesting (or not) stuff happens. Your logs don't match as they are a few minutes out and both logs show Libreswan initiating and not responding. I could do with the response log, say from site B, at around the time you restarted IPsec at site A. I only have a responding set up so I can't necessarily compare, but your logs are missing what I would expect but that could only be a responding thing.
In your conn, have you defined the subnets you want to link? I suspect not. Can you post the contents of any conf file in /etc/ipsec.d.
I think you are using one of the ClearOS packages because your log file is being redirected. Can you clarify which one? Is it the free Static IPsec VPN for home?
[edit]
BTW, if you do a tcpdump on the external interface you should only see encrypted packets.
[/edit]
[edit2]
Also, if not set, please set your Local LAN IP to your ClearOS LAN IP. It helps server to server communication. There is no need to set the Remote LAN IP. This does nothing.
[/edit2] -
Accepted Answer
Additionally, here are the startup logs for both sites /var/log/ipsec
Side A
Jun 15 20:45:30 router pluto[2832]: Starting Pluto (Libreswan Version 3.15 XFRM(netkey) KLIPS NSS DNSSEC FIPS_CHECK LABELED_IPSEC LIBCAP_NG LINUX_AUDIT XAUTH_PAM NETWORKMANAGER CURL(non-NSS) LDAP(non-NSS)) pid:2832
Jun 15 20:45:30 router pluto[2832]: core dump dir: /var/run/pluto/
Jun 15 20:45:30 router pluto[2832]: secrets file: /etc/ipsec.secrets
Jun 15 20:45:30 router pluto[2832]: leak-detective disabled
Jun 15 20:45:30 router pluto[2832]: NSS crypto [enabled]
Jun 15 20:45:30 router pluto[2832]: XAUTH PAM support [enabled]
Jun 15 20:45:30 router pluto[2832]: NAT-Traversal support [enabled]
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_TWOFISH_CBC_SSH: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_TWOFISH_CBC: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_SERPENT_CBC: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_AES_CBC: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_AES_CTR: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_AES_GCM_A: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_AES_GCM_B: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_AES_GCM_C: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_hash(): Activating DISABLED-OAKLEY_AES_XCBC: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_CAMELLIA_CBC: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating OAKLEY_CAMELLIA_CTR: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_hash(): Activating OAKLEY_SHA2_512: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_hash(): Activating OAKLEY_SHA2_384: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_hash(): Activating OAKLEY_SHA2_256: Ok
Jun 15 20:45:30 router pluto[2832]: starting up 3 crypto helpers
Jun 15 20:45:30 router pluto[2832]: started thread for crypto helper 0 (master fd 10)
Jun 15 20:45:30 router pluto[2832]: started thread for crypto helper 1 (master fd 13)
Jun 15 20:45:30 router pluto[2832]: started thread for crypto helper 2 (master fd 15)
Jun 15 20:45:30 router pluto[2832]: Using Linux XFRM/NETKEY IPsec interface code on 3.10.0-327.10.1.v7.x86_64
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating aes_ccm_8: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating aes_ccm_12: Ok
Jun 15 20:45:30 router pluto[2832]: ike_alg_register_enc(): Activating aes_ccm_16: Ok
Jun 15 20:45:30 router pluto[2832]: | selinux support is NOT enabled.
Jun 15 20:45:31 router pluto[2832]: | certificate not loaded for this end
Jun 15 20:45:31 router pluto[2832]: | certificate not loaded for this end
Jun 15 20:45:31 router pluto[2832]: added connection description "pvt"
Jun 15 20:45:31 router pluto[2832]: | certificate not loaded for this end
Jun 15 20:45:31 router pluto[2832]: | certificate not loaded for this end
Jun 15 20:45:31 router pluto[2832]: added connection description "v6neighbor-hole-in"
Jun 15 20:45:31 router pluto[2832]: | certificate not loaded for this end
Jun 15 20:45:31 router pluto[2832]: | certificate not loaded for this end
Jun 15 20:45:31 router pluto[2832]: added connection description "v6neighbor-hole-out"
Jun 15 20:45:31 router pluto[2832]: listening for IKE messages
Jun 15 20:45:31 router pluto[2832]: adding interface tun1/tun1 10.8.0.1:500
Jun 15 20:45:31 router pluto[2832]: adding interface tun1/tun1 10.8.0.1:4500
Jun 15 20:45:31 router pluto[2832]: adding interface tun0/tun0 10.8.10.1:500
Jun 15 20:45:31 router pluto[2832]: adding interface tun0/tun0 10.8.10.1:4500
Jun 15 20:45:31 router pluto[2832]: adding interface enp3s0/enp3s0 192.168.3.1:500
Jun 15 20:45:31 router pluto[2832]: adding interface enp3s0/enp3s0 192.168.3.1:4500
Jun 15 20:45:31 router pluto[2832]: adding interface enp2s0/enp2s0 my.public.ipA:500
Jun 15 20:45:31 router pluto[2832]: adding interface enp2s0/enp2s0 my.public.ipA:4500
Jun 15 20:45:31 router pluto[2832]: adding interface lo/lo 127.0.0.1:500
Jun 15 20:45:31 router pluto[2832]: adding interface lo/lo 127.0.0.1:4500
Jun 15 20:45:31 router pluto[2832]: adding interface lo/lo ::1:500
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface lo:500 fd 32
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface lo:4500 fd 31
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface lo:500 fd 30
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface enp2s0:4500 fd 29
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface enp2s0:500 fd 28
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface enp3s0:4500 fd 27
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface enp3s0:500 fd 26
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface tun0:4500 fd 25
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface tun0:500 fd 24
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface tun1:4500 fd 23
Jun 15 20:45:31 router pluto[2832]: | setup callback for interface tun1:500 fd 22
Jun 15 20:45:31 router pluto[2832]: loading secrets from "/etc/ipsec.secrets"
Jun 15 20:45:31 router pluto[2832]: loading secrets from "/etc/ipsec.d/ipsec.unmanaged.pvt.secrets"
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: initiating Main Mode
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: received Vendor ID payload [Dead Peer Detection]
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: received Vendor ID payload [FRAGMENTATION]
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: received Vendor ID payload [RFC 3947]
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: STATE_MAIN_I2: sent MI2, expecting MR2
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: STATE_MAIN_I3: sent MI3, expecting MR3
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: received Vendor ID payload [CAN-IKEv2]
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: Main mode peer ID is ID_IPV4_ADDR: 'my.public.ipB'
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
Jun 15 20:45:31 router pluto[2832]: "pvt" #1: STATE_MAIN_I4: ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
Jun 15 20:45:31 router pluto[2832]: "pvt" #2: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEV1_ALLOW+IKEV2_ALLOW+SAREF_TRACK+IKE_FRAG_ALLOW {using isakmp#1 msgid:5f82639d proposal=defaults pfsgroup=OAKLEY_GROUP_MODP2048}
Jun 15 20:45:31 router pluto[2832]: "pvt" #2: transition from state STATE_QUICK_I1 to state STATE_QUICK_I2
Jun 15 20:45:31 router pluto[2832]: "pvt" #2: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP=>0x9a5f902b <0x06a53b87 xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
Side B
Jun 15 20:41:19 router pluto[12847]: Starting Pluto (Libreswan Version 3.15 XFRM(netkey) KLIPS NSS DNSSEC FIPS_CHECK LABELED_IPSEC LIBCAP_NG LINUX_AUDIT XAUTH_PAM NETWORKMANAGER CURL(non-NSS) LDAP(non-NSS)) pid:12847
Jun 15 20:41:19 router pluto[12847]: core dump dir: /var/run/pluto/
Jun 15 20:41:19 router pluto[12847]: secrets file: /etc/ipsec.secrets
Jun 15 20:41:19 router pluto[12847]: leak-detective disabled
Jun 15 20:41:19 router pluto[12847]: NSS crypto [enabled]
Jun 15 20:41:19 router pluto[12847]: XAUTH PAM support [enabled]
Jun 15 20:41:19 router pluto[12847]: NAT-Traversal support [enabled]
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_TWOFISH_CBC_SSH: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_TWOFISH_CBC: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_SERPENT_CBC: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_AES_CBC: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_AES_CTR: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_AES_GCM_A: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_AES_GCM_B: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_AES_GCM_C: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_hash(): Activating DISABLED-OAKLEY_AES_XCBC: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_CAMELLIA_CBC: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating OAKLEY_CAMELLIA_CTR: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_hash(): Activating OAKLEY_SHA2_512: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_hash(): Activating OAKLEY_SHA2_384: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_hash(): Activating OAKLEY_SHA2_256: Ok
Jun 15 20:41:19 router pluto[12847]: starting up 7 crypto helpers
Jun 15 20:41:19 router pluto[12847]: started thread for crypto helper 0 (master fd 10)
Jun 15 20:41:19 router pluto[12847]: started thread for crypto helper 1 (master fd 13)
Jun 15 20:41:19 router pluto[12847]: started thread for crypto helper 2 (master fd 15)
Jun 15 20:41:19 router pluto[12847]: started thread for crypto helper 3 (master fd 17)
Jun 15 20:41:19 router pluto[12847]: started thread for crypto helper 4 (master fd 19)
Jun 15 20:41:19 router pluto[12847]: started thread for crypto helper 5 (master fd 21)
Jun 15 20:41:19 router pluto[12847]: started thread for crypto helper 6 (master fd 23)
Jun 15 20:41:19 router pluto[12847]: Using Linux XFRM/NETKEY IPsec interface code on 3.10.0-327.3.1.el7.x86_64
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating aes_ccm_8: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating aes_ccm_12: Ok
Jun 15 20:41:19 router pluto[12847]: ike_alg_register_enc(): Activating aes_ccm_16: Ok
Jun 15 20:41:19 router pluto[12847]: | selinux support is NOT enabled.
Jun 15 20:41:20 router pluto[12847]: | certificate not loaded for this end
Jun 15 20:41:20 router pluto[12847]: | certificate not loaded for this end
Jun 15 20:41:20 router pluto[12847]: added connection description "pvt"
Jun 15 20:41:20 router pluto[12847]: | certificate not loaded for this end
Jun 15 20:41:20 router pluto[12847]: | certificate not loaded for this end
Jun 15 20:41:20 router pluto[12847]: added connection description "v6neighbor-hole-in"
Jun 15 20:41:20 router pluto[12847]: | certificate not loaded for this end
Jun 15 20:41:20 router pluto[12847]: | certificate not loaded for this end
Jun 15 20:41:20 router pluto[12847]: added connection description "v6neighbor-hole-out"
Jun 15 20:41:20 router pluto[12847]: listening for IKE messages
Jun 15 20:41:20 router pluto[12847]: adding interface tun1/tun1 10.8.0.1:500
Jun 15 20:41:20 router pluto[12847]: adding interface tun1/tun1 10.8.0.1:4500
Jun 15 20:41:20 router pluto[12847]: adding interface tun0/tun0 10.8.10.1:500
Jun 15 20:41:20 router pluto[12847]: adding interface tun0/tun0 10.8.10.1:4500
Jun 15 20:41:20 router pluto[12847]: adding interface eno2/eno2 192.168.10.1:500
Jun 15 20:41:20 router pluto[12847]: adding interface eno2/eno2 192.168.10.1:4500
Jun 15 20:41:20 router pluto[12847]: adding interface eno1/eno1 my.public.ipB:500
Jun 15 20:41:20 router pluto[12847]: adding interface eno1/eno1 my.public.ipB:4500
Jun 15 20:41:20 router pluto[12847]: adding interface lo/lo 127.0.0.1:500
Jun 15 20:41:20 router pluto[12847]: adding interface lo/lo 127.0.0.1:4500
Jun 15 20:41:20 router pluto[12847]: adding interface lo/lo ::1:500
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface lo:500 fd 40
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface lo:4500 fd 39
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface lo:500 fd 38
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface eno1:4500 fd 37
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface eno1:500 fd 36
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface eno2:4500 fd 35
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface eno2:500 fd 34
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface tun0:4500 fd 33
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface tun0:500 fd 32
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface tun1:4500 fd 31
Jun 15 20:41:20 router pluto[12847]: | setup callback for interface tun1:500 fd 30
Jun 15 20:41:20 router pluto[12847]: loading secrets from "/etc/ipsec.secrets"
Jun 15 20:41:20 router pluto[12847]: loading secrets from "/etc/ipsec.d/ipsec.unmanaged.pvt.secrets"
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: initiating Main Mode
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: received Vendor ID payload [Dead Peer Detection]
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: received Vendor ID payload [FRAGMENTATION]
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: received Vendor ID payload [RFC 3947]
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: STATE_MAIN_I2: sent MI2, expecting MR2
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: STATE_MAIN_I3: sent MI3, expecting MR3
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: received Vendor ID payload [CAN-IKEv2]
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: Main mode peer ID is ID_IPV4_ADDR: 'my.public.ipA'
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
Jun 15 20:41:20 router pluto[12847]: "pvt" #1: STATE_MAIN_I4: ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
Jun 15 20:41:20 router pluto[12847]: "pvt" #2: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEV1_ALLOW+IKEV2_ALLOW+SAREF_TRACK+IKE_FRAG_ALLOW {using isakmp#1 msgid:2b5c61a8 proposal=defaults pfsgroup=OAKLEY_GROUP_MODP2048}
Jun 15 20:41:20 router pluto[12847]: "pvt" #2: transition from state STATE_QUICK_I1 to state STATE_QUICK_I2
Jun 15 20:41:20 router pluto[12847]: "pvt" #2: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP=>0x20f58768 <0x1f315deb xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}
-
Accepted Answer
Here is the dumps from side A
[root@router ~]# iptables -nvL
Chain INPUT (policy DROP 3443 packets, 484K bytes)
pkts bytes target prot opt in out source destination
758 57608 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:123 state RELATED,ESTABLISHED
1519 71689 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID
0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:0x12/0x12 state NEW reject-with tcp-reset
1171 71943 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:!0x17/0x02 state NEW
0 0 DROP all -- enp2s0 * 127.0.0.0/8 0.0.0.0/0
0 0 DROP all -- enp2s0 * 169.254.0.0/16 0.0.0.0/0
179K 21M ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
62754 9140K ACCEPT all -- enp3s0 * 0.0.0.0/0 0.0.0.0/0
3407 98913 ACCEPT icmp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 icmptype 0
10 3744 ACCEPT icmp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 icmptype 3
43 1700 ACCEPT icmp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 icmptype 8
0 0 ACCEPT icmp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 icmptype 11
145 48927 ACCEPT udp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
0 0 ACCEPT tcp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 tcp spt:67 dpt:68
7 784 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpt:4500
0 0 ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:1194
0 0 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpt:1194
11756 1399K ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:22
259 37120 ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:81
32 7292 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp spt:500 dpt:500
8632 1228K ACCEPT esp -- * * 0.0.0.0/0 my.public.ip
0 0 ACCEPT ah -- * * 0.0.0.0/0 my.public.ip
0 0 ACCEPT all -- * * 0.0.0.0/0 my.public.ip mark match 0x64
109 9709 ACCEPT all -- * * 0.0.0.0/0 192.168.3.1 mark match 0x64
19271 2503K ACCEPT udp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 udp dpts:1024:65535 state RELATED,ESTABLISHED
12997 31M ACCEPT tcp -- enp2s0 * 0.0.0.0/0 0.0.0.0/0 tcp dpts:1024:65535 state RELATED,ESTABLISHED
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
8521 602K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x64
15M 17G ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
143K 9976K ACCEPT all -- enp3s0 * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
180K 21M ACCEPT all -- * lo 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * pptp+ 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
25122 3694K ACCEPT all -- * enp3s0 0.0.0.0/0 0.0.0.0/0
10161 743K ACCEPT icmp -- * enp2s0 0.0.0.0/0 0.0.0.0/0
2 656 ACCEPT udp -- * enp2s0 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67
0 0 ACCEPT tcp -- * enp2s0 0.0.0.0/0 0.0.0.0/0 tcp spt:68 dpt:67
0 0 ACCEPT udp -- * enp2s0 my.public.ip 0.0.0.0/0 udp spt:4500
0 0 ACCEPT tcp -- * enp2s0 my.public.ip 0.0.0.0/0 tcp spt:1194
0 0 ACCEPT udp -- * enp2s0 my.public.ip 0.0.0.0/0 udp spt:1194
11225 2836K ACCEPT tcp -- * enp2s0 my.public.ip 0.0.0.0/0 tcp spt:22
205 90852 ACCEPT tcp -- * enp2s0 my.public.ip 0.0.0.0/0 tcp spt:81
37 15348 ACCEPT udp -- * enp2s0 my.public.ip 0.0.0.0/0 udp spt:500 dpt:500
12162 1763K ACCEPT esp -- * enp2s0 my.public.ip 0.0.0.0/0
0 0 ACCEPT ah -- * enp2s0 my.public.ip 0.0.0.0/0
46762 3205K ACCEPT all -- * enp2s0 0.0.0.0/0 0.0.0.0/0
Chain DROP-lan (0 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
[root@router ~]# iptables -nvL -t nat
Chain PREROUTING (policy ACCEPT 141K packets, 17M bytes)
pkts bytes target prot opt in out source destination
Chain INPUT (policy ACCEPT 39978 packets, 4710K bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 55148 packets, 3576K bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 28088 packets, 1777K bytes)
pkts bytes target prot opt in out source destination
636 41519 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 policy match dir out pol ipsec
0 0 ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
104K 7738K MASQUERADE all -- * enp2s0 0.0.0.0/0 0.0.0.0/0
Here are the dumps from side B
[root@router ~]# iptables -nvL
Chain INPUT (policy DROP 151K packets, 8854K bytes)
pkts bytes target prot opt in out source destination
476 36176 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:123 state RELATED,ESTABLISHED
6900 315K DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID
0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:0x12/0x12 state NEW reject-with tcp-reset
168 82312 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:!0x17/0x02 state NEW
0 0 DROP all -- eno1 * 127.0.0.0/8 0.0.0.0/0
0 0 DROP all -- eno1 * 169.254.0.0/16 0.0.0.0/0
96209 10M ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
163K 15M ACCEPT all -- eno2 * 0.0.0.0/0 0.0.0.0/0
4191 237K ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 0
2 144 ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 3
221 14796 ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 8
0 0 ACCEPT icmp -- eno1 * 0.0.0.0/0 0.0.0.0/0 icmptype 11
22235 7768K ACCEPT udp -- eno1 * 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
0 0 ACCEPT tcp -- eno1 * 0.0.0.0/0 0.0.0.0/0 tcp spt:67 dpt:68
64 17617 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpt:4500
0 0 ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:1194
41 17723 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpts:5060:5061
0 0 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp dpt:1194
15218 1507K ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:22
1108 804K ACCEPT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:81
35 15108 ACCEPT udp -- * * 0.0.0.0/0 my.public.ip udp spt:500 dpt:500
9191 1324K ACCEPT esp -- * * 0.0.0.0/0 my.public.ip
0 0 ACCEPT ah -- * * 0.0.0.0/0 my.public.ip
0 0 ACCEPT all -- * * 0.0.0.0/0 my.public.ip mark match 0x64
81 5909 ACCEPT all -- * * 0.0.0.0/0 192.168.10.1 mark match 0x64
106K 15M ACCEPT udp -- eno1 * 0.0.0.0/0 0.0.0.0/0 udp dpts:1024:65535 state RELATED,ESTABLISHED
137K 126M ACCEPT tcp -- eno1 * 0.0.0.0/0 0.0.0.0/0 tcp dpts:1024:65535 state RELATED,ESTABLISHED
Chain FORWARD (policy DROP 4121 packets, 259K bytes)
pkts bytes target prot opt in out source destination
5 225 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x64
458K 181M ACCEPT tcp -- * eno2 0.0.0.0/0 192.168.10.73 tcp dpt:443
7546 448K ACCEPT tcp -- * eno2 0.0.0.0/0 192.168.10.37 tcp dpt:80
15139 890K ACCEPT tcp -- * eno2 0.0.0.0/0 192.168.10.20 tcp dpt:22
126 7940 DROP tcp -- * * 192.168.10.0/24 0.0.0.0/0 tcp dpt:9339
33M 26G ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
555K 42M ACCEPT all -- eno2 * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
96317 10M ACCEPT all -- * lo 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * pptp+ 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
114K 14M ACCEPT all -- * eno2 0.0.0.0/0 0.0.0.0/0
19849 2773K ACCEPT icmp -- * eno1 0.0.0.0/0 0.0.0.0/0
4575 1501K ACCEPT udp -- * eno1 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67
0 0 ACCEPT tcp -- * eno1 0.0.0.0/0 0.0.0.0/0 tcp spt:68 dpt:67
13 890 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spt:4500
0 0 ACCEPT tcp -- * eno1 my.public.ip 0.0.0.0/0 tcp spt:1194
1 74 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spts:5060:5061
0 0 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spt:1194
14210 3530K ACCEPT tcp -- * eno1 my.public.ip 0.0.0.0/0 tcp spt:22
1035 367K ACCEPT tcp -- * eno1 my.public.ip 0.0.0.0/0 tcp spt:81
29 6948 ACCEPT udp -- * eno1 my.public.ip 0.0.0.0/0 udp spt:500 dpt:500
6326 860K ACCEPT esp -- * eno1 my.public.ip 0.0.0.0/0
0 0 ACCEPT ah -- * eno1 my.public.ip 0.0.0.0/0
252K 16M ACCEPT all -- * eno1 0.0.0.0/0 0.0.0.0/0
Chain DROP-lan (0 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
[root@router ~]# iptables -nvL -t nat
Chain PREROUTING (policy ACCEPT 755K packets, 96M bytes)
pkts bytes target prot opt in out source destination
17323 1108K DNAT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:443 to:192.168.10.73:443
713 42632 DNAT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:80 to:192.168.10.37:80
15 960 DNAT tcp -- * * 0.0.0.0/0 my.public.ip tcp dpt:22222 to:192.168.10.20:22
Chain INPUT (policy ACCEPT 141K packets, 12M bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 157K packets, 10M bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 39386 packets, 2509K bytes)
pkts bytes target prot opt in out source destination
1031 65951 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 policy match dir out pol ipsec
1917 122K ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
12592 806K SNAT tcp -- * * 192.168.10.0/24 192.168.10.73 tcp dpt:443 to:192.168.10.1
41 2624 SNAT tcp -- * * 192.168.10.0/24 192.168.10.37 tcp dpt:80 to:192.168.10.1
0 0 SNAT tcp -- * * 192.168.10.0/24 192.168.10.20 tcp dpt:22 to:192.168.10.1
471K 37M MASQUERADE all -- * eno1 0.0.0.0/0 0.0.0.0/0
Where my.public.ip is obviously different between site A and B. -
Accepted Answer
Hi Mike,
Thanks for fixing the thread.
Can you dump the firewall with:
And please put the results between code tags (the <> on a piece of paper button at the top of the reply box).iptables -nvL
iptables -nvL -t nat
Why are you saying the tunnel is up? Have you seen "IPsec SA established tunnel mode" messages in /var/log/messages at both ends or are you getting it from one of the IPsec VPN packages? -
Accepted Answer
Thanks Dave.
I do not have any proxy servers running right now, so that is why the port forward issue is a bit perplexing. I feel like there is a firewall issue on one side blocking traffic on the tunnel, but that isn't exactly scientific.
Given the issue, I'm not entirely sure which tcpdump path I should take. I ran ping tests and pings to hosts behind the tunnel at site B are actually received by the local router.
Some site info:
Site A - subnet 192.168.3.0/24
Site B - subnet 192.168.10.0/24
Here is what I did so far:
Fired up tcpdump -i <interface> icmp[icmptype]=icmp-echo on all four of my ClearOS interfaces (internal and external on both systems)
Pinged from a client behind ClearOS A to a client behind ClearOS B
I see the pings showing up on the appropriate interfaces, but I get timeouts on the ping
Pinged from a client behind ClearOS B to a client behind ClearOS A
I see the exact same tcpdump behavior (in reverse this time for obvious reasons) but the pings actually return.
I'm open for any/all other tests I can run here to determine what might be going on. The tunnel is up and no errors in the logs, so this seems to be firewall related. Again, I'm guessing a bit here. -
Accepted Answer
Do you have a proxy server running and one that you are using that gives you the 80/443 successes because it is actually coming from a different host?
Usually issues where the tunnel is up but things aren't happy is firewall related. TCPdump will help here in diagnosing where packets are dropping. This is especially useful if you run it on both sides of the tunnel.
https://www.clearos.com/resources/documentation/clearos/content:en_us:kb_troubleshooting_connectivity#tcpdump
Please login to post a reply
You will need to be logged in to be able to post a reply. Login using the form on the right or register an account if you are new here.
Register Here »