I have been going though every part of the OS and I can not find away to block ads.
I did find the following in the forum.
but I found that this didnt work. Any other ideaS?
I did find the following in the forum.
logon to server via e.g. putty & run:
Code:
wget -q -O - http://www.mvps.org/winhelp2002/hosts.txt | grep 127.0.0.1 | sed -e 's/127.0.0.1/0.0.0.0/g' -e 's/[[:space:]]*#.*$//' | tr ' ' '\t' | tr -s '\t' | tr -d '\015' | grep '^0\.0\.0\.0' | grep -v localhost >/tmp/dlhosts
edit /etc/dsnmasq/dhcp.conf by adding line:
Code:
addn-hosts=/tmp/dlhosts
restart DNS service:
Code:
service dnsmasq restart
but I found that this didnt work. Any other ideaS?
Share this post:
Accepted Answer
In addition to blocking add sites in the content filter I use poisoning of the DNS on my dnsmasq. You can do hostnames like Sorin recommends but I find that blacklisting WHOLE domains is easily accomplished by directing the DNS lookups for add site domains to bogus network IPs. Since the DNSMasq daemon processes all .conf files in /etc/dnsmasq.d/ simply create a file called:
/etc/dnsmasq.d/poison.conf
In it, create listings similar to this:
server=/.doubleclick.net/10.0.0.0
server=/.pointroll.com/10.0.0.0
Any network address of RFC 1918 will work great. The best to use is the network address of your network. For example, if you use 192.168.4.1/255.255.255.0 for your ClearOS server. Then you can use 192.168.4.0 for the bogus DNS IP. The packet will instantly fail and will not route.
/etc/dnsmasq.d/poison.conf
In it, create listings similar to this:
server=/.doubleclick.net/10.0.0.0
server=/.pointroll.com/10.0.0.0
Any network address of RFC 1918 will work great. The best to use is the network address of your network. For example, if you use 192.168.4.1/255.255.255.0 for your ClearOS server. Then you can use 192.168.4.0 for the bogus DNS IP. The packet will instantly fail and will not route.
Responses (33)
-
Accepted Answer
Hi,
I'm the author of pixelserv-tls. It was brought to my attention another guy is disseminating misinformation i.e. pixelserv-tls is man-in-the-middle attack
pixelserv-tls is not MITM attack, not hurting anyone. In fact, it'll help users in many ways.
People seem to naturally believe MITM when it comes to certificate & its applications. I believe it's due to lack of understanding how PKI (public key infrastructure) works in general and pixelserv-tls specifically.
The Pi-Hole devs seem totally not understand the topic either but couldn't refrain from telling misunderstandings to their relatively large user base.
Not long ago a few people including myself had briefly run through the same topic regarding MITM with the maintainer of adblock script on OpenWRT, who appeared to not understand the issue either back then.
Here is the thread: https://forum.lede-project.org/t/adblock-support-thread/507/381
Hope it provides some useful background to close this FUD or as the foundation for further discussion.
Cheers -
Accepted Answer
If you follow your link you'll see that the changes proposed were adopted in the code in github, so I'd have thought it will probably now compile natively in both 32-bit and 64-bit. The problem with this approach is that it is effectively a min-in-the-middle attack on https which, conceptually, a lot of people are very anti.
I am unlikely to implement anything like this again as it is a pain if it stops a website you want to visit from working. (Some) browser adblockers such as adblock plus and uBlock have a lot of diagnostics built in and, crucially they have a disable button. If you hit a problem on a device with Pixelserv you then have to manipulate it from the command line in ClearOS, and it can be hard to find the critical blocked link. pixelserv-tls looks like it has a logging facility which will help, but I don't think my wife would be too happy if I told her she had to wait until I got home before she could visit a particular site. I could be tempted to investigate PiHole as it seems to have a webconfig of its own and, hopefully, easier management facilities. I think they are also against mitm type blocking(/attacks), and only block http -
Accepted Answer
Hi Nick
I shall try your suggestions tomorrow, but i see that there is a "pixelserv-tls" that does https.
I found the following links:
How-To: build pixelserv-tls for 32-bit x86 in 64-bit amd64 environment
Create and Import the CA Certificate
I am busy reading this for now, but you will probably have it implemented before i am done reading... -
Accepted Answer
That is the https problem I was thinking about. Pixelserv cannot serve a valid certificate for doubleclick.net. Taking a leaf out of PiHole's blog, lets split the blocking into http and https and add a couple of rules for fun:
This is only using pixelserv for normal http traffic. Google appear to be doing some fancy stuff with UDP. My reference for this which I bumped into today is here. You have to follow a link there for an explanation.iptables -t nat -I PREROUTING -p tcp --dport 80 -d 192.168.88.1 -i eth1 -j DNAT --to-destination 192.168.2.1:84
iptables -I FORWARD -i eth1 -d 192.168.88.1 -p tcp --dport 443 -j REJECT --reject-with tcp-reset
iptables -I FORWARD -i eth1 -d 192.168.88.1 -p udp --dport 80 -j REJECT --reject-with icmp-port-unreachable
iptables -I FORWARD -i eth1 -d 192.168.88.1 -p udp --dport 443 -j REJECT --reject-with icmp-port-unreachable
Please try these at the command line first before making them permanent. In ClearOS 7.x, change "iptables" to "$IPTABLES" when applying them to the Custom Firewall. It will probably also work with "$IPTABLES" in 6.x but it is not so important. -
Accepted Answer
Hi Nick
In the ad_poison.conf it is listed as - address=/doubleclick.net/192.168.88.1
I have checked both http:// and https:// and both give me the same "Connection has timed out"
Searching for "Doubleclick.net" i see that it is https://doubleclick.net.
When i then click on the link to go to that page, i get the same "Connection has timed out" -
Accepted Answer
-
Accepted Answer
Hi Nick
ClearOS 7
netstat -npl | grep "84 "
tcp 0 0 192.168.2.1:84 0.0.0.0:* LISTEN 10679/perl
I checked ClearOS process viewer and pixelserv is running as 10679.
Yes, i updated ad-poison.conf to point to 192.168.88.1 then restart dnsmasq
It seems to be working as i don't get any ads on any mobile device apps anymore, this was working even when i pointed to 192.168.2.0
The only part that has not been working it that i do not get the pixelserv page when i check with http://doubleclick.net, i just get a page timeout.
Now this might not be an issue, but it seems that this has introduced a delay in some pages loading -
Accepted Answer
This tells me your ClearOS LAN IP is 192.168.2.1 and LAN interface is eth1 (so ClearOS 6.x?). Can you confirm pixelserv is listening on port 84:netstat -npl | grep "84 "
Also did you change your ad-poison.conf to point to 192.168.88.1 then restart dnsmasq?
As a technique I have no idea how this will work these days with many pages coming from https rather than http. I don't use it (or Dave's method) at all as it blocked a couple of sites from working and it is relatively hard to diagnose and needs a server change to fox which is harder than using an ad blocking extension in a browser. Another option is PiHole which can probably run on ClearOS. I don't know if its blocking lists are more up to date and, if you get a failure of a site, you may get similar issues finding and unblocking the relevant URL.
My suggestion would be to aim for as small a blocklist as possible even if it means letting a few things through, because the more you try to block, the more chances you have of hitting a false positive. -
Accepted Answer
Hi Nick
I have updated it as you have indicated.
iptables -nvL PREROUTING -t nat
Chain PREROUTING (policy ACCEPT 532 packets, 99215 bytes)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- eth1 * 0.0.0.0/0 192.168.88.1 to:192.168.2.1:84
if i do a test with http://doubleclick.net, i get a connection timed out instead of the pixel page -
Accepted Answer
You are mixing and matching methods there, I think. If you want to redirect the traffic to a valid IP you can't use a .0 (in a /24 network) as it is invalid. You'll have to use a real address. Then as the packet comes in, it needs to hit a PREROUTING rule where you redirect it to you correct ClearOS LAN IP but switch the port it is listening on to whatever you've configured pixelserv to listen on.
Using a .0 is Dave's original method. Using non .0 address on another LAN subnet is my modified method incorporating pixelserv. -
Accepted Answer
Hi Nick
address=/doubleclick.net/192.168.2.0 in /etc/dnsmasq.d/ad_poison.conf
iptables -nvL PREROUTING -t nat
Chain PREROUTING (policy ACCEPT 875K packets, 155M bytes)
pkts bytes target prot opt in out source destination -
Accepted Answer
-
Accepted Answer
Hi Nick
Hope you can help here, i followed your instructions Ad Block, i get the "Pixel" page when i type the pageand port address in, but when i do the test with Doubleclick.net i do not see the "pixel" page
Regards
Leon -
Accepted Answer
That is similar to my issues. I found things like the google syndcation and google analytics filters needed to go for some of my page reloads to work. Then my wife complained she could not log onto Freeads and I don't really know how to debug it or can't put the time into it. Perhaps someone needs to add some logging to the pixelserv but I don't know perl at all. -
Accepted Answer
-
Accepted Answer
-
Accepted Answer
-
Accepted Answer
-
Accepted Answer
Hi Nick,
Great posting !!
Downloading the posion list is not working for me.
wget -O -q "http://pgl.yoyo.org/as/serverlist.php?showintro=0;hostformat=dnsmasq" | sed '/^Ad\s/d' |sed 's|</b>|-|g' | sed 's|<[^>]*>||g' | sed '/^$/d' | sed 's|127.0.0.1|10.0.0.1|g' > /etc/dnsmasq.d/ad_poison.conf && service dnsmasq restart
The file is empty.
When using the http i'm getting the complete list, but generating with 1 line is not working. -
Accepted Answer
OK, I could not resist having had a look at the pixelserv server which is only a few lines of code. What I've done is set up pixelserv to listen on an odd port (85), use the dnsmasq poisoning to redirect to 10.0.0.1 and then iptables to redirect 10.0.0.1 to the ClearOS_LAN port 85. This stops pixelserv from interfering with the normal web browser. Here we go, keeping the file locations the same as http://www.bsdnow.tv/tutorials/dnsmasq" target="_blank">www.bsdnow.tv/tutorials/dnsmasq :
With your favourite editor in /usr/local/bin/pixelserv change LocalHost to your ClearOS LAN IP and LocalPort to something non-standard which ClearOS does not use (not 80, 81, 82, 443, 8080 or 3128). I used 85.wget -O /usr/local/bin/pixelserv http://proxytunnel.sourceforge.net/files/pixelserv.pl.txt
chmod 755 /usr/local/bin/pixelserv
Create a file /etc/init.d/pixelserv and in it put:
Note if following the link to the original article I have changed '$1' to "$1". Make the file executable:### BEGIN INIT INFO
# Provides: pixelserv
# Required-Start: $network
# Required-Stop: $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: pixelserv server for ad blocking
# Description: Server for serving 1x1 pixels
### END INIT INFO
case "$1" in
start)
echo 'pixelserv: starting'
/usr/local/bin/pixelserv &
;;
stop)
echo 'pixelserv: stopping'
killall pixelserv
;;
*)
echo 'Usage: service $0 {start|stop}'
exit 1
;;
esac
Start the pixelserv server and set it to auto-start:chmod 555 /etc/init.d/pixelserv
service pixelserv start
chkconfig pixelserv on
Set up the dnsmasq poisoning as I did before:
Add the iptables redirection rule to /etc/clearos/firewall.d/local:wget -O - -q "http://pgl.yoyo.org/as/serverlist.php?showintro=0;hostformat=dnsmasq" | sed '/^Ad\s/d' |sed 's|</b>|-|g' | sed 's|<[^>]*>||g' | sed '/^$/d' | sed 's|127.0.0.1|10.0.0.1|g' > /etc/dnsmasq.d/ad_poison.conf && service dnsmasq restart
Change 172.17.2.1 to your ClearOS LAN IP; the port to whichever port you chose for your pixelserv server and 10.0.0.1 to whichever private IP you redirected your traffic to with the dnsmasq poisoning.# For pixelserv adblocker from http://http://www.bsdnow.tv/tutorials/dnsmasq" target="_blank">www.bsdnow.tv/tutorials/dnsmasq
iptables -t nat -I PREROUTING -p tcp -d 10.0.0.1 -i eth1 -j DNAT --to-destination 172.17.2.1:85
You can test your new set up by navigating to http://doubleclick.net/ and you should be served with a 1x1 gif (check the tab name) if all is working OK.
HTH,
Nick
[edit]
I have had to remove the Google blocks form the DNS poisoning as it was stopping page loads so I've split the wget line into 3 in cron.weekly:
[/edit]wget -O -q "http://pgl.yoyo.org/as/serverlist.php?showintro=0;hostformat=dnsmasq" | sed '/^Ad\s/d' |sed 's|</b>|-|g' | sed 's|<[^>]*>||g' | sed '/^$/d' | sed 's|127.0.0.1|10.0.0.1|g' > /etc/dnsmasq.d/ad_poison.conf
sed -i '/google/d' /etc/dnsmasq.d/ad_poison.conf
service dnsmasq restart
[edit2]
I've pulled it completely as it is causing other issues on sites such as freeads.co.uk where I can no longer log in.
[/edit2] -
Accepted Answer
Thanks Patrick but I'll give this a miss for the moment. It may conflict with my own web server and I don't fancy trying to debug.
As a slight improvement on the page load issue from Dave Loper's one-liner and using an idea from the pgl.yoyo.org site, instead of redirecting to 127.0.0.1, you can redirect to another private IP not on your LAN (I've used 10.0.0.1) and then add a REJECT rule in iptables to give an instant response rather than having to wait for a timeout:
and then in /etc/clearos/firewall.d/local add:wget -O - -q "http://pgl.yoyo.org/as/serverlist.php?showintro=0;hostformat=dnsmasq" | sed '/^Ad\s/d' |sed 's|</b>|-|g' | sed 's|<[^>]*>||g' | sed '/^$/d' | sed 's|127.0.0.1|10.0.0.1|g' > /etc/dnsmasq.d/ad_poison.conf && service dnsmasq restart
The page load issue improves but does not go away.iptables -I FORWARD -d 10.0.0.1 -j REJECT
iptables -I OUTPUT -d 10.0.0.1 -j REJECT
Me, I'm going to stick with browser adblock solutions. -
Accepted Answer
Nick Howitt wrote:
I'm sure I've seen somewhere methods to serve a 1px gif instead from a local web server. That way the response is instantaneous and page loads should be quicker.[/quote]
Nick,
See my previous post with the link to this website : http://www.bsdnow.tv/tutorials/dnsmasq
Here is a howto for using a 1px webserver using pixelserv -
Accepted Answer
One issue I am having with this is page load times. They seem to be much slower with DNS poisoning as the page has to wait for each ad request to time out rather than get an instant request fail message then proceed with the next page element.
I'm sure I've seen somewhere methods to serve a 1px gif instead from a local web server. That way the response is instantaneous and page loads should be quicker. -
Accepted Answer
# Redirect DNS traffic through the server
iptables -t nat -I PREROUTING -p tcp --dport 53 ! -d 172.17.2.1 -i eth1 -j DNAT --to-destination 172.17.2.1
iptables -t nat -I PREROUTING -p udp --dport 53 ! -d 172.17.2.1 -i eth1 -j DNAT --to-destination 172.17.2.1
In this case eth1 is my LAN NIC and "-i eth1" is redundant as there is only one LAN NIC. The IP address should be the ClearOS LAN IP associated with the LAN NIC. I also use a tcp rule as, in some cases, I believe DNS lookups can fall back to tcp. The rules can be repeated multiple times for multiple NICs.
The basic idea is to pick up any traffic coming into your LAN NIC for port 53 and if it is not directed to your LAN address, override its destination so it goes to your LAN IP instead. -
Accepted Answer
-
Accepted Answer
-
Accepted Answer
-
Accepted Answer
Find out if the ads are even processing through your content filter. You can do this by watching the output of your content filter stream:
http://www.clearcenter.com/support/documentation/clearos_guides/live_monitoring_of_web_traffic_in_proxy_and_content_filter
If you are using transparent mode, realize that any https traffic is not being processed by your content filter and if an ad source is https, then you will have to stop it via DNS or switch to non-transparent mode for your block against the domain name to work.
The above code snippet is for DNS blocking which is not part of the content filter at all. It will only work if ClearOS is the DNS server used by your workstations. If your workstations use ClearOS and another DNS source, then it will only block when the site is resolved by ClearOS. ClearOS is optimized for caching so if it is your gateway, it makes sense to have it be your ONLY DNS server for clients. The exception to this is if your clients are part of a MS AD domain.
When you find an AD site, you can make your own poison.conf file and include any extras. This is effective, by the way, for blocking things like Facebook generally as well. -
Accepted Answer
-
Accepted Answer
-
Accepted Answer
Nice Patrick!
Here is a one line code:
wget -O - -q "http://pgl.yoyo.org/as/serverlist.php?showintro=0;hostformat=dnsmasq" | sed '/^Ad\s/d' |sed 's|</b>|-|g' | sed 's|<[^>]*>||g' | sed '/^$/d' > /etc/dnsmasq.d/ad_poison.conf && service dnsmasq restart
-
Accepted Answer
Works great, David.
You can download a,nice list HERE.
If you choose the option DNSMASQ and LINK BACK TO THIS PAGE, you will get the list in proper format.
Change address=/ to server=/
See also this page : http://www.bsdnow.tv/tutorials/dnsmasq
would be a nice feature for ClearOS. ;-)
Please login to post a reply
You will need to be logged in to be able to post a reply. Login using the form on the right or register an account if you are new here.
Register Here »