[Rabbit-dev] ad blocking

[Luis Soltero]

Hi Robo,

DNS and hosts tables is a bad way to do this... I attempted it since it required no changes to the rabbit code.  Its
much better to run the table entries through AdFilter and BlockFilter so that the correct appropriate action is taken. 
Using DNS or the system resolver slows things down and causes additional overhead since localhost is queried for the
url.  Also... there is no control over which filter in rabbit gets triggered.

I have looked at the code and it doesn't look like it will be very difficult to implement a look up table approach.

The way I see it it would be good to have 2 tables... One for AdFilter and one fro BlockFilter.  These the lists could
be read either from files on the disk or from tables in a db.  You already have the interface for sql based
authentication so it seems that adding code to do sql queries for Ad/BlockFilter should be simple.

The idea is to first check the url against a the tables...If there is a hit then you execute the Block/NoAd.Gif action. 
If there is no hit then you fall through to the regex evaluation as you do now.

>From an administrators view point this would be really good.  Lists of ad and malware sites could be downloaded by an
external script which dynamically updates the tables in the db.  There are quite a few sites which provide these lists.

This would be a ** VERY ** nice addition to Rabbit since the proxy could then be used to save bandwidth by effectively
removing ads and protecting users by keeping them away from know malware and phishing sites.

I appreciate your willingness to look at this.  Although the coding looks pretty straight forward it would be a big
effort for us to implement it here.  We are not java programmers and don't have a development environment setup so there
would be quite a bit of effort just to get to the point where we could experiment with the code.  You could probably
implement this in a matter of hours...

Can we help by sending you a donation?

Let me know if there is anything we can do to help.

--luis


On 12/14/10 2:39 PM, Robert Olofsson wrote:
> On Tue, 14 Dec 2010 00:49:48 -0500
> Luis Soltero <lsoltero at globalmarinenet.com> wrote:
>
>> Does rabbit have the ability to block ads and malware sites using lists
>> from aggregator sites such as ...
> Depends on what you mean by blocking. Rabbit can easily deny requests
> based on the url. The question is what action you want to take.
>
>> This facility is very flexible but difficult to maintain in an every changing internet landscape.  A better approach
>> might be to use lists from aggragators who's mission is to keep up to date lists of sites that offer ads and malware.
> Yes that might be a good idea.
>
>> My first naive approach at solving this problem was to augment /etc/hosts on our proxy server with lists from the above
>> sites. I soon discovered rabbit ignored these.  It seems that rabbit uses javands to access the DNS service directly to
>> do queries ignoring the system resolver.   So replacing /etc/hosts does not work.
> Rabbit uses dnsjava by default, but you can change it in rabbit.conf:
> [dns]
> dnsHandler=rabbit.dns.DNSSunHandler
>
> But please note that some jvms/OS:es will do only one dns lookup at 
> a time, causing big problems if you have many users. My linux used
> to do such things.
>
>> 1. rabbit reads the bad host table on startup and then keeps an internal table for lookups (our current host table has
>> over 600K entries so this approach should be manageable)
> Easy, will use some memory, but probably manageable amounts by todays
> hardware specifications.
>
>> 2. a better approach might be to query an sql table for bad hosts prior to the lookup.  This would be faster and more
>> dynamic since the table could be updated automatically from an external process.
> Easy to do. I can try to take a hack at it during the winter holidays.
>
> /robo
>
> _______________________________________________
> Rabbit-dev mailing list
> Rabbit-dev at khelekore.org
> http://khelekore.org/cgi-bin/mailman/listinfo/rabbit-dev
>


-- 


Luis Soltero, Ph.D., MCS
Director of Software Development, CTO
Global Marine Networks, LLC
StarPilot, LLC
Tel: 865-379-8723
Fax: 865-681-5017
E-Mail: lsoltero at globalmarinenet.net
Web: http://www.globalmarinenet.net
Web: http://www.starpilotllc.com