[Rabbit-dev] ad blocking

[Ping ni]

On 12/15/10, Ping ni <ping.ni.bupt at gmail.com> wrote:
> I think you can extend the the block listener and write your
> processing logistics. it can do what you want.
>
> about luis question, you are right. actually rabbit has left an
> interface for sql. you could put the malware host or ip into the
> database. get the list from databases as the map structure for
> accelerating efficiency.
>
> On 12/14/10, Luis Soltero <lsoltero at globalmarinenet.com> wrote:
>>
>> Hello All,
>>
>> Does rabbit have the ability to block ads and malware sites using lists
>> from
>> aggregator sites such as
>>
>> http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext
>> http://someonewhocares.org/hosts/hosts
>> http://www.montanamenagerie.org/hostsfile/hosts.txt
>> http://www.hosts-file.net/hphosts-partial.asp
>> http://www.mvps.org/winhelp2002/hosts.txt
>>
>> I know that rabbit has the following blocking facility
>>
>> ---
>> [rabbit.filter.BlockFilter]
>> # This is a filter that blocks access to resources.
>>
>> # return a 403 forbidden for these requests.
>> blockURLmatching=(\.sex\.|[-.]ad([sx]?)\.|/ad\.|adserving\.|ad101com-|pagead/imgad|as-us.falkag.net|clicktorrent.info)
>> ---
>>
>> This facility is very flexible but difficult to maintain in an every
>> changing internet landscape.  A better approach
>> might be to use lists from aggragators who's mission is to keep up to
>> date
>> lists of sites that offer ads and malware.
>>
>> My first naive approach at solving this problem was to augment /etc/hosts
>> on
>> our proxy server with lists from the above
>> sites. I soon discovered rabbit ignored these.  It seems that rabbit uses
>> javands to access the DNS service directly to
>> do queries ignoring the system resolver.   So replacing /etc/hosts does
>> not
>> work.
>>
>> A better solution would be for rabbit check against a preconfigured
>> "block"
>> list of sites and then return 403 errors
>> when the urls containing these hosts names are requested.  It should be
>> pretty simple thing to do to query the bad host
>> table prior to doing DNS query.  I can see two implementations of this
>> approach.
>> 1. rabbit reads the bad host table on startup and then keeps an internal
>> table for lookups (our current host table has
>> over 600K entries so this approach should be manageable)
>> 2. a better approach might be to query an sql table for bad hosts prior
>> to
>> the lookup.  This would be faster and more
>> dynamic since the table could be updated automatically from an external
>> process.
>>
>> I think that adding this facility to Rabbit should be pretty easy and
>> quite
>> valuable to the community.  Those of us
>> using rabbit are mostly running in a bandwidth limited environment and
>> what
>> better way to save bandwidth than to strip
>> out ads. This approach also has the benefit protecting users from known
>> malware sites.
>>
>> Anyone have any thoughts on this?
>>
>> Thanks,
>>
>> --luis
>>
>>
>> --
>>
>>
>> Luis Soltero, Ph.D., MCS
>> Director of Software Development, CTO
>> Global Marine Networks, LLC
>> StarPilot, LLC
>> Tel: 865-379-8723
>> Fax: 865-681-5017
>> E-Mail: lsoltero at globalmarinenet.net
>> Web: http://www.globalmarinenet.net
>> Web: http://www.starpilotllc.com
>>
>>
>>
>> _______________________________________________
>> Rabbit-dev mailing list
>> Rabbit-dev at khelekore.org
>> http://khelekore.org/cgi-bin/mailman/listinfo/rabbit-dev
>>
>