The image filter caught 3 messages so far today. Two were obvious spam the 3rd was a sales rep blasting out emails to his customers. kida grey area but i'm personally fine with it sitting in there.

Good Job!

R,

Within the AllowedDomainFilterMatrix.txt file I noticed that there are 2 additional parameters at the end with this version.  Can you let me know what they are?

Thanks,
Dan B

Roberto thank you very much for the SFDB. If you can make this work I believe this will be a unique feature that sets Spamfilter ISP from any other product on the market.

After reading your description of how this works I thought maybe I would make another suggestion (sorry) :) What about a ranking system. The idea is that if an IP reach X number of hits it stays in the cache longer or maybe for good.

My thought is if an IP is only seen a few times over some period of time it gets flushed but if an IP continues to show up OR reaches a high reporting level it never gets purged and is considered blacklisted. Who knows may be it can get added to the my local blacklist listing.

Lee

Marco my suggestion has more to do with the way the SF collective (my term for the body of Spamfilter nodes) holds on to IP addresses of those systems that are sending out spam.

If you think about it this is not really any different than any of the Blackhole servers. When an IP is found to be a relaying server or a source of spam it gets blocked and added to the world wide listing. Spamfilter ISP would do the same except it sounds like Roberto has this using a cache mode where it gets purged quite frequently.

My suggestion is that once an IP address reaches some threshold it does not get purged but stays on the list. Maybe the way to handle this is at some point in a future release of 3.0 this could be set by the SF owner.

Say for example I set my threshold at 50 or 100, once an IP is reported by that many nodes in the collective then my Spamfilter server adds that to the static list of blacklisted addresses on my server.

To be honest I am not sure if this is really necessary and we will all have to do testing with this new feature to determine that. My only concern was that cronic spammers don't keep showing up in the list. But in a way this might be self correcting because others will be reporting those addresses.

Either way I am excited to see how this feature evolves and I trust Roberto and the team to make this determination. There will no doubt be changes and tweaks but once this gets worked out I believe this will be another great tool we can all use to keep spam out of our systems.

Lee

Roberto,

I was aware of the points in your comments and to me it would be a useless feature to block based on anonymous uncategorized data. 

Each installation's cached IP blocks are the result of unique filter configurations that match the goals and standards of the individual user.

To block messages based on blocks caused by any filter would be too aggressive for any environment where no false positives is the primary goal. Half of the filters in the program are too aggressive for low or no false positive results.  Some people are so sick of spam they are quite happy with blocking email as well as spam.  These differences in philosophy on how "I" will fight spam and the flexibility of the program is what makes it a superior solution. You should not disregard this value when creating a new feature.

If SFDB is not filter specific, it would be useless. Making it fitler specific would be quite simple.  Use a bit value and the additional data would be miniscule.  SpamFilter should query for common blocks that match the bit value of the running configuration. THEN, you really have something that upgrades the program.

Using a "confidence level" probably would not be needed if the data was filter specific.  A "confidence level" as it is used now does not achieve any type of data accuracy and is virtually useless. All it does is create an illogical damper on using non-specific data.  The path the feature is on now makes it an email blocking feature, not a spam blocking feature.

-MJR

In less than a day I saw the filter with default confidence level block spam and legitimate email equally. 

The opportunity exists to follow this new feature on a path that is contiguous with the rest of the program and makes it a powerful "spam" blocking filter.  I have no use for a filter where I can not explain to the sender nor the recipient the reason an email was rejected or quarantined.

There are other design issues to be considered too. For example, this feature should likely use UDP only to ahchieve the best performance and lowest overhead.  How are "rogue" users posting garbage to the database going to be detected and blocked?  How will access to the SFDB database be restricted so it is secure?  What other controls and checks will be needed?

Don't misunderstand, I like this idea if it upgrades the program for us all or at least the majority.

just brainstorming here; how about this idea:

Suppose some users are catching an IP through the keyword filter, while others are catching the same ip through other means.

If the bitvalues Matt suggested AND the keywords are sent to SFDB, which in turn would 'validate' the triggering keywords,once enough users are confirming that the IP is indeed a spammer, and would return the keyword (RegEx), plus the found ip to all SF users. My bet is that loads of IP's would show up and in turn be blocked if confirmed by enough sf users.

This way we would be creating a network that uses all of our combined knowledge and distribute it to all of us.

I do see trouble with this system as well though.. a keyword that is too simple would catch nearly anything, so those will have to be discarded somehow.

what do you all think?

I think we have a great group of thinkers  here.  If the rest of the admins in the world were as vigilant, we wouldn't have a spam problem.  I like the Idea of a bit value, but I am more concerned about overhead, than false positives.  That said, it becomes a matter of choice really.  Marco's percent Idea would help to temper the more aggressive admins as well as offset the less aggressive admins. 

Matt,  as far as telling an offender why they were rejected, I agree that this is an important aspect of spam fighting.  For this reason I would side with the bitvalue approach.  I have given this some thought and aside from the fact that the SPF database would need new fields to id why something was rejected, there is little extra overhead for the clients.  I for one believe that I would not want someone else's list of keyword determining what is spam and what isn't. Also, there are MAPS servers out there that are way to aggressive for my liking.

Thats my 2 cents for now

John,

Obviously we run completely different operations. Luckily Spamfilter is flexible enough to serve many different types of operations. Our users are strictly corporate users and our strategy for fighting spam begain development years before SpamFilter was out.  Our configuration is obviously much different than yours. That does not make yours nor ours invalid nor problematic, just different.

Therefore my comments on hoping for new features that can serve us ALL and not just you OR me, for example.