Updated Filter order
Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5171
Printed Date: 14 March 2025 at 3:56am
Topic: Updated Filter order
Posted By: LogSat
Subject: Updated Filter order
Date Posted: 17 May 2005 at 11:03pm
We have moved the "official" filter order to a new standalone post at: http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077 - www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077
=======================
Below is the latest list containing the order in which filters are
processed. In general, all whitelists take precedence over blacklists.
There are two exceptions:
- Viruses - if an email is infected, there is no whitelist to save it... it will be blocked, period.
- Allowed Domains - to avoid becoming an open-relay, no email is
ever delivered unless the recipient domain is listed in the "Allowed
Domains" list. The only whitelist that can be used to allow delivery of
emails to non-local addresses is the IP whitelist. The theory is that
any spammer can eventually guess how to use/fake your whitelists to
then abuse your SpamFilter as an open relay. The only thing that they
can't really fake is the IP address (IP spoofing won't help here...).
In the list below, in red are the blacklists, in green the whitelists.
- Whitelisted IP
- Allowed
Domains
- Whitelisted Email Address To
- Whitelisted EMail Address From
- Whitelisted Email From Domain
- Whitelisted Auto White List Force Delivery
- Local
Domain Blacklist
- Local
Emails Blacklist
- Local
Emails TO Blacklist
- Not
in Authorized TO Emails
- Country
Blacklist
- Reject
No Reverse DNS
- Reject
Empty Mail From
- Reject
Same To From Email address
- Reject
if Recipient email in Honeypot email list
- Reject
if IP in Honeypot-generated autoban list
- Reject
Same To From Domain
- Recipient
Count > Max RCPTTO
- MX
Record check
- SPF
Filter
- MAPS
check
- Keyword Whitelist
- Attachment
Filter
- Keywords
- Bayesian
Filtering
- SURBL
check
- Antivirus
Plugin
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Replies:
Posted By: WebGuyz
Date Posted: 06 December 2005 at 11:56pm
Where would the new IP Cache Black List you were talking about fit into the filter order? I am going nuts trying trying to think of a way to stop the 24x7 dictionary attacks that are hitting my mail server (yes, I use AuthorizedTo list extensivley). I can write a script to gather all the IP's of the rejected maurauders, but there is no SF file to stick them since in the order list the AuthorizedTo check comes after the Whitelist check and before the Block IP list. There would have to be a list that was checked upon connection for the IP address and if it was in the cache black list be stopped before the first whitelist check was ever performed.
Is that what the new blacklist you mentioned recently going to do?
My SF machine is getting pounded, but that means my regular mailserver gets to go about its business as usual and everyone is happy so SF is really doing a great job for me. Thanks Roberto!
------------- http://www.webguyz.net
|
Posted By: LogSat
Date Posted: 07 December 2005 at 3:50am
We're still not 100% sure, but so far it looks as if it's going to be
the 1st in the list, so that if the remote IP is in the blacklist
cache, it won't even be allowed to connect. This should greatly reduce
the strain on the Spam Filter server, but will have the downside of the
attempted email to not even being quarantined. But as this is a
"repeated offender", the first few email attempts have already been
quarantined, so we do not think this will be so bad.
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: WebGuyz
Date Posted: 07 December 2005 at 9:46am
Great! Looking forward to this feature. These mail harvesters really p*ss me off. Tired of daily log files > 40 meg filled with AuthorizedTo rejects.
------------- http://www.webguyz.net
|
Posted By: LogSat
Date Posted: 21 December 2005 at 5:03pm
We have moved the "official" filter order to a new standalone post at: http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077 - www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077
As of SpamFilter ISP version 2.7.1.511, following is the updated order of the filters. In red are the blacklists, in green the whitelists:
- Cached IP blacklist
- Whitelisted IP
- Allowed Domains
- Whitelisted Email
Address To
- Whitelisted EMail
Address From
- Whitelisted Email
From Domain
- Whitelisted Auto
White List Force Delivery
- Local IP Blacklist
- Local Domain Blacklist
- Local Emails Blacklist
- Local Emails TO Blacklist
- Not in Authorized TO Emails
- Country Blacklist
- Reject No Reverse DNS
- Reject Empty Mail From
- Reject Same To From Email address
- Reject if Recipient’s email in Honeypot email list
- Reject if IP in Honeypot-generated auto-ban list
- Reject Same To From Domain
- Recipient Count > Max RCPTTO
- MX Record check
- SPF Filter
- MAPS check
- Keyword Whitelist
- Attachment Filter
- Keywords
- Bayesian Filtering
- SURBL check
- Antivirus Plugin
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: Guests
Date Posted: 01 February 2006 at 10:49am
Roberto,
Is there a way for users to customize the filter order? For instance, I would prefer that the Keywords blacklist fires before the MAPS check, since I am not quarantining spam flagged due to keywords, but I am quarantining spam flagged due to MAPS. I've found that a lot of the messages in quarantine have the keywords that I've blacklisted, and if the keywords filter would fire first, I would have less email to sift through in the quarantine list.
|
Posted By: LogSat
Date Posted: 01 February 2006 at 5:24pm
Customizing the filtering order is a rather complex update, and we've always been hesitant in implementing it do to the problems it may cause. SpamFilter is optimized for speed and the low number of resources used, and part of this is due to the way the various filters are employed. For example, the MAPS check is performed very soon after a new incoming connection is detected, before the email's contents are received. This allows SpamFilter to decide if an email is spam even before the actual content is received and analyzed, which can be very cumbersome (CPU-wise) for a server. If the order of these filters is reversed, every single email will have to be fully received before and analyzed any of the DNS-based tests can be applied. If your keyword file is large, this can cause the required server's resources to increase by 10x-100x...
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: LogSat
Date Posted: 16 April 2006 at 10:56pm
We have moved the "official" filter order to a new standalone post at: http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077 - www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077
As of SpamFilter
ISP version 3.0.1.553, following is the updated order
of the filters. In red are the blacklists, in
green the whitelists:
-
Cached IP blacklist
-
Whitelisted IP
-
Whitelisted Email Address To
-
Whitelisted EMail Address From
-
Whitelisted Email From Domain
-
Whitelisted Auto White List Force Delivery
-
Allowed
Domains
-
Local IP
Blacklist
-
Local Domain
Blacklist
-
Local Emails
Blacklist
-
Local Emails
TO Blacklist
-
Not in
Authorized TO Emails
-
Country
Blacklist
-
Reject No
Reverse DNS
-
Reject Empty
Mail From
-
Reject Same
To From Email address
-
Reject if
Recipient�s email in Honeypot email list
-
Reject if IP
in Honeypot-generated auto-ban list
-
Reject Same
To From Domain
-
Recipient
Count > Max RCPTTO
-
MX Record
check
-
SFDB Filter
-
SPF
Filter
-
MAPS check
-
Keyword Whitelist
-
Attachment
Filter
-
Keywords
- Image Filtering
-
Bayesian
Filtering
-
SURBL check
-
Antivirus
Plugin
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: mikek
Date Posted: 23 February 2007 at 10:26am
Read the list, but am still unsure in my case:
I'm trying to take some load off my main smtp server and want to route domains through spamfilter although the customer does not want any spam filtering (yes, those still exist...)
Anyway, I'm using the "Authorized TO EMails" List, which is generated by script off the main mail server. Now I was thinking about adding those domains which do not want spamfiltering to the "Unfiltered Emails" (with :tag), but as I found out until now, then the "Authorized TO EMails" does not get checked and all emails get forwarded to the main smtp server, not only those with valid EMail addresses (which would be my goal - filter out viruses and invalid email addresses on the spamfilter server and forward everything else).
Any possibility to achieve this?
Regards,
Mike
|
Posted By: LogSat
Date Posted: 23 February 2007 at 10:24pm
Sorry Mike. As the list states, the "Unfiltered emails" has a higher priority than the "not in Authorized TO". Thus as soon as the email is whitelisted, all other tests are skipped (except the antivirus one).
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: mikek
Date Posted: 24 February 2007 at 6:09am
OK, so i would have to add each email address separately in the "Unfiltered emails" list with a :tag option, then it would work. I could write a script to do that. Do you think performance could be an issue with this workaround?
|
Posted By: LogSat
Date Posted: 06 January 2008 at 11:25pm
We have moved the "official" filter order to a new standalone post at: http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077 - www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6726&PID=13077#13077
As of SpamFilter
ISP version 4.0.0.766, following is the updated order
of the filters. In red are the blacklists, in
green the whitelists:
-
Cached IP blacklist
- Greylist
-
Whitelisted IP
-
Whitelisted Email Address To
-
Whitelisted EMail Address From
-
Whitelisted Email From Domain
-
Whitelisted Auto White List Force Delivery
-
Allowed
Domains
-
Local IP
Blacklist
-
Local Domain
Blacklist
-
Local Emails
Blacklist
-
Local Emails
TO Blacklist
-
Not in
Authorized TO Emails
-
Country
Blacklist
-
Reject No
Reverse DNS
-
Reject Empty
Mail From
-
Reject Same
To From Email address
-
Reject if
Recipient’s email in Honeypot email list
-
Reject if IP
in Honeypot-generated auto-ban list
-
Reject Same
To From Domain
-
Recipient
Count > Max RCPTTO
-
MX Record
check
-
SFDB Filter
-
SPF
Filter
-
MAPS check
-
Exceeded MaxMsgSizeForSpamFiltering
- Keyword Whitelist
- SFCD Filter
- Blank emails with attachments only
- Spam Images in PDFs
- Attachment Filter
-
Keywords
- Image Filtering
-
Bayesian
Filtering
-
SURBL check
- Resolve URLs and check IPs in MAPS
-
Antivirus
Plugin
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: dcook
Date Posted: 08 January 2008 at 3:40pm
Does that mean that we can't whitelist a sender to avoid the greylist?
I have a client that is not getting some mails and if the client is rejected by the greylist it appears whitelisting will not help?
------------- Dwight www.vividmix.com
|
Posted By: LogSat
Date Posted: 08 January 2008 at 4:00pm
That is correct. Please do note that the greylist will only delay the reception of the first email ever sent by a specific server. Once that email has been received (because the remote SMTP server has retried sending it), the IP will always be allowed to pass the greylist filter in the future.
If really necessary (but there has not been a need for anyone to do this yet...), you can manually add IPs to the greylist files in the \SpamFilter\domains\GreyListAllowed.txt file (which requires stopping/restarting SpamFilter for it to be reloaded).
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: Desperado
Date Posted: 08 January 2008 at 4:13pm
Roberto,
I actually came up with one issue which also happened with sending to Yahoo and I am not sure there is any real answer. Gammadyne, (a semi-mass mailer), totally chokes when it hits any grey-listing server including SpamFilter and Yahoo. I have contacted their Tech Support on this problem and have not yet heard back. It also happens when I stupidly forget to add the Gammadyne server (on our network) to the donothoneypot list and the IP get's put in the BlackList Cache. It seems that the SMTP engine just doesn't want to disconnect and kill it's own thread it it gets a disconnect it was not expecting. This is NOT a SpamFilter issue. Just thought I would warn folks if they use direct SMTP mailers to send notices to their internal users.
------------- The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com
|
Posted By: kitti
Date Posted: 06 August 2008 at 12:33am
Roberto
Good day Roberto. Is it possible to include updated filter order in the help file? It's very hard to find in support forum, it's take alot of time to find it when I look for.
Many thanks Kitti J
------------- From Siam
|
Posted By: LogSat
Date Posted: 06 August 2008 at 4:50pm
Very valid point. We've just updated the documentation, and it will be included in the next release of SpamFilter.
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
|