Not impressed
Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6306
Printed Date: 05 February 2025 at 1:03pm
Topic: Not impressed
Posted By: Hyldsgaard
Subject: Not impressed
Date Posted: 27 November 2007 at 3:03am
|
Every day I receive lots of emails that is obviously spam but Spam Filter just lets it right through.
It catches a lot of spam to, so something is working. Am I the only one that is not impressed by Spam Filter's efficiency? Do I need to tweak some settings? |
Replies:
Posted By: LogSat
Date Posted: 27 November 2007 at 10:37am
|
Hyldsgaard, The most common cause of problems is that SpamFilter is not seeing the real source IP address of the senders. If you have mail being routed in your network so that other servers/firewalls process incoming email and mask the IP address of the sender, then most of the IP-based filtering rules will fails. Another possible issue if you upgraded from SpamFilter "standard" to the "Enterprise" version can occur if you did not import all your file-based filtering settings during the upgrade. If you can zip and email us a few hours worth of SpamFilter's activity logfile, we'll take a look to see if there's any major configuration problems in your setup. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: LogSat
Date Posted: 29 November 2007 at 6:36pm
|
Hyldsgaard, We received your logfile, but it honestly looks as if SpamFilter is working very, very efficiently. In the 24 hours of the log, SpamFilter received 10,016 connection attempts. Of these, it only allowed 475 emails to be forwarded to your server (of these, 75 of them were delivered as the recipients were in the WhitelistedTO whitelist). Even if 1 out of 10 good emails you receive is spam, that will still mean that SpamFilter only missed 47 spam emails out of 10,000, and that's about 99.5% accuracy, which is a great ratio. As a side-note, I noticed that some of your domains have multiple MX records. As you only forwarded us the logfile for one SpamFilter server, I assume you are not running any anti-spam software on the server handling your secondary MX. Please note that spammers are known to send emails to *all* servers listed in the MX records, not just the primary. This means that you are receiving unfiltered spam for the domains with multiple MX records. You can easily verify this by looking at the headers for some of the spam you are receiving. For most of them, you will probably not see the X-SF headers that SpamFilter adds to all the emails it processes. This will tell you which emails are not being processed by SpamFilter. If we're wrong with our analysis, if you can please forward us 5-10 spam emails that you received on 11/27/07, we'll be able to locate those emails in the logfile you sent, and should be able to find out how those emails slipped thru the cracks. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: Desperado
Date Posted: 30 November 2007 at 3:13pm
|
Hyldsgaard,
I do not know if this will post correctly but, below is a spread of stats on my server for a 24 hour period. The green are messages that were delivered and the rest were blocked.
So, out of 787K messages, only about 91K were delivered and most of those were delivered due to customer white-lists. I find this to be rather impressive.
Roberto may be on to something with the multiple MX records. My secondary MX has a whopping 98.8 block rate due to spammers attempting to go there because they think it may bypass spam filtering. Their mistake!
------------- The Desperado Dan Seligmann. Work: http://www.mags.net Personal: http://www.desperado.com |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
javascript:; - Messages 
Posted By: atifghaffar
Date Posted: 30 November 2007 at 3:21pm
|
Desperado,
How are you collecting these stats? thanks for sharing. ------------- best regards Atif |
Posted By: Desperado
Date Posted: 30 November 2007 at 3:37pm
|
Atif,
I use "Sawmill" by Flowerpower. It is a generic log parsing program which I have written a custom "filter Plugin" for. When shipped, Sawmill does support the LogSat format but does not have my latest filter. I will be sending that to them. ------------- The Desperado Dan Seligmann. Work: http://www.mags.net Personal: http://www.desperado.com |
Posted By: LogSat
Date Posted: 30 November 2007 at 3:59pm
|
Atif, As an FYI, there's a simple query you can execute that will give you the breakdown of the types of spam currently in the quarantined database. It is nowhere as detailed as the reports Sawmill provides, but could be useful for quick reports: SELECT tblQuarantine.RejectID, tblRejectCodes.RejectDesc, COUNT(tblQuarantine.RejectID) AS Total FROM tblQuarantine INNER JOIN tblRejectCodes ON tblQuarantine.RejectID = tblRejectCodes.RejectID GROUP BY tblQuarantine.RejectID, tblRejectCodes.RejectDesc ORDER BY TOTAL DESC ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: Desperado
Date Posted: 30 November 2007 at 4:13pm
|
Atif,
Bare in mind that the SQL query is FAST. The only issue is that if a customer has forced messages through or removed messages, those stats will not be included. I do, however, rely on the SQL queries for most checking as it is WAY faster than parsing all the logs. ------------- The Desperado Dan Seligmann. Work: http://www.mags.net Personal: http://www.desperado.com |
Posted By: Stupid
Date Posted: 03 December 2007 at 9:25am
|
You need to tune it properly for about a month before declare it's ineffective. After that, you will find you will face a totally different problem, not with Sapmfilter, but with spam - what are you going to do with them? |
Posted By: atifghaffar
Date Posted: 05 December 2007 at 2:58pm
Roberto, thanks, but please some consideration for us running a caseSensitive OS. this should be SELECT tblquarantine.rejectid, tblrejectcodes.rejectdesc, COUNT(tblquarantine.rejectid) AS Total FROM tblquarantine INNER JOIN tblrejectcodes ON tblquarantine.rejectid = tblrejectcodes.rejectid GROUP BY tblquarantine.rejectid, tblrejectcodes.rejectdesc ORDER BY TOTAL DESC ------------- best regards Atif |
LogSat wrote:Posted By: atifghaffar
Date Posted: 05 December 2007 at 3:01pm
|
Desperado, Do you make reports from your queries every day or can you run your queries at any time. I dont see how it can be really fast with 20-30 million rows in the tblquarantine. The parsing the log way is much slower but at least it can be done offline on a different report-server. Thanks for sharing ------------- best regards Atif |
Posted By: Desperado
Date Posted: 05 December 2007 at 3:17pm
|
Atif,
We try to keep our quarantine below a million records. At that level, a "quick Query" like Robertos takes about 5 seconds MAX. A full query which has to do tome text searches still only takes about 35 to 40 seconds. I would not even think about doing a full query on 20 million records. ------------- The Desperado Dan Seligmann. Work: http://www.mags.net Personal: http://www.desperado.com |
Posted By: atifghaffar
Date Posted: 05 December 2007 at 3:40pm
|
Desperado, I have to to keep 10 days record. So I keep 11 days. It used to be 20-30 million entries but these days its getting like 35-40 million. The good thing about this bad effect is that SF is working very well.! :-) ------------- best regards Atif |
Posted By: atifghaffar
Date Posted: 05 December 2007 at 3:42pm
|
I get an alert if there are more than 30K quarantines/hour. These days I am filtering them out a a different folder! ------------- best regards Atif |
Posted By: atifghaffar
Date Posted: 05 December 2007 at 4:46pm
Not trying to be a fan boy, Here are the result of ONE day analysis sorry about the fomattig, +----------+-------------------------------------------+---------+ | rejectid | rejectdesc | Total | +----------+-------------------------------------------+---------+ | 19 | URL in email found in SURBL search | 1347328 | | 13 | Keywords found in content | 1103101 | | 21 | SFDB filter match | 607959 | | 15 | SPF Sender Policy Framework match | 150683 | | 1 | Domain is in local blacklist file | 116063 | | 8 | IP address is from a blacklisted country | 73295 | | 18 | IP blocked by honeypot entry | 36390 | | 7 | Exceeded maximum number of RCPT TO | 4452 | | 23 | Found prohibited attachment | 2835 | | 20 | Detected spam signature in embedded image | 1312 | | 24 | Blank email with PDF attachment only | 902 | | 9 | EmailTO is in local blacklist file | 890 | | 25 | Too many spaces in subject | 92 | +----------+-------------------------------------------+---------+ 13 rows in set (1 min 45.03 sec) ------------- best regards Atif |