Every day I receive lots of emails that is obviously spam but Spam Filter just lets it right through.

It catches a lot of spam to, so something is working.

Am I the only one that is not impressed by Spam Filter's efficiency? Do I need to tweak some settings?

Hyldsgaard,

We received your logfile, but it honestly looks as if SpamFilter is working very, very efficiently.

In the 24 hours of the log, SpamFilter received 10,016 connection attempts. Of these, it only allowed 475 emails to be forwarded to your server (of these, 75 of them were delivered as the recipients were in the WhitelistedTO whitelist).

Even if 1 out of 10 good emails you receive is spam, that will still mean that SpamFilter only missed 47 spam emails out of 10,000, and that's about 99.5% accuracy, which is a great ratio.

As a side-note, I noticed that some of your domains have multiple MX records. As you only forwarded us the logfile for one SpamFilter server, I assume you are not running any anti-spam software on the server handling your secondary MX. Please note that spammers are known to send emails to *all* servers listed in the MX records, not just the primary. This means that you are receiving unfiltered spam for the domains with multiple MX records. You can easily verify this by looking at the headers for some of the spam you are receiving. For most of them, you will probably not see the X-SF headers that SpamFilter adds to all the emails it processes. This will tell you which emails are not being processed by SpamFilter.

If we're wrong with our analysis, if you can please forward us 5-10 spam emails that you received on 11/27/07, we'll be able to locate those emails in the logfile you sent, and should be able to find out how those emails slipped thru the cracks.

Desperado,

How are you collecting these stats?

thanks for sharing.

Atif,

As an FYI, there's a simple query you can execute that will give you the breakdown of the types of spam currently in the quarantined database. It is nowhere as detailed as the reports Sawmill provides, but could be useful for quick reports:

SELECT tblQuarantine.RejectID, tblRejectCodes.RejectDesc,
COUNT(tblQuarantine.RejectID) AS Total
FROM tblQuarantine INNER JOIN
tblRejectCodes ON tblQuarantine.RejectID =
tblRejectCodes.RejectID
GROUP BY tblQuarantine.RejectID, tblRejectCodes.RejectDesc
ORDER BY TOTAL DESC

You need to tune it properly for about a month before declare it's ineffective.

After that, you will find you will face a totally different problem, not with Sapmfilter, but with spam - what are you going to do with them?

Desperado,

Do you make reports from your queries every day or can you run your queries at any time.

I dont see how it can be really fast with 20-30 million rows in the tblquarantine.

The parsing the log way is much slower but at least it can be done offline on a different report-server.

Thanks for sharing

Desperado, I have to to keep 10 days record. So I keep 11 days.
It used to be 20-30 million entries but these days its getting like 35-40 million.

The good thing about this bad effect is that SF is working very well.! :-)

Hyldsgaard wrote: Every day I receive lots of emails that is obviously spam but Spam Filter just lets it right through.

Not trying  to be a fan boy,  Here are the result of  ONE day analysis
sorry about the fomattig,

+----------+-------------------------------------------+---------+
| rejectid | rejectdesc                                | Total   |
+----------+-------------------------------------------+---------+
|       19 | URL in email found in SURBL search        | 1347328 |
|       13 | Keywords found in content                 | 1103101 |
|       21 | SFDB filter match                         |  607959 |
|       15 | SPF Sender Policy Framework match         |  150683 |
|        1 | Domain is in local blacklist file         |  116063 |
|        8 | IP address is from a blacklisted country  |   73295 |
|       18 | IP blocked by honeypot entry              |   36390 |
|        7 | Exceeded maximum number of RCPT TO        |    4452 |
|       23 | Found prohibited attachment               |    2835 |
|       20 | Detected spam signature in embedded image |    1312 |
|       24 | Blank email with PDF attachment only      |     902 |
|        9 | EmailTO is in local blacklist file        |     890 |
|       25 | Too many spaces in subject                |      92 |
+----------+-------------------------------------------+---------+
13 rows in set (1 min 45.03 sec)



Edited by atifghaffar - 05 December 2007 at 4:47pm