Emails Quarantined for Nonexistent users
Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449
Printed Date: 12 March 2025 at 8:46pm
Topic: Emails Quarantined for Nonexistent users
Posted By: jerbo128
Subject: Emails Quarantined for Nonexistent users
Date Posted: 27 March 2008 at 12:40pm
|
I am getting a fair amount of mail in the quarantine for email addresses that do not appear in my tblwl_authorizedtoemails.
I have a script that expires any message in tbl_quarantine where the email field does not have a match in tblwl_authorizedtoemails.
I cleaned such messages this am, and now 2 hours later, I have 2600 more messages that match this rule. The first time that I ran this rule, it removed 200K messages from my database.
After looking at the logs, It appears that the messages are being quarantined because they failed the local blacklist check. So, the authorized To Email list is never checked.
Roberto - is this the intended behavior? What can we do to stop these from clogging my database up?
Running 773 SFE.
Any idea's?
Jeremy
|
Replies:
Posted By: LogSat
Date Posted: 27 March 2008 at 4:23pm
|
Jeremy, The order in which the filters is applied is listed at http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=5171#5826 - http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=5171#5826 . If a filter blocks an email, all subsequent filters will be ignored to avoid wasting resources. This will indeed cause a number of emails where the recipient is not in the "Autorized TO" list to be quarantined instead of being rejected right away. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: jerbo128
Date Posted: 27 March 2008 at 4:44pm
|
Roberto,
I understand, but in my opinion, this could be a huge isse. With my retention of 7 days - that is 200,000 messages. To me, that is a waste of resources also.
Maybe we need to weigh it out -to see which has a higher "cost"
Anyone else have thoughts on this? How about some of you bigger guys who get a lot of mail, have you ever looked at the number of orphaned messages?
|
Posted By: LogSat
Date Posted: 27 March 2008 at 5:47pm
|
jeremy, The only three filters that are before the "Authorized TO" list are:
emails blocked by these are usually very, very small in numbers, if any at all. Are you actually receiving 200,000 emails each week blocked by these 3? If you have a huge blacklist of blacklisted sender's domains and emails, that may explain this large number, but if not, you may be targeted by very specific spammers/attackers. In "normal" installations, the 3 lists above are often much smaller than the "Authorized TO" list, which is why SpamFilter examines the smaller ones first to try saving some resources. An option may be to change the order of the filters by moving up the "Authorized TO". We'll be monitoring the thread for feedback on this. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: jerbo128
Date Posted: 27 March 2008 at 7:16pm
Local domain blacklist =23,000 rows
Emails to/from = less than 100 each
Local IP blacklist = 250,000 rows (does not count for subnets)
Most of these emails that I am seeing to unAutorized addresses are quarantined with ID 12 - Local IP Blacklist. I see a few with reason 7 and 9.
Yes, you are correct - I am getting over 200,000 per week to these unauthorized addresses. Looking at the IP's and subjects of the connections - they are scattered everywhere.
You said above that local IP blacklist is AFTER the Authorized User Check, then why am I seeing all of these? For clarification -This post lists a different filter order. http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=5171 - http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=5171
Which is right?
Jeremy |
LogSat wrote:Posted By: LogSat
Date Posted: 27 March 2008 at 10:49pm
|
Geez, I can't even read my own warning. And think that I had even super bolded the notice "Please always check the latest post in this thread" on that thread. You're right, I did not see the updated order. What you posted is correct, the 4 filters (not 3) that cause emails to be quarantined before the "Authorized TO" list is hit are:
I did not mention the others that appear first (allowed domains, and the "forgotten" blacklist cache and greylist) because these ones reject emails immediately, without allowing them to be quarantined, so do not apply in ths discussion. c ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: jerbo128
Date Posted: 27 March 2008 at 11:35pm
|
Do you have any suggestions for dealing with such a huge amount of crap mail? My SQL server stays much happier with 200K less emails in it's tummy.
|
Posted By: LogSat
Date Posted: 28 March 2008 at 5:45pm
|
That depends on the reliability of your IP blacklist, which is causing the bulk of these blocks. If you are note receiving false positives from it, you could enable the option to "do not quarantine" emails from this filter. Also, if you have the list of allowed users in a database, it's rather simple to create a query that will weed out the unwanted recipients. If the list is only in a text file, there's a few extra steps involved to bulk import it in a temp table in MySQL (don't know yet if MySQL can perform a query directly against a text file, as MS SQL can do). I'll take a stab at it tonite to see if I can help you here, but please email me to stop me if the data is already in the DB! ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: jerbo128
Date Posted: 28 March 2008 at 5:51pm
|
Don't worry about the script. The users are in the DB. And I have a query to weed out the bad emails.
My point is that when they come in at a rate of 30,000 per day, I just see it as a lot of unneeded SQL traffic. Both inbound and then deleting.
As for the reliability of my blacklist, it's pretty good. But when you have that many entries, it only takes 1 bed entry to get a chewing from the customer. So I don't know if I am ready to flip the flag for "do not quarantine"
Jeremy
|
Posted By: LogSat
Date Posted: 28 March 2008 at 8:57pm
|
Oh well.. since I've already started, and as it can be used by others who do have a file-based list, here it is anyways :-) CREATE TABLE test( email varchar(320) ); LOAD DATA INFILE "/tmp/WL_AuthorizedTOEmails.txt" INTO TABLE test LINES TERMINATED BY '\r\n'; UPDATE tblquarantine LEFT JOIN test ON tblquarantine.emailto = test.email SET expire=1 WHERE ISNULL(test.email) DROP TABLE test; The above will mark the emails for deletion so that SpamFilter can then delete them during regular cleanup. If you want to delete them right away, this one will do the job: CREATE TABLE test( email varchar(320) ); LOAD DATA INFILE "/tmp/WL_AuthorizedTOEmails.txt" INTO TABLE test LINES TERMINATED BY '\r\n'; DELETE tblquarantine FROM tblquarantine LEFT JOIN test ON tblquarantine.emailto = test.email WHERE ISNULL(test.email); DELETE tblmsgs FROM tblmsgs LEFT JOIN tblquarantine ON tblmsgs.msgid = tblquarantine.msgid WHERE ISNULL(tblquarantine.msgid); DROP TABLE test; ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |