Spam Filter ISP Forums : Emails Quarantined for Nonexistent users https://www.logsat.com/spamfilter/forums/ Sat, 14 Mar 2026 16:06:33 +0000 Fri, 28 Mar 2008 20:57:26 +0000 http://blogs.law.harvard.edu/tech/rss Web Wiz Forums 11.04 360 https://www.logsat.com/spamfilter/forums/RSS_post_feed.asp?TID=6449 <![CDATA[Spam Filter ISP Forums]]> https://www.logsat.com/spamfilter/forums/forum_images/web_wiz_forums.png https://www.logsat.com/spamfilter/forums/ <![CDATA[Emails Quarantined for Nonexistent users : Oh well.. since I&#039;ve already...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11945&title=emails-quarantined-for-nonexistent-users#11945 Author: LogSat
Subject: 6449
Posted: 28 March 2008 at 8:57pm

Oh well.. since I've already started, and as it can be used by others who do have a file-based list, here it is anyways :-)


CREATE TABLE test(
email varchar(320)
);

LOAD DATA INFILE "/tmp/WL_AuthorizedTOEmails.txt"
INTO TABLE test
LINES TERMINATED BY '\r\n';

UPDATE tblquarantine  LEFT JOIN test ON tblquarantine.emailto = test.email
SET expire=1 WHERE ISNULL(test.email)

DROP TABLE test;



The above will mark the emails for deletion so that SpamFilter can then delete them during regular cleanup. If you want to delete them right away, this one will do the job:

CREATE TABLE test(
email varchar(320)
);

LOAD DATA INFILE "/tmp/WL_AuthorizedTOEmails.txt"
INTO TABLE test
LINES TERMINATED BY '\r\n';

DELETE tblquarantine FROM tblquarantine LEFT JOIN test ON tblquarantine.emailto = test.email
WHERE ISNULL(test.email);

DELETE tblmsgs FROM tblmsgs LEFT JOIN tblquarantine ON tblmsgs.msgid = tblquarantine.msgid
WHERE ISNULL(tblquarantine.msgid);

DROP TABLE test;


]]> Fri, 28 Mar 2008 20:57:26 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11945&title=emails-quarantined-for-nonexistent-users#11945 <![CDATA[Emails Quarantined for Nonexistent users : Don&#039;t worry about the script....]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11943&title=emails-quarantined-for-nonexistent-users#11943 Author: jerbo128
Subject: 6449
Posted: 28 March 2008 at 5:51pm

Don't worry about the script.  The users are in the DB.  And I have a query to weed out the bad emails. 
 
My point is that when they come in at a rate of 30,000 per day, I just see it as a lot of unneeded SQL traffic. Both inbound and then deleting.
 
As for the reliability of my blacklist, it's pretty good.  But when you have that many entries, it only takes 1 bed entry to get a chewing from the customer.  So I don't know if I am ready to flip the flag for "do not quarantine"
 
Jeremy
 
]]> Fri, 28 Mar 2008 17:51:58 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11943&title=emails-quarantined-for-nonexistent-users#11943 <![CDATA[Emails Quarantined for Nonexistent users : That depends on the reliability...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11942&title=emails-quarantined-for-nonexistent-users#11942 Author: LogSat
Subject: 6449
Posted: 28 March 2008 at 5:45pm

That depends on the reliability of your IP blacklist, which is causing the bulk of these blocks. If you are note receiving false positives from it, you could enable the option to "do not quarantine" emails from this filter.

Also, if you have the list of allowed users in a database, it's rather simple to create a query that will weed out the unwanted recipients. If the list is only in a text file, there's a few extra steps involved to bulk import it in a temp table in MySQL (don't know yet if MySQL can perform a query directly against a text file, as MS SQL can do). I'll take a stab at it tonite to see if I can help you here, but please email me to stop me if the data is already in the DB!
]]> Fri, 28 Mar 2008 17:45:50 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11942&title=emails-quarantined-for-nonexistent-users#11942 <![CDATA[Emails Quarantined for Nonexistent users : Do you have any suggestions for...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11936&title=emails-quarantined-for-nonexistent-users#11936 Author: jerbo128
Subject: 6449
Posted: 27 March 2008 at 11:35pm

Do you have any suggestions for dealing with such a huge amount of crap mail?

My SQL server stays much happier with 200K less emails in it's tummy.
 
 
]]> Thu, 27 Mar 2008 23:35:52 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11936&title=emails-quarantined-for-nonexistent-users#11936 <![CDATA[Emails Quarantined for Nonexistent users : Geez, I can&#039;t even read my...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11935&title=emails-quarantined-for-nonexistent-users#11935 Author: LogSat
Subject: 6449
Posted: 27 March 2008 at 10:49pm

Geez, I can't even read my own warning. And think that I had even super bolded the notice "Please always check the latest post in this thread" on that thread.

You're right, I did not see the updated order. What you posted is correct, the 4 filters (not 3) that cause emails to be quarantined before the "Authorized TO" list is hit are:
  1. Local IP Blacklist
  2. Local Domain Blacklist
  3. Local Emails Blacklist
  4. Local Emails TO Blacklist

I did not mention the others that appear first (allowed domains, and the "forgotten" blacklist cache and greylist) because these ones reject emails immediately, without allowing them to be quarantined, so do not apply in ths discussion.
c
]]> Thu, 27 Mar 2008 22:49:02 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11935&title=emails-quarantined-for-nonexistent-users#11935 <![CDATA[Emails Quarantined for Nonexistent users : LogSat wrote:jeremy,The only...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11933&title=emails-quarantined-for-nonexistent-users#11933 Author: jerbo128
Subject: 6449
Posted: 27 March 2008 at 7:16pm

Originally posted by LogSat LogSat wrote:

jeremy,

The only three filters that are before the "Authorized TO" list are:

  1. Local Domain Blacklist
  2. Local Emails Blacklist
  3. Local Emails TO Blacklist
Local domain blacklist =23,000 rows
Emails to/from = less than 100 each
Local IP blacklist = 250,000 rows (does not count for subnets)
 
Most of these emails that I am seeing to unAutorized addresses are quarantined with ID 12 - Local IP Blacklist.  I see a few with reason 7 and 9.
 
Yes, you are correct - I am getting over 200,000 per week to these unauthorized addresses.  Looking at the IP's and subjects of the connections - they are scattered everywhere.
 
You said above that  local IP blacklist is AFTER the Authorized User Check, then why am I seeing all of these?   For clarification -This post lists a different filter order.   http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=5171 
Which is right?
  1. Cached IP blacklist
  2. Greylist
  3.         Whitelisted IP
  4.         Whitelisted Email Address To
  5.         Whitelisted EMail Address From
  6.         Whitelisted Email From Domain
  7.         Whitelisted Auto White List Force Delivery
  8. Allowed Domains
  9. Local IP Blacklist
  10. Local Domain Blacklist
  11. Local Emails Blacklist
  12. Local Emails TO Blacklist
  13. Not in Authorized TO Emails

Jeremy



Edited by jerbo128 - 27 March 2008 at 7:20pm]]> Thu, 27 Mar 2008 19:16:00 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11933&title=emails-quarantined-for-nonexistent-users#11933 <![CDATA[Emails Quarantined for Nonexistent users : jeremy,The only three filters...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11932&title=emails-quarantined-for-nonexistent-users#11932 Author: LogSat
Subject: 6449
Posted: 27 March 2008 at 5:47pm

jeremy,

The only three filters that are before the "Authorized TO" list are:

  1. Local Domain Blacklist
  2. Local Emails Blacklist
  3. Local Emails TO Blacklist


emails blocked by these are usually very, very small in numbers, if any at all. Are you actually receiving 200,000 emails each week blocked by these 3? If you have a huge blacklist of blacklisted sender's domains and emails, that may explain this large number, but if not, you may be targeted by very specific spammers/attackers.

In "normal" installations, the 3 lists above are often much smaller than the "Authorized TO" list, which is why SpamFilter examines the smaller ones first to try saving some resources. An option may be to change the order of the filters by moving up the "Authorized TO". We'll be monitoring the thread for feedback on this.
]]> Thu, 27 Mar 2008 17:47:24 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11932&title=emails-quarantined-for-nonexistent-users#11932 <![CDATA[Emails Quarantined for Nonexistent users : Roberto, I understand, butin...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11931&title=emails-quarantined-for-nonexistent-users#11931 Author: jerbo128
Subject: 6449
Posted: 27 March 2008 at 4:44pm

Roberto,
 
I understand, but in my opinion, this could be a huge isse.  With my retention of 7 days - that is 200,000 messages.  To me, that is a waste of resources also.
 
Maybe we need to weigh it out -to see which has a higher "cost"
 
Anyone else have thoughts on this?  How about some of you bigger guys who get a lot of mail, have you ever looked at the number of orphaned messages?
]]> Thu, 27 Mar 2008 16:44:06 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11931&title=emails-quarantined-for-nonexistent-users#11931 <![CDATA[Emails Quarantined for Nonexistent users : Jeremy,The order in which the...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11930&title=emails-quarantined-for-nonexistent-users#11930 Author: LogSat
Subject: 6449
Posted: 27 March 2008 at 4:23pm

Jeremy,

The order in which the filters is applied is listed at http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=5171#5826. If a filter blocks an email, all subsequent filters will be ignored to avoid wasting resources. This will indeed cause a number of emails where the recipient is not in the "Autorized TO" list to be quarantined instead of being rejected right away.
]]> Thu, 27 Mar 2008 16:23:03 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11930&title=emails-quarantined-for-nonexistent-users#11930 <![CDATA[Emails Quarantined for Nonexistent users : I am getting a fair amount of...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11929&title=emails-quarantined-for-nonexistent-users#11929 Author: jerbo128
Subject: 6449
Posted: 27 March 2008 at 12:40pm

I am getting a fair amount of mail in the quarantine for email addresses that do not appear in my tblwl_authorizedtoemails. 
 
I have a script that expires any message in tbl_quarantine where the email field does not have a match in tblwl_authorizedtoemails.
 
I cleaned such messages this am, and now 2 hours later, I have 2600 more messages that match this rule.  The first time that I ran this rule, it removed 200K messages from my database.
 
After looking at the logs, It appears that the messages are being quarantined because they failed the local blacklist check.  So, the authorized To Email list is never checked.
 
Roberto - is this the intended behavior?  What can we do to stop these from clogging my database up? 
 
Running 773 SFE.
 
Any idea's?
 
Jeremy


Edited by jerbo128 - 27 March 2008 at 2:39pm]]> Thu, 27 Mar 2008 12:40:51 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6449&PID=11929&title=emails-quarantined-for-nonexistent-users#11929