SF has a problem with multi-recipients in general.

IF a spammer sends an email to 10 users (same domain) in a single email and any one of those uses is whitelisted, all the recipients get the email.

We use AuthorizedTo lists and if a spammer sends a single email 10 users in the same domain, 9 bogus users and 1 user who is real and who happens to be whitelistedTo, then SF will send to all 10 users and 9 of them will bounce because they are invalid users.

Why doesn't SF recognize at least AuthorizedTo users. I think your handling of multi recipients needs to be re-evaluated. I know its hard to do, but its a pretty big hole and I have a few spammers who have figured it out and continually send spam lists with multi-recipients. They don't know why it gets thru, they just know its not being rejected and keep on sending it in that same format.

Roberto,

  I sent a zip with an example of the failure of multi-recips and AuthorizedTo list. It has a zip attachment so if you don't get it check your spam filter.

HI,

I have too encountered this bug, and personally

think / suggest that Spamfilter should split message

to handle this special case. as it is fairly commonly found senario.

Thnaks

Whats bad is if the spammer crowd does not get a message rejection they score that as a win and will keep using that same set of recipients over and over again. We have a few cases where a handfull of people are always hit and because one lone user in that bunch insists on being unfiltered all the users in that same 8 recipient group get the spam even though only one user wants it.

And sgeorge, mailto:*whitelisteduser@mydomain.com - *|whitelisteduser@mydomain.com does not really help, users still do not get spamfiltered properly and spam that should have been caught gets on through.

just a thought: maybe SPF can treat the mail as spam all the way, but make it check if any of the recipients is "allowed" to receive the spam. if a match on one of the adresses is found, attach a new header to the mail with only *that* adress and send it on to the mail host.

I'm not sure if that is legal, since it is a form of tampering with mails (adding a new header in front of the existing mail), but maybe this could be a solution to this particular problem. Spammers know the effect of multiple recipients and, lame as they are, will not hesitate to abuse this mail feature. The need of the many should outweigh the need of the one is my opinion, smoking in public places is no longer allowed too.

Ofcourse adding a new header, and inserting the old header into the mailbody creates overhead for the SF box, not to meantion headaches for Roberto, but perhaps it is doable.

Marco

I decided to try utilizing the information in this thread concerning the bayseian filter forcing email to be delivered to multiple recipients when one or more recipients is on our exclusion list and/or has a mailto:*mailbox@domain - *|mailbox@domain entry in the autowhitelistforceddelivery file.

Turning of the "learn new emails" function in the bayseian filter seems to have done the trick...I think....we're seeing a lot more disconnects now when the emailauthorizedto list is violated.  Maybe this will help to block all those spams getting through because of multiple recipients.  We've been getting a huge amount of them here in the last few weeks.

vrspock wrote: Turning of the "learn new emails" function in the bayseian filter seems to have done the trick...I think....we're seeing a lot more disconnects now when the emailauthorizedto list is violated.  Maybe this will help to block all those spams getting through because of multiple recipients.  We've been getting a huge amount of them here in the last few weeks.

Keep us appraised please. We are plagued by this as well. Thanks!!

Roberto,

with regard to your post on May 17th...

As SF needs to accept the email, and at least one recipient is "unfiltered", when the email is passed to the bayseian filter....is it learnt as ham or spam ????

oh, even though it's obviously spam ??

I would have thought that you would learn it as spam, but still deliver it to the unfiltered user.

Surely doing it the current way could render the baysian database useless, as it's learning spam as ham ?

"hurt performance" .....  even at the expense of letting in spam?? Doesn't that defeat the whole idea of SF ??

So far so good webguys...I will hear a report from our end users on Monday, but from our logs it appears to be doing the trick:  Authorized To in combination with disabling the bayesian filter learning new emails and using *|mailbox for exlcuding accounts on the autowhitelistforceddelivery list.

What appers to happen is when a multiple-recipeint email comes in with a hit-or-miss list of mailboxes, typically the very first mailbox is an invalid one, the authorized to list gives its error and the session is immediately disconnected without further processing.  Also, we've set our blacklist cache and limbo settings to some very agressive settings.  We had already done this before trying to disable the bayesian filter as mentioned in this thread.  I'm not sure how doable these settings would be for those with a much higher volume of email coming through, but we have ours set to 3 failures within a 2-hour period lands you on our blacklist cache for 6-hours.  With all of the authroized-to failures dropping connections, our black list cache limbo is starting to become quite extensive, but our actual black list cache will typically only have 4 or 5 ip's at any given time, which confirms that the spam that has been getting through is hitting us from zombie machines.

I've only seen one or two spams getting through to my mailbox this weekend which is a significant drop from the past few weeks.

We've still got some very persistant spam getting through....one appears to have been a connection from Korea that happen to hit a valid, excluded email address on the first try and had a field day with random from/to addresses from then on.  Managed to get quite a few spam to pass through and generate a bunch of NDR's when spam filter attempted to deliver the message to about 20 or so invalid mailboxes.  This was one of the pharmacuetical spams.

Since this causes a mass of NDR's to be generated anyhow when spam filter accepts an email that has 1 valid, excluded address and about 20 to 30 invalid addresses, what difference would it make if we just split the message and NDR the non-excluded addresses?  Either way, we are generating a bunch of bounce backs.  At least by splitting the message, only those on the exclude list are getting what they asked for while everyone else is still filtered.

Think I'm going to lower our acceptable number of CC's and see if that helps any. 

I have a solution which works for us, although Roberto will go nuts when he hears it !!!

I configure SF to not quarantine anything, but instead to tag and deliver. It then delivers the mail to IIS's SMTP server on the same box, which is configured to allow all domains. I then run a custom script in the OnArrival SMTP event which splits the email, sending it only to the recipients that either are "unfiltered" of specifically want email from the sender/domain. I then just abort delivery to the rest of the recipients, as they either dont exist, or don't want the spam. Therefore, the people who want the email get it, and the people who dont....dont. Also, as the email delivery gets aborted, no NDR's are generated.

Yes, Roberto, I hear you....I know you don't like splitting emails, and it's not strictly to the letter of the RFC, but if you won't do it, then I'll have to ...  ....talking of the RFC...I understand being in strict compliance....but the RFC has major failings. If they had got the RFC right in the first place, we wouldn't be getting this spam now....

For the rest of you....this custom script includes a bunch of code that needs to remain confidential, but I'll try to pull out the relevant parts and post the code here just in case anyone wants to have a play. Give me a week or so as were quite busy at the moment.

i advise you to be very  'paranoid' when giving out code and explaining how you do it *in public*  im convinced some of the registered users here are indeed spammers. Topics have been started on this subject before, just be aware.

I agree, caution is definetly in order on here with regards to our methods of dealing with the constant flow of spam.

I have thought about using our mail server's own anti-spam features in combination with spamfilter to do something similar, but the whole point of the authorizedto list is to reduce the number of hits we get from people who are blind spamming using lists that are populated with a bunch of non-existant mailboxes.  By not getting an NDR, they assume that their list is good and will continue using it.

On a side note, I did find something kind of humerous on the web with regards to email scavenging.  It was a .asp script that allowed for the random and dynamic generation of an infinite number of cross-linked pages filled with fake email addresses...it'd keep dishing them out as long as the scavenging program kept scanning for them.

SpamFilter should facilitate the the flow of good email while blocking the spam.  When the recipient list is not processed completely, the software falls short of this objective.  The difference is SpamFilter needs to potentially make more than one outbound connection to deliver an email message. I thought all email servers did this anyway for outbound email and SpamFilter should be no different as it now supports authenticated outbound and relayed email.

1. SpamFilter currently sends a single NDR on a failure of one or more spam checks per the ini settings.  That single NDR SpamFilter can send per config is the only NDR SpamFilter should continue to send; there is never more than ONE SENDER to report an NDR to.

2. SpamFilter does not need to send multiple NDR's in this case. One NDR is generated if a virus or spam test fails AND the configuration determines whether or not to send NDR for that failure.  The text "relay or delivery to one or more recipients failed" along with the failed recipient list SHOULD be included in the NDR message when one or more recipients in a multiple recipient email is processed and NDR is supposed to be sent.

3.  There is no email splitting.  It's a matter of issuing one connection for each authorized domain when sending out and issuing one rcpt to: command for each recipient in the connected domain.  Copies of an email are sent to forwardable recipient domains.  Forwardable recipient domains are defined as those that do not fail a spam test and whose domain is in the list of allowed domains.

4.  The recipient list SHOULD be sorted by domains and sent in a loop to recipients (rcpt to:) grouped by domains (connections per authorized domain list).

5. If there are recipients in the headers that are not in the allowed domains; they are irrelavant and are ignored.  Spamfilter would have ALREADY refused those rcpt to: commands when the message was received.  SpamFilter could always build it's own alternate recipient x-header based on the rcpt to: commands it accepted on the inbound email receive.

So, in summary, with all of the flexible and powerful features already in this program; adding a feature that provides greater degree of accuracy is an intelligent direction for the product to take. I'm sure I'm missing a bit and overlooking a few gotchas in my ideas above, but this recipient accuracy feature deserves more consideration.

-Matt

>>>>Vrspock...By not getting an NDR, they assume that their list is good and will continue using it.

I disagree. 99.9999% of spammers don't use conventional SMTP servers to send their spam. The programs that they use completely ignore conventional denials in the smtp conversation i.e. they ignore the "550 unknown recipient" reply. Also, do you honestly think that they send a real email address in the "Mail From:" header? Do you think that they actually pickup the NDR's that are sent?  err...no.

After careful investigation of our logs, it is clear to me that by SF accepting the mail, then our systems re-writing the "RCPT TO:" header (hence dropping the recipients that don't exist or don't want spam) we are not getting any more spam attempts than we used to get. Therefore, the spammers aren't sending us even more spam just because we accepted the first email and didn't send an NDR.

This subject is obviously up for debate, but our opinion is that we will block spam from getting to our customers inbox because that's what they pay us to do (apart from the occasional paranoid user who insists on an unfiltered inbox...).

Each of you must do what you feel is right for you. This is right for us.

LogSat wrote: Matt, As usual you have valid points. We are preparing (trying to) a new version (SpamFilter Enterprise) that will have a major new feature.

YES!!!