Print Page | Close Window

Beta questions

Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6333
Printed Date: 30 January 2025 at 10:31pm


Topic: Beta questions
Posted By: WebGuyz
Subject: Beta questions
Date Posted: 06 January 2008 at 5:40pm
The Greylisting beta is VERY imipressive. The question is what kind of problems might we see with use over time. A few questions I'm sure will come up:
 
Where does Greylisting fit in Filter Order? Before whitelists?
 
In the even ANY ip has an issue and we need to make sure it gets through, is there any way to force an IP like adding it to an IP whitelist. Also some SFE users have custom filters for ALL domains so it would have to be a generic text file or table.
 
Also in the SFDC, what do the log entries look like if successful (or not)
 
Thanks for a truly impressive filter (that we've been bugging you for forever LOL)
 
 


-------------
http://www.webguyz.net



Replies:
Posted By: WebGuyz
Date Posted: 06 January 2008 at 7:02pm
Never mind the qestion about the SFDC, it is working.
 
Can you tell me what the SFDC threshold is? How many 'hits' before an entry is quarantined and where does this filter fit in the Filter Order.
 
Thanks!


-------------
http://www.webguyz.net


Posted By: LogSat
Date Posted: 06 January 2008 at 11:43pm
We've updated the filter order at http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5171&PID=11418#11418 - logsat.com/spamfilter/forums/forum_posts.asp?TID=5171&PID=11418#11418 to show the new filter order with all the latest updates.

In regards to the SFDC thresholds, in this first beta there is only one paramenter for it in the various "Filters.ini" files under the \SpamFilter\Domains directories, and it's:
SFDC_Threshold=1

We are overriding that minimm vale on our server by setting it to (currently) 6, however on our SFC server we also take into consideration the separate number of installations that report the same hash, and will only blacklist it if there is a minimum number of SpamFilter's reporting the same hash, and that hash is being send by another minimum of separate source IPs.. We won't go into further details as o not give away the innerworkings of this to spammers, sorry!


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: WebGuyz
Date Posted: 07 January 2008 at 11:22am
Looking at the filter order it does not appear there is a way to manually add an IP that might be having a problem, short of shutting down SFE and manually updating the greylistallowed.txt file and then restarting SFE to read the list it. Haven't had that need occur but today is the first full day of testing and its the busiest
 
Also, would have thought that greylisting would be at the top of the list instead of Blacklist cache.
 
Looking very good ...


-------------
http://www.webguyz.net


Posted By: Desperado
Date Posted: 07 January 2008 at 11:38am
Originally posted by WebGuyz WebGuyz wrote:

Looking at the filter order it does not appear there is a way to manually add an IP that might be having a problem, short of shutting ....
 
WebGuyz,
 
What do you mean exactly?  An IP that may be having a problem?  The Greylist is not supposed to be manually edited as I understand it and it is not an "allow" per se'.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: WebGuyz
Date Posted: 07 January 2008 at 11:49am
Desperado,
 
 Postini uses about 20 (or more) different outgoing IP's to send outbound mail. They rotate those IP's when sending mail to avoid looking like they are spamming when they send hotmail.com or yahoo.com users a bunch of email.
 
Any mail from Postini (there are other ISP's who have banks of ougoing servers with different IP's) may take a LONG time to get to our users for  them to cycle thru all their IP's and in our case we have 2 SFE's (others have more) so it can cause enough of a delay that our customers might complain.
 
Playing 'what if' and trying to be proactive and think of ways that greylisting might be a liability and find a way around them.
 
So far so good. We hold all quarantined email for 3 days and I can see whats going to start happening in about 2-3 days. People will go into the quarantine and notice there are so few entries they will think something is wrong and start calling ...Wink


-------------
http://www.webguyz.net


Posted By: Desperado
Date Posted: 07 January 2008 at 12:04pm
WebGuys,
 
On the first part ... Hotmail and Yahoo and MANY others use many IP's also and it really did not take very long for our "GreyListAllowed.txt" to populate with most of the IP's.  Prior to the population, the delay was only 5 minutes plus a couple of seconds and most of the major services *seem* to be GreyList aware ... meaning that they retried within seconds of the GreyList time-out.  During the "GreyListAllowed.txt" build up, we had ZERO customer complaints ... which frankly did surprise me.
 
On the second part, Grey-Listed messages are not quarantined but rather are rejected with an SMPT reject of "421 This server implements greylisting, please try again in %Time% seconds" where %Time% is the differance between the connection and the "GreyListInterval" (300 seconds by default).  So, I am not sure what you are getting at in the second part of your message.
 
What I can say is I was totally against the GreyList theory since Yahoo started using it but see a HUGE reduction in garbage in my quarantine (2/3 reduction) and a huge reduction in overall server load as a direct result so I am becoming a convert!


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: LogSat
Date Posted: 07 January 2008 at 12:09pm
Currently the greylist file is only imported once when SpamFilter starts up. We were going to change thing so it would be re-imported when it changed, however...:
1 - the filter is working so well, with such almost undetectable delays after a couple hours of implementing it, that we may not see the need for this
2 - this greylist file can easily contain millions of IPs, and allowing SpamFilter to read changes by an external program while SpamFilter itself writes to it may introduce too many problems..

We're leaving things "as-is" right now and we'll see how this filter evolves.

For the order, both the blacklist cache and the greylist will immediately terminate a connection if it doesn't pass the tests. The blacklist cache is smaller, and is thus slightly more efficient to check it first so we can block any spammer that will pound SpamFilter with multiple connection attempts before checking them against the greylist.


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: Desperado
Date Posted: 07 January 2008 at 12:20pm
Originally posted by LogSat LogSat wrote:

The blacklist cache is smaller, and is thus slightly more efficient to check it first so we can block any spammer that will pound SpamFilter with multiple connection attempts before checking them against the greylist.
I thought I would mention that on Jan 1, we were "pounded" by 2.6 MILLION connections from the same IP over a 4 hour period.  SpamFilter handled it so well that I only saw the resulting HUGE log file!  Other than that, the system never seemed to notice or care.

-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: LogSat
Date Posted: 07 January 2008 at 12:27pm
:-)  the "problem" with the blacklist cache is that, since it blocks connections at the TCP level right away, all the spam that would have been received is never seen, and thus, unless looking at the logs, you never see how much spam was really blocked (a lot!!)

-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: kspare
Date Posted: 07 January 2008 at 2:03pm
I have a question regarding the greylistinterval setting. Are most people leaving this at the 300 seconds? I'm wondering if 9 minutes wouldn't be more effective? I know it would make customer mail servers have to try twice but maybe it would help reduce spam just that much more??? Just a thought going through my head.


Posted By: Desperado
Date Posted: 07 January 2008 at 3:44pm
My 2 Cents:
GreyListInterval=420
GreyListLimboHold=8
GreyListAllowedHold=30


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: kspare
Date Posted: 07 January 2008 at 5:39pm
Whats your reasoning Dan? I'm curious if we're on the same page?


Posted By: Desperado
Date Posted: 07 January 2008 at 6:54pm
Nothing very scientific:
 
I felt 5 minuts was a little short but 10 is really too long to wait for a message.
 
The 8 Hour ... Really a server should not wait any longer than 4 hours to retry and most do not wait that long.  I did not want to go too short for fear that messages may NEVER get delivered if the sendmail default of a 4 hour queue flush was in place (most admins speed that up)
 
Last one ... Jury is still out.  I felt that my IP list would get way to big (already at half a million) and it also may be too long to allow possible "bad" ip's to not be grey-listed.  I did not want to go too short because I do not want IP's like hotmail's to have to re-establish a "trust" more often than not.  So ... I still do not know on this value but 90 is longer than I wanted.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: kspare
Date Posted: 07 January 2008 at 6:57pm
Thats kind of what I was thinking too. I'm pretty impressed with how much less spam is even coming through to the queue now...it's very impressive!


Posted By: Desperado
Date Posted: 07 January 2008 at 7:04pm

kspare,

I have fully 1/3 the load on my Database and most of the dictionary attacks and address probes have been nearly eliminated so I think we have a winner here.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: atifghaffar
Date Posted: 07 January 2008 at 7:16pm
Roberto,

When something is runnign so fine, I usually suspect that something is wrong.
Still cant find it though.

kudos.



-------------
best regards

Atif


Posted By: LogSat
Date Posted: 07 January 2008 at 7:28pm
Atif,

I don't know what to say here... this was supposed to be an alpha version as we were just about to start testing it here at LogSat internally. I got tricked into leaking it here on the forums, and that same build then suddenly became a beta. As of now we still did not receive a single bug report on it, so it may as well be promoted to official release...
With this kind of luck, I may just disappear for a few days as I'll be spending them in Las Vegas!!


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: WebGuyz
Date Posted: 07 January 2008 at 7:31pm

The botnet herds really do get stopped with the addition of the SFE version of greylisting.

But thinking about the original Greylisting spec (using triplet data) I think that over time it might be more desireable to go that route instead of just IP.
 
Looking at the logs I see junk coming thru from IP's that have been added to the greylist, probably from compromised mail servers since a mailserver will retry as its supposed to. Once that IP is added all spammers using that IP will have their junk come thru. If the triplet info was used, then it would probably stop more (or less would get thru) over time.
 
Roberto, you know us end user, never happy LOL
 
 


-------------
http://www.webguyz.net


Posted By: LogSat
Date Posted: 07 January 2008 at 7:36pm
As usual, our ears are always open to advice. We'll keep an eye on this, but please do note that with our "flavor" of greylisting, we are greatly reducing the risk of delaying delivery of emails due to the greylisting. yes, the side effect is that more IPs will slip thru, but (1) the other filters should get them, and (2) we can always tweak the greylisting parameters to reduce the number of days (90 by default, which is maybe excessive) permitted IPs remain in the "permitted" state.

-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: Desperado
Date Posted: 07 January 2008 at 7:42pm
Hey Roberto,
 
Pick me up one of those new 150" flat-screen TV's while you are in LV!


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: kspare
Date Posted: 07 January 2008 at 7:43pm
I hear ya Dan, I just reset greylisting to the same settings as you and wow, the difference on my database is just night and day, there is no way you could write a sql script to remove all this automatically and be 100% accurate.

I'd normally have 1000+ spams alone in the queue for me personally and now I have zero, so i'm pretty impressed!


Posted By: kspare
Date Posted: 08 January 2008 at 1:45am
Without a word of a lie. I'm seeing a 90-95% reduction in spam. The 10-5% that make it in are a combination of actual emails and spam that made it through. This is amazing.


Posted By: ImInAfrica
Date Posted: 08 January 2008 at 10:32am
although we are very impressed with the greylisting, i see a lot (ten of thousands) of ip's which are clearly spammers ip's.

I'm thinking the following:
As an ISP we host email for over 2000 domains.
lets say spambots start sending emails at 15:00 and domain 1 is first on the list.
connection is rejected, and delayed for 300 seconds (or whatever the setting is)
even is this ip then tries to reconnect, it will only be allowed to reconnect at 15:05 right?
at 15:01 there are a couple of emails for domain 2 - 10
at 15:02 there are a couple of emails for domains 20-50
and so on.
at 15:05 we start receiving emails from this ip, bypassing the greylist.

The problem here, is that with this method of greylisting if you're places under spam attack for lets same 15 minutes, from the same ip range, then after 5 minutes they've broken through the first barrier. chances are they'll get caught straight away, but this situation is theoretical only.

We've observed that an ip range was sending emails (spam) for various domains, for over 1 hour. not spam flood, just a trickle. however since the greylist (on an installation with a lot of domains) "stops" working after 5 minutes,  it kinda defeats the point.

My suggestion is as follows:
on first connection start counting the time (by default 300 secs)
If further connection attempt are tried BEFORE the full 300 secs have expired, reset the count.
example:
first connect from 196.197.101.101 at 15:00
time to allow ip 15:05
second connect from 196.197.101.101 at 15:01
time to allow ip 15:06
third connect from 196.197.101.101 at 15:05
time to allow ip 15:10
and so on.

a correctly configured smtp server SHOULD not retry in a less then 5 minute period.
this change will greatly reduce the number of spam bots which are bypassing the greylist, as the timeout will continuously increment.  of course this 'may' lead to issues if the connection delay is set to too long.

any thoughts?

Amir


Posted By: Desperado
Date Posted: 08 January 2008 at 10:51am

Amir,

I understand your point and I, myself was worried about the Spammers getting through after the initial timeout was satisfied but I do not really see a meaningful degradation in the effectiveness and I have other filters that grab most of the persistent abusers.  I also feel that simple is better ... both from a functionality / reliability view but also I can't imagine my help-desk guys (and they are very sharp indeed) following the trail of an IP that has a moving target time-out.  Just my 2 cents.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: WebGuyz
Date Posted: 08 January 2008 at 11:05am
The greylisting has helped a lot, but your right, the spammers find a way a usual to circumvent this.
 
I am working on a vbs script to run against previous days log file and extract all IP's of SFDB, SURBL, and AuthorizedTo failures and from this list any sending IP's that have at least 10 failures will get added to our blacklist or possibly to our firewall block list.
 
Also, some of these IP's are from compromised servers so those will always get through the greylist.
 
At least the Bot herders have been slowed down a bit, as that where I see the biggest difference. Fire and forget  spamming is where greylisting really shines.


-------------
http://www.webguyz.net


Posted By: dcook
Date Posted: 08 January 2008 at 3:42pm
 
Suggestion for Beta: I often search the logs for information and it would be helpful if the greylist entry in the logs included the senders email address as well - for ease in quickly tracing greylist false positives.  I usually have the senders email address but not necessarly the IP number.  Also the returned message from the greylist should be customizable in the final release.
Will the greylisting be per domain in enterprise spamfilter?
 


-------------
Dwight
www.vividmix.com


Posted By: LogSat
Date Posted: 08 January 2008 at 4:02pm
The greylisting occurs at the TCP level right after a connection attempt is detected. The server is disconnected before they even have a chance to output the commands that specify the sender and the recipient. For this reason, neither the "from" nor the "to" domains are known, and thus the filter can't be customized per domains in SFE, nor can we log that in the SpamFilter logs, sorry!

-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: dcook
Date Posted: 08 January 2008 at 5:55pm
I really knew that answer before I asked it. But I always want the impossible, don't you?
I am already getting real fast at log trace now and you do have a winner here! 
 
I think the 150" TV is much better than the new thin 11" to watch.  Enjoy CES Roberto!


-------------
Dwight
www.vividmix.com


Posted By: LogSat
Date Posted: 09 January 2008 at 1:44pm
FYI - an updated beta is available in the registered user area.

-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: dcook
Date Posted: 10 January 2008 at 3:00pm
I know we don't have an enterprise version for greylisting -- but I see we need it.  I have had several clients say they are getting bounces.  Here is what happened:
1. First email is sent to MX-10 and greylisting starts
2. Second retry email is send to MX-20
3. No more retry's - email is non-deliverable
 
First of all, the mail server should be more persistant trying more than twice for delivery. Second if enterprise servers shared the greylist - it would not matter.
 
Since I can't change the sender, I suggest that the next release include enterprise database storage of the greylist, please. Thanks
 
 


-------------
Dwight
www.vividmix.com


Posted By: Desperado
Date Posted: 10 January 2008 at 3:09pm
Dwight,
 
The enterprise mode does greylist but shares a single file.  I am not seeing the same issue you are.  Can you expand on it a little?


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: WebGuyz
Date Posted: 10 January 2008 at 3:12pm

I use greylisting with 2 SFE's. I thought the same thing, that there might be instances of mail not getting through, but so far (since Sunday afternoon) I have not had anyone complain.

Would be nice to share a greylist database, but the question becomes are we willing to put up with a performance penalty. And if a greylist db became a reality, I would like to see a more standard version of greylisting using triplet info.

Perfomance penalty to me means its just time to move SFE's to beefier hardware LOL


-------------
http://www.webguyz.net


Posted By: Desperado
Date Posted: 10 January 2008 at 3:39pm
My 2.5 Cents.
 
I worry about the triplet setup in the case of some of our domains that have 2-3 thousand accounts.  2 problems here ... First, I am very happy that it only took a day or so to get most, if not all of AOL's, Hotmail's and the other "big guys" IP's to populate the list and therefore not delay messages to our customers.  This would not be the case if the "triplet" setup was used.  Second, I already have well over half a million lines in my GreyListAllowed.  I can not imaging how large it would get if the triplet method was used.
 
On the other side of the fence, the triplet *seems* like it would be more effective over the long term ... seeing that Spammers always figure out what is going on and always adapt. 
 
So ... which is best?  dunno yet.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: dcook
Date Posted: 10 January 2008 at 3:45pm

There I go assuming things again - I thought the limbo or temporary greylist was NOT shared. 

I have a client that complained about getting bounces.  They are a mortgage company and can't afford to miss clients emails.  I did a trace and found that several of their missing emails went to our first MX record (SF) and were put in the limbo grey area.  The next email went to another MX (different SF) and it too was put into the grey limbo. So the email source saw the greylist messages and said they were being bounced.  My mortgage company never got the email from the sender.

I had the mortgage company in a separate individual configuration of SFE but since the greylist is a global configuration, I could not tweak this clients email settings.  For the time being I am just forwarding all of their email unfiltered directly to their exchange server.

I have not had complaints from other clients on the greylist.

 



-------------
Dwight
www.vividmix.com


Posted By: Desperado
Date Posted: 10 January 2008 at 3:56pm
Dwight,
 
So the sender stopped attemting to send after 2 rapid fire attempts?  They should have retried *at least* for a couple of days ... even 1 day.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: dcook
Date Posted: 10 January 2008 at 4:00pm

Some "so called" IT people can't help but tweak those mail server settings.  (smile)



-------------
Dwight
www.vividmix.com


Posted By: LogSat
Date Posted: 10 January 2008 at 4:15pm
Originally posted by dcook dcook wrote:

I know we don't have an enterprise version for greylisting -- but I see we need it.  I have had several clients say they are getting bounces.  Here is what happened:
1. First email is sent to MX-10 and greylisting starts
2. Second retry email is send to MX-20
3. No more retry's - email is non-deliverable
 
First of all, the mail server should be more persistant trying more than twice for delivery. Second if enterprise servers shared the greylist - it would not matter.


Per RFC2821, when there are multiple MX records, the sender MUST try sending an email to all servers in the list if one is unavailable. This would mean that if an attempt is made to the primary MX server, and this fails due to the graylisting, the remote server should then immediately try connecting to the secondary server. The relevant section of the RFC is:

   When the lookup succeeds, the mapping can result in a list of
alternative delivery addresses rather than a single address, because
of multiple MX records, multihoming, or both. To provide reliable
mail transmission, the SMTP client MUST be able to try (and retry)
each of the relevant addresses in this list in order, until a
delivery attempt succeeds. However, there MAY also be a configurable
limit on the number of alternate addresses that can be tried. In any
case, the SMTP client SHOULD try at least two addresses.

Now, this process will fail for the first attemp due to the greylisting. The same RFC2821 now states that the sender MUST retry a particular destination if the first attempt has failed. This means that the remote server MUST retry sending the email to the primary MX server. if you see servers that are not being able to deliver mail to you in your configuration, it's very likely that they are violating this RFC, which is considered the RFC when it relates to emails. The relevant section here is:

   The sender MUST delay retrying a particular destination after one
attempt has failed. In general, the retry interval SHOULD be at
least 30 minutes; however, more sophisticated and variable strategies
will be beneficial when the SMTP client can determine the reason for
non-delivery.

Originally posted by Desperado Desperado wrote:

Dwight,
 
The enterprise mode does greylist but shares a single file.  I am not seeing the same issue you are.  Can you expand on it a little?

Dan, I have to correct you on this one. Each SpamFilter uses its own copy of the greylist file, and that file is only imported once when SpamFilter startup. Unlike all other configuration files, this one is not reloaded by SpamFilter if modified externally.



Originally posted by WebGuyz WebGuyz wrote:

Would be nice to share a greylist database, but the question becomes are we willing to put up with a performance penalty. And if a greylist db became a reality, I would like to see a more standard version of greylisting using triplet info.
Perfomance penalty to me means its just time to move SFE's to beefier hardware LOL

WebGuyz, using the full triplet would be potentially a disaster waiting to happen in large installations. If, in addition to keeping track of individual IPs, we also kept track of the to/from email addresses for all emails regarding that IP, the number of entries in the list would row exponentially, going from the millions of entries you will be reaching now, to the billions. SpamFilter is efficient, but that may be too much for a dinky 3MB executable!


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: Desperado
Date Posted: 10 January 2008 at 4:24pm
Roberto,
 
I was refering to the single file in the SFE configurating which I thought was shared by ALL domains under that same SFE instance.  Am I wrong there?  I did not mean that the file itself could be shared by differant servers instances.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: LogSat
Date Posted: 10 January 2008 at 4:29pm
Originally posted by Desperado Desperado wrote:

Roberto,
 
I was refering to the single file in the SFE configurating which I thought was shared by ALL domains under that same SFE instance.  Am I wrong there?  I did not mean that the file itself could be shared by differant servers instances.

Misunderstood you. Yes, in this case you're absolutely correct. The file will apply to the entire SE installation, for all the domains it handles.


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: Desperado
Date Posted: 10 January 2008 at 4:32pm
Whew!  Thought I had lost my mind! ... Well ... that may still apply!Geek

-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: WebGuyz
Date Posted: 10 January 2008 at 4:38pm
Originally posted by LogSat LogSat wrote:


WebGuyz, using the full triplet would be potentially a disaster waiting to happen in large installations. If, in addition to keeping track of individual IPs, we also kept track of the to/from email addresses for all emails regarding that IP, the number of entries in the list would row exponentially, going from the millions of entries you will be reaching now, to the billions. SpamFilter is efficient, but that may be too much for a dinky 3MB executable!
 
Like I said, Wink way beefier hardware Wink
Think I saw a old CRAY super computer available on Ebay or CraigsList ....


-------------
http://www.webguyz.net


Posted By: WebGuyz
Date Posted: 10 January 2008 at 4:48pm
I remember another poster talking about a greylisting proxy server in front of all their SF servers. Maybe that would be a better way to go then trying to share it in a SFE table. The technology definitely works, but is it scalable is the question.

-------------
http://www.webguyz.net


Posted By: atifghaffar
Date Posted: 11 January 2008 at 12:20am
Webguyz,

The proxy will have to proxy the connection to keep all the connection information. (so all other tests based on the ip address can be performed)

My solution was with a NATting firewall.

I still have a firewall in front of the SF boxes but it now does just the blocking based on the limbo information.

Dont know how this can be done in windows.

The rules are quiet simple.
if ip in limbo drop
if ip in greylistok forward to another ip

I can share my firewall script if someone is willing to implement it in windows.



-------------
best regards

Atif


Posted By: WebGuyz
Date Posted: 11 January 2008 at 12:30am
Was going to try this transparent proxy: http://www.hermes-project.com/pages/hermes - http://www.hermes-project.com/pages/hermes
But then I realized there still is no easy way to go from a single central greylisting proxy to 2 or more SFE servers unless you get something like a load balancer.
 
Don't really want to implement this in firewall as I may be uprading mine in the near future.  Will keep looking around and fine tuning.
 
I marvel at how much of my time is spent trying to stop spam. What a shame ...Dead


-------------
http://www.webguyz.net


Posted By: dcook
Date Posted: 11 January 2008 at 10:57am

 

 

Originally posted by Desperado

Roberto,

 

I was referring to the single file in the SFE configurating which I thought was shared by ALL domains under that same SFE instance.  Am I wrong there?  I did not mean that the file itself could be shared by differant servers instances.


Misunderstood you. Yes, in this case you're absolutely correct. The file will apply to the entire SE installation, for all the domains it handles.

 

Each spam filter server currently keeps it own independent grey list. In our case, in this spam filter build SFE-1 and SFE-2 keep their own, separate grey list that is used globally per server. The information is never shared between SFE-1 and SFE-2.

 

Roberto did you say that in a future build grey list sharing for the enterprise version using the database should be supported?

 


-------------
Dwight
www.vividmix.com


Posted By: LogSat
Date Posted: 11 January 2008 at 5:07pm
Originally posted by dcook dcook wrote:

 

Roberto did you say that in a future build grey list sharing for the enterprise version using the database should be supported?

 

Originally that was the plan, but that was during our early development stages, before we had a chance to see how well this beta worked :-)
For now we'll wait and see, as storing that list in a database does concern us due to its size and the frequency of updates to it.


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: atifghaffar
Date Posted: 11 January 2008 at 6:19pm
Roberto,
How about an option for the admin to decide  the interval when to flush the greylist to the db. and how often it should be reloaded.
I think a "select ip from table where id>max_id_we_have" every 10 minutes will not be so resource intensive.




-------------
best regards

Atif


Posted By: Guests
Date Posted: 15 January 2008 at 7:04am
Do have a little error that seems to be returning:
 
01-15-08 12:59:25:096 -- (4340) Exception occurred during TTimerMinuteTimerThread.DoUpdateChartCountries: Access violation at address 00401D3D in module 'SpamFilterSvc.exe'. Write of address 00000001


Posted By: LogSat
Date Posted: 15 January 2008 at 6:12pm
sysiq, can you please let us know what build of SpamFilter you are using?

-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: caratking
Date Posted: 15 January 2008 at 11:54pm
Just tossing out an idea here...

Currently configuration information is in a database and shared by all servers.

If you want to share your grey list, putting millions of IP Addresses into a database is probably going to be a BIG hit on performance.

Why not enable server to server communication directly between spamfilter machines?  There is no configuration, no changes are done by the administrator.  It is only the machines that need to share the information.  Have them do a sync directly between themselves.

Servers are registered in the database anyway, have each server update the database with its IP Address and some type of random key.  The servers can use that info to find each other and the key to ensure they are taking with who they think they are.


Thats it...




Posted By: Simone
Date Posted: 23 January 2008 at 1:29pm
A little question:

On SFE the Grey Listing could be apllied to all domains or you can choose wich one use and wich one will not?

Reading your post it seems the first option the right one, but will the second be possible in the future?

Thank you,
Simone


Posted By: Desperado
Date Posted: 23 January 2008 at 1:36pm
Simone,
 
The GrayListing works at the SMTP level and as a result, it is applied to ALL domains on the SFE instance.


-------------
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com



Posted By: WebGuyz
Date Posted: 24 January 2008 at 9:38am
I love getting emails like this from our customers ;-)
 
(He's referring to the amount of spam in quarantine that he has to slog thru)
 
----------------------------------------
I have noticed a huge decrease in the number of spams I get every day. It seems like about 60% have gone away.

Did you put them all in jail, or have they given up on me?

I don't want them back, I am just curious as to what has changed.

Thanks

Chuck
-----------------------------------------



-------------
http://www.webguyz.net


Posted By: kspare
Date Posted: 24 January 2008 at 10:39am
haha i've been getting the opposite emails. People are wondering if we've been having problems because they don't have the usual amount of spam! haha



Print Page | Close Window