SpamFilter v2.7 with IP cache is avail
Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5413
Printed Date: 05 February 2025 at 1:07pm
Topic: SpamFilter v2.7 with IP cache is avail
Posted By: LogSat
Subject: SpamFilter v2.7 with IP cache is avail
Date Posted: 11 December 2005 at 9:13pm
|
We have pre-released in the registered user area of our website a
newbuild of SpamFilter ISP v2.7.1.508. The major additions to this
version are (1) a new feature that stores in a memory cache a list of
IP addresses that have been blocked, and (2) a greatly improved
Connections tab showing in realtime what commands the remote IPs are
sending The new IP Cache: After an IP address send a spam/virus email, it will be added to an IP "limbo" cache. If an IP in the limbo cache sends more that a certain number of spam emails (3) during a certain amount of time (10 minutes), that IP will be moved to a temporary "blacklist cache", and from that point on all connections from that IP will be immediately rejected. The IP will be automatically be removed from the cache after a period of time (60 minutes) and will be thus given more chances to send "clean" emails. Important! Any addresses in the DoNotAddIPToHoneypot list will also not be added to this blacklist cache, thus preventing the blocking of "friendly" servers. This IP cache should greatly reduce the load on the SpamFilter server because since the connection is rejected, the IP will never have a chance to send an email, saving CPU and bandwidth resources. All of the above parameters in parenthesis can be user-configured by changing their values in the SpamFilter.ini file as follows: ;If an IP sends more than this number of spams in a certain period of time then it is temporarily banned (blacklisted) IPCacheLimboCountTrigger=3 ;If an IP sends more than a certain number of spams during this number of minutes then it is temporarily banned (blacklisted) IPCacheLimboTimeTrigger=10 ;If an IP address was banned because it sent too many spams in a certain time interval, it will be un-banned after this number of minutes IPCacheBlacklistDuration=60 The release notes for the latest builds are as follows: // New to VersionNumber = '2.7.1.508'; {TODO -cNew : Implemented an IP cache to temporarily deny further connections to IPs that sent multiple spams recently. This can greatly reduce the load on the server} {TODO -cNew : Improved "Connections" tab, showing in real-time what commands the remote IPs are sending} {TODO -cFix : Sometimes the "Current Connections" counter could not decrease when a remote connection is dropped, thus displaying a number higher than reality} // New to VersionNumber = '2.6.3.502'; {TODO -cFix : Duplicate entries were being created in the logfiles} {TODO -cFix : Bug introduced in v2.6.3.491. When forwarding emails to the destination SMTP server, sometimes the leading "<" and trailing ">" where missing in the MAIL FROM} // New to VersionNumber = '2.6.3.495'; {TODO -cNew : Added options to not quarantine or send to NULL viruse-infected emails} // New to VersionNumber = '2.6.3.493'; {TODO -cNew : Added DNSTimeout option in SpamFilter.ini to customize the DNS timeout for all of SpamFilter's DNS queries} {TODO -cNew : Added EnableDbgLogs SpamFilter.ini option to enable separate detailed logging for troubleshooting purposes} {TODO -cNew : Added to SpamFilter.ini several of the optional entries with their default values for users to see} {TODO -cFix : Clicking on "Check if IP in ORBS" button in GUI could result in Access Violations being logged} // New to VersionNumber = '2.6.3.491'; {TODO -cNew : Added support for maximum message size in reply to EHLO and MAIL FROM, as per RFC1870} ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Replies:
Posted By: WebGuyz
Date Posted: 12 December 2005 at 2:32pm
|
Does this include 'Not in AuthorizedTo List' failures?? <fingers crossed> ------------- http://www.webguyz.net |
Posted By: LogSat
Date Posted: 12 December 2005 at 4:18pm
How could we ignore your pleas for help on the "Not in Authorized list" problems...! Yes, 3 attempts from the same IP within 10 minutes will cause the sender's IP to be blacklisted in cache for 60 minutes. They will be immediately disconnected from then on without even giving them a chance to issue any commands. All these parameters are configurable in the spamfilter.ini file. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: Guests
Date Posted: 12 December 2005 at 5:25pm
|
So the squeaky wheel does get the grease. |
Posted By: Guests
Date Posted: 12 December 2005 at 9:01pm
|
I have run into a small issue, unless I'm mis-reading this. Below are my current settings. What I'm finding is the IP's are being released from cache blacklist after 10 minutes, not 60 like I thought my setting would give me. Also, I have to get a whole bunch of rejects (more then what I set at 8) before SF puts the IP in limbo. From the readme I infered that my setting mean: if 8 rejects occur in 10 minutes, the IP would be put in limbo for 60 minutes. But if thats the case, thats not what is happening. Andy
----------spamfilter.ini------------ |
Posted By: LogSat
Date Posted: 12 December 2005 at 9:32pm
|
Andy, Using your settings, let me describe what SpamFilter will do. You receive a spam email from a sender. The IP address of the sender will immediately be added to the limbo cache. From the time it's added to the limbo, should that IP address send 7 more spam emails within 10 minutes from the 1st one (a total of 8 spams within 10 minutes), then the IP address will be removed from the limbo and will be added to the blacklist cache, where it will remain for 60 minutes. All connections from IP addresses in the blacklist cache will be immediately rejected. The limbo is just a temporary location that holds all IP addresses that recently sent spam, connections will *not* be rejected simply due to the presence of the IP in the limbo. Consider the limbo halfway between good and evil... The senders are not totally innocent, but they won't be punished immediately. If they insist on behaving badly, they'll be treated as evil. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: Guests
Date Posted: 12 December 2005 at 9:43pm
|
aha!! But it seems like it takes a while for it to kick in. I'll have 50 AuthorizedTo rejects fly by in the log before I even see it put the IP into limbo. Will study the logs some more this evening and tomorrow. I think I like it better than greylisting as a means to cut down on spammer connections. THANKS! |
Posted By: LogSat
Date Posted: 12 December 2005 at 9:56pm
|
Please note that if the spammer is attempting multiple (50) RCPT TO
commands to send the email to multiple recipients in a single mail
session, then when it will be rejected that will only count as a single
connection... If that's the case, you may want to limit the maximum
number of recipients allowed in a single session, so that the spammer
gets dropped sooner.
------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: Guests
Date Posted: 12 December 2005 at 10:08pm
|
Starting to come to that same conclusion looking at the log. I have to keep the recipients allowed in single session high becaue some of our larger customers are on mailing lists and some have a LOT of mailboxes. I was getting complaints from the senders that half their emails were being returned to them. <sigh> Can't make everyone happy. |
Posted By: Guests
Date Posted: 13 December 2005 at 10:25am
|
Do we have access to the IP Blacklist Cache, or is it builtin to SF? I would like to build a log parser to gather IP's from dictionary attacks but have never had a way to use it since the AuthorizedTo check came early in the filter list(before block IP's). If I was able to feed the IP Blacklist cache manually I could fix this. Is it possible? or not? THANKS! |
Posted By: Desperado
Date Posted: 13 December 2005 at 10:39am
|
The IP Cache is not a file based cache. It can be viewed under the "Statistics" tab. I was going to wait at least a full week before I started begging for features! I rarely am "on" my server so everything I do is either SQL queries or log parsing. I am trying to find time to make modifications to the SawMill log parse file to extract the information out and then an export of that data may help you. However, the whole point is that the spammers are in "Hit & Run" mode so that the value may be limited. Per Roberto, The log entries are as follows:
12/11/05 13:01:31:515 -- (2272) Connection from: 172.27.4.50 - Originating country : N/A
12/11/05 13:01:31:515 -- (2272) IP is in local blacklist cache. Disconnecting: 172.27.4.50 12/11/05 13:01:32:468 -- (2272) No Data Received 12/11/05 13:01:32:468 -- (2272) Disconnect So ... should be easy to parse out. ------------- The Desperado Dan Seligmann. Work: http://www.mags.net Personal: http://www.desperado.com |
Posted By: Guests
Date Posted: 13 December 2005 at 10:51am
|
I'm not too proud to beg Roberto, how about a txt file that gets checked periodically and if it finds any IP in there it adds it to the internal IP Blacklist Cache. I would even do my own cleanup of this text file. I use MS Logparser with excellent results and could easily harvest the harvesters IP's and add them myself. I'm shocked at how well these guys are organized and the hit and run tactics you mentioned are definitly there to get around most spam checkers. They will try a handful of addresses with one IP and switch to another IP in the blink of an eye and they have a LOT of IP's so if I have my blacklist cache timer set to 10 minitues, they can get around this my using many, many different IP's. But I would have them in the log and their IP's would be mine to blacklist. I checked some of the IP's and they are cable connection or residential accounts. I really hate these guys .... |
Posted By: LogSat
Date Posted: 13 December 2005 at 8:11pm
|
The cache is in memory only, and is emptied when SpamFilter is
restarted. There's really no plans to allow updating it with external
data. Can I ask why you do not use the already-available local IP
blacklist to reject spammers? If that list is modified, SpamFilter will
automatically reload it. As a side note, we do realize that the blacklist IP check is performed after many other tests, thus needlessly wasting resources. What we could do it to prioritize it, so that blacklisted IPs are tested and rejected before other tests occur (ex. before MAPS, SURBL, MX, Reverse DNS, SPF, AuthorizedTo). Would this help? ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: Guests
Date Posted: 13 December 2005 at 9:05pm
|
Roberto, Putting the IP Blacklist test at the top of the filter list would be a very good thing. |
Posted By: Guests
Date Posted: 16 December 2005 at 8:44pm
|
Will the prioritized IP Blacklist be in 2.7.1.510 ? |
Posted By: LogSat
Date Posted: 16 December 2005 at 10:12pm
|
It didn't make it in 510, but it did in 511 which we just uploaded,
along with a couple of tweaks that should increase performance if the
ReverseDNS filter is not used. The release notes for 511 are: // New to VersionNumber = '2.7.1.511'; {TODO -cNew : Changed the priority of the IP blacklist filter, it is now placed before the local domains blacklist} {TODO -cNew : Changed the logfile entry if the IP address is blacklisted to: "IP is in local blacklist file..."} {TODO -cNew : Performing reverse DNS queries only if the ReverseDNS filter is enabled, thus improving performance when it's off} ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |