Logging Question
Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5439
Printed Date: 22 December 2024 at 3:38pm
Topic: Logging Question
Posted By: pcmatt
Subject: Logging Question
Date Posted: 10 January 2006 at 10:53pm
|
"IP is in local blacklist file..." appears in my log files since some new "feature" was added in realese 511: {TODO -cNew : Changed the logfile entry if the IP address is blacklisted to: "IP is in local blacklist file..."} Anybody know what this message is actually for ? There are Customized Items for every scenario I thought that should be used for errors and log entries? Is SpamFilter now disregarding use of certain Customized items or am I missing something? Thanks! -Matt ------------- -Matt R |
Replies:
Posted By: LogSat
Date Posted: 10 January 2006 at 11:49pm
|
Previously the check to see if the remote IP was listed in the "Local IP blacklist" was performed just prior the MAPS and other DNS-based tests. It made more sense, performance-wise, to do that test before the other DNS-based ones. In the past, when the local IP blacklist test gave a match, it was incorrectly being logged within the MAPS log entry as follows: MAPS search done... The IP %IP% is Blacklisted. Now the match will be logged, more correctly, with a "IP is in local blacklist file" log entry. The remote server will however still receive the same customized error message as before. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: pcmatt
Date Posted: 11 January 2006 at 8:03am
|
So what then are these customized items for in SpamFilter.ini and in the Customized Items screen: ResponseBlacklistLocalIP= ResponseIPCacheBlacklist= It makes no sense to not use those configurable items. It also makes no sense to send one message and to log something different. This looks like either an oversight during all the recent major changes that were done, or simply unfinished changes. -Matt
------------- -Matt R |
Posted By: LogSat
Date Posted: 11 January 2006 at 8:17am
|
Actually I still don't see the problem. The two customized responses above determine what error message the remote server will receive when their connection is rejected. This has not changed for the existing "ResponseBlacklistLocalIP". Since we added a new filter (the Cache blacklist), we thus also added one more customizable item for it so that you can decide to customize the error that the remote server will receive when it is disconnected by SpamFilter when this new filter is triggered. This is a new feature. What *has* changed is the way we log, for our own internal troubleshooting purposes, the *event* that occurs when SpamFilter disconnects a server because its IP is in the "Blacklist Local IP". The log entry, and only that, has changed so that its easier to recognize this event from the logs. The remote server will not see any change. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: pcmatt
Date Posted: 11 January 2006 at 8:34am
|
So you think having different messages in the log for just a few scenarios instead of a consistent approach is LESS confusing? I'm going to need two more cups of coffee to catch up with your logic ------------- -Matt R |
Posted By: pcmatt
Date Posted: 11 January 2006 at 8:37am
|
While I'm drinking that coffee can you please tell me what I should see in the log if someone is blocked by an entry in the IP Cache? ------------- -Matt R |
Posted By: LogSat
Date Posted: 11 January 2006 at 10:29am
|
Matt, Earlier the event of the IP being blacklisted was logged under a "MAPS search done... The IP aa.bb.cc.dd is Blacklisted" event. Now I'd say that *that* was confusing, as MAPS RBL DNS searches have nothing to do with an IP being in a local IP blacklist. Furthermore, before the IP local blacklist checks were performed during a MAPS DNS test, which was very, very inefficient. I believe that now the new log entry is very specific and consistent with its action. And also remember that the IP blacklist check is now being performed right after the connection is made, many, many "filters" before the MAPS check occurrs. Also, the log entry "IP is in local blacklist file" is the perfect exaplanation of what is happening. The IP is in a local blacklist file, and will be disconnected. I still do not see where the problem/confusion is. In regards to your second question, the log entry if the remote IP is in the new cached blacklist, the entry will be: IP is in local blacklist cache. Disconnecting: aa.bb.cc.dd ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: pcmatt
Date Posted: 11 January 2006 at 11:35am
|
So for all of the other scenarios you log the message that is sent to the sender, EXCEPT these two scenarios where you write your new hard coded message into the log and send the sender the Customized Item message. The problem is that this is inconsistent and I have no way to be confident that the sender received the customized item message because my log does not match what was sent to the sender. It's not that your hard coded log entries are not a good explanation of what happened, it's that they are not consistent with all the other scenarios and they are not what was sent to the sender, which I would prefer to see in the log like all other scenarios. If you now prefer to log a hard coded message instead of what was sent to the sender, why have you not hard coded perfectly good explanations for all other scenarios that we have Customized Items for? (please don't do this -Matt ------------- -Matt R |
)Posted By: LogSat
Date Posted: 11 January 2006 at 12:58pm
Matt, I believe that is incorrect. Following are some of the log description entries when some filters are failing (I'm copying from our source code..). The SendMsg function logs the entry to the logfile, the S variable gets sent to the remote server. As you see we're very consistent. One explanation in the logfile describing what happened, then the customized response to the sender. TSyncSendResult2.SendMsg(ASender.Thread, FromTO + ' - rejected - no relay allowed or % found in FROM address'); S:=FormatErrorResponse(FResponseRelayRestricted, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- IP is in local blacklist file...'); S:=FormatErrorResponse(FResponseBlacklistLocalIP, PeerIP, '', '', '', ''); TSyncSendResult2.SendMsg(ASender.Thread, '- Domain is in local blacklist file...'); S:=FormatErrorResponse(FResponseBlacklistLocalDomain, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- EmailFrom is in local blacklist file...'); S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- EmailTO is in local blacklist file...'); ConnectionDetails.RejectID:=9; EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- EmailTO is not in AuthorizedTOEmail list...'); s:=FormatErrorResponse(FResponseRelayRestricted, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- IP address is from a blacklisted country...'); S:=FormatErrorResponse(FResponseCountryBlacklist, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- Reverse DNS not found - '); ConnectionDetails.RejectID:=3; EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- Empty Mail From - '); S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- Mail From and Mail To are equal - '); S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- EMail To is in honeypot emails - '); S:=FormatErrorResponse(FResponseBlacklistLocalIP, PeerIP, '', '', '', ''); TSyncSendResult2.SendMsg(ASender.Thread, '- IP blocked by honeypot autofilter - '); S:=FormatErrorResponse(FResponseBlacklistLocalIP, PeerIP, '', '', '', ''); TSyncSendResult2.SendMsg(ASender.Thread, '- Mail From Domain and Mail To Domain are equal - '); S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, 'Exceeded maximum number of RCPT TO (' + IntToStr(ConnectionDetails.RecipientCount)+ ') - Disconnecting ' + PeerIP); S:=FormatErrorResponse(FResponseMaxRCPTTO, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); TSyncSendResult2.SendMsg(ASender.Thread, '- Invalid MX record - ' + tmpResult); S:=FormatErrorResponse(FResponseNoMX, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, ''); ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
pcmatt wrote:Posted By: pcmatt
Date Posted: 13 January 2006 at 3:30pm
|
Got it so now only "- MAPS search done... " concatenates the customized message in the logs. Makes sense, so we know which MAPS entry caused the failure. Since the MAPS entry used to be used for the local blacklist also, I was used to looking for the full string that included my customized message. I guess I just need to start the coffee earlier in the day. Thanks. -Matt ------------- -Matt R |
Posted By: LogSat
Date Posted: 13 January 2006 at 3:37pm
|
Yeap, exactly. It was actually bad programming on our part before having placed the IP blacklist lookup embedded in the MAPS searches as well. Years ago, without all the filers SpamFilter has today, it made it simpler, but that is just an excuse... It was not its place.
------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: pcmatt
Date Posted: 17 January 2006 at 1:02pm
|
You removed the standard log line: Resolving Ipaddress - hostname Big gaping hole in my reporting now; Bummer. Any plans on updating the quarantine database so that it contains the current set of Rejection Codes?
------------- -Matt R |
Posted By: LogSat
Date Posted: 17 January 2006 at 10:32pm
|
Matt, You can find the answer to your first question in the other thread at http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5449 - http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID= 5449. For the second question, I'm not sure I follow. SpamFilter will (or better *should*...) automatically add all missing reject codes from the database when the program is started. Right now there should be 19 codes, the last one being the "URL in email found in SURBL search". Is this what you were asking? ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |