Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Logging Question
  FAQ FAQ  Forum Search   Register Register  Login Login

Logging Question
 Post Reply Post Reply
Author
  Topic Search Topic Search  Topic Options Topic Options
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116 		Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Topic: Logging Question
    Posted: 10 January 2006 at 10:53pm

"IP is in local blacklist file..." appears in my log files since some new "feature" was added in realese 511:

{TODO -cNew : Changed the logfile entry if the IP address is blacklisted to: "IP is in local blacklist file..."}

Anybody know what this message is actually for ?  There are Customized Items for every scenario I thought that should be used for errors and log entries?  Is SpamFilter now disregarding use of certain Customized items or am I missing something?

Thanks!

-Matt
-Matt R
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2006 at 11:49pm
Previously the check to see if the remote IP was listed in the "Local IP blacklist" was performed just prior the MAPS and other DNS-based tests. It made more sense, performance-wise, to do that test before the other DNS-based ones.
In the past, when the local IP blacklist test gave a match, it was incorrectly being logged within the MAPS log entry as follows:

MAPS search done... The IP %IP% is Blacklisted.

Now the match will be logged, more correctly, with a "IP is in local blacklist file" log entry.

The remote server will however still receive the same customized error message as before.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116 		Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2006 at 8:03am

So what then are these customized items for in SpamFilter.ini and in the Customized Items screen:

ResponseBlacklistLocalIP=

ResponseIPCacheBlacklist=

It makes no sense to not use those configurable items.  It also makes no sense to send one message and to log something different.  This looks like either an oversight during all the recent major changes that were done, or simply unfinished changes.

-Matt

 
-Matt R
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2006 at 8:17am
Actually I still don't see the problem. The two customized responses above determine what error message the remote server will receive when their connection is rejected. This has not changed for the existing  "ResponseBlacklistLocalIP". Since we added a new filter (the Cache blacklist), we thus also added one more customizable item for it so that you can decide to customize the error that the remote server will receive when it is disconnected by SpamFilter when this new filter is triggered. This is a new feature.

What *has* changed is the way we log, for our own internal troubleshooting purposes, the *event* that occurs when SpamFilter disconnects a server because its IP is in the "Blacklist Local IP". The log entry, and only that, has changed so that its easier to recognize this event from the logs. The remote server will not see any change.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116 		Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2006 at 8:34am

So you think having different messages in the log for just a few scenarios instead of a consistent approach is LESS confusing?

I'm going to need two more cups of coffee to catch up with your logic

-Matt R
Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116 		Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2006 at 8:37am
While I'm drinking that coffee can you please tell me what I should see in the log if someone is blocked by an entry in the IP Cache?
-Matt R
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2006 at 10:29am
Matt,

Earlier the event of the IP being blacklisted was logged under a "MAPS search done... The IP aa.bb.cc.dd is Blacklisted" event. Now I'd say that *that* was confusing, as MAPS RBL DNS searches have nothing to do with an IP being in a local IP blacklist. Furthermore, before the IP local blacklist checks were performed during a MAPS DNS test, which was very, very inefficient.

I believe that now the new log entry is very specific and consistent with its action. And also remember that the IP blacklist check is now being performed right after the connection is made, many, many "filters" before the MAPS check occurrs. Also, the log entry "IP is in local blacklist file" is the perfect exaplanation of what is happening. The IP is in a local blacklist file, and will be disconnected. I still do not see where the problem/confusion is.

In regards to your second question, the log entry if the remote IP is in the new cached blacklist, the entry will be:

IP is in local blacklist cache. Disconnecting: aa.bb.cc.dd
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116 		Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2006 at 11:35am

So for all of the other scenarios you log the message that is sent to the sender, EXCEPT these two scenarios where you write your new hard coded message into the log and send the sender the Customized Item message. 

The problem is that this is inconsistent and I have no way to be confident that the sender received the customized item message because my log does not match what was sent to the sender. 

It's not that your hard coded log entries are not a good explanation of what happened, it's that they are not consistent with all the other scenarios and they are not what was sent to the sender, which I would prefer to see in the log like all other scenarios.

If you now prefer to log a hard coded message instead of what was sent to the sender, why have you not hard coded perfectly good explanations for all other scenarios that we have Customized Items for? (please don't do this )

-Matt
-Matt R
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2006 at 12:58pm
Originally posted by pcmatt pcmatt wrote:

So for all of the other scenarios you log the message that is sent to the sender, EXCEPT these two scenarios where you write your new hard coded message into the log and send the sender the Customized Item message. 



Matt,

I believe that is incorrect. Following are some of the log description entries when some filters are failing (I'm copying from our source code..). The SendMsg function logs the entry to the logfile, the S variable gets sent to the remote server. As you see we're very consistent. One explanation in the logfile describing what happened, then the customized response to the sender.

             TSyncSendResult2.SendMsg(ASender.Thread, FromTO + ' - rejected - no relay allowed or % found in FROM address');
             S:=FormatErrorResponse(FResponseRelayRestricted, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- IP is in local blacklist file...');
             S:=FormatErrorResponse(FResponseBlacklistLocalIP, PeerIP, '', '', '', '');


             TSyncSendResult2.SendMsg(ASender.Thread, '- Domain is in local blacklist file...');
             S:=FormatErrorResponse(FResponseBlacklistLocalDomain, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');


             TSyncSendResult2.SendMsg(ASender.Thread, '- EmailFrom is in local blacklist file...');
             S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- EmailTO is in local blacklist file...');
             ConnectionDetails.RejectID:=9;
EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- EmailTO is not in AuthorizedTOEmail list...');
             s:=FormatErrorResponse(FResponseRelayRestricted, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- IP address is from a blacklisted country...');
             S:=FormatErrorResponse(FResponseCountryBlacklist, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

               TSyncSendResult2.SendMsg(ASender.Thread, '- Reverse DNS not found - ');
               ConnectionDetails.RejectID:=3;
EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- Empty Mail From - ');
             S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- Mail From and Mail To are equal - ');
             S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- EMail To is in honeypot emails - ');
             S:=FormatErrorResponse(FResponseBlacklistLocalIP, PeerIP, '', '', '', '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- IP blocked by honeypot autofilter - ');
             S:=FormatErrorResponse(FResponseBlacklistLocalIP, PeerIP, '', '', '', '');

             TSyncSendResult2.SendMsg(ASender.Thread, '- Mail From Domain and Mail To Domain are equal - ');
             S:=FormatErrorResponse(FResponseBlacklistLocalEMail, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

             TSyncSendResult2.SendMsg(ASender.Thread, 'Exceeded maximum number of RCPT TO (' + IntToStr(ConnectionDetails.RecipientCount)+ ') - Disconnecting ' + PeerIP);
             S:=FormatErrorResponse(FResponseMaxRCPTTO, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');

               TSyncSendResult2.SendMsg(ASender.Thread, '- Invalid MX record - ' + tmpResult);
               S:=FormatErrorResponse(FResponseNoMX, PeerIP, EMailFromDomain, EmailAddressTo, EMailAddressFrom, '');
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116 		Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 13 January 2006 at 3:30pm

Got it so now only  "- MAPS search done... "

concatenates the customized message in the logs. Makes sense, so we know which MAPS entry caused the failure.

Since the MAPS entry used to be used for the local blacklist also, I was used to looking for the full string that included my customized message.

I guess I just need to start the coffee earlier in the day. Thanks.

-Matt
-Matt R
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 13 January 2006 at 3:37pm
Yeap, exactly. It was actually bad programming on our part before having placed the IP blacklist lookup embedded in the MAPS searches as well. Years ago, without all the filers SpamFilter has today, it made it simpler, but that is just an excuse... It was not its place.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116 		Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 17 January 2006 at 1:02pm

You removed the standard log line:

Resolving Ipaddress - hostname

Big gaping hole in my reporting now; Bummer.  Anyway we can get that back in the standard logging?

Any plans on updating the quarantine database so that it contains the current set of Rejection Codes?

 
-Matt R
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 17 January 2006 at 10:32pm
Matt,

You can find the answer to your first question in the other thread at http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID= 5449.

For the second question, I'm not sure I follow. SpamFilter will (or better *should*...) automatically add all missing reject codes from the database when the program is started. Right now there should be 19 codes, the last one being the "URL in email found in SURBL search". Is this what you were asking?
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.344 seconds.

Copyright (c)2002-2021 LogSat Software LLC - Sales: sales@LogSat.com
Contact Us