Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - honeypot question
  FAQ FAQ  Forum Search   Register Register  Login Login

honeypot question
 Post Reply Post Reply
Author
  Topic Search Topic Search  Topic Options Topic Options
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Topic: honeypot question
    Posted: 08 June 2005 at 9:08am

I noticed that after i had entered a few mail adresses that were beeing sent to our domain in the honeypot file, that those messages got rejected anyway, because of the adress absense in the authenticated TO list.

Isnt the honeypot intended to catch the flies? shouldnt the honeypot mail adresses be let thru in order to get effective ip blocks?

I'd rather not add the honeypot adresses to the authedTOlist, it might become confusing

Marco
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 08 June 2005 at 11:03am

Marco,

LogSat can correct me if I am wrong but, from what I see in my logs, if a user is NOT on the Authorized list (if it is being used) the connection is somewhat unceremoniously dropped ... rather than continuing on with any filter checking.  Therefore, the offending message would never get subjected to any other filters ... hence, not Honeypot blocking.   This is not a bug but is by design.

Regards,
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 08 June 2005 at 4:26pm
Marco,

Dan is correct. Once a filter is triggered, for efficiency all other filters are skipped. Since the AuthorizedTo filter takes precedence over the honeypots (an updated list of precedences is found below), the honeypot filter is skipped. The only way to have the honeypot catch those files is to add the honey-email addresses to the AuthorizedTO list so they can be further processed.

Updated filter order:
http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID= 5171
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Posted: 09 June 2005 at 2:42am

ok, thanks for the explanation.

 

p.s. a database with details of the dropped connections might proove useful in combatting the spammers...



Edited by Marco
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 09 June 2005 at 11:44am

Marco,

See my posts on the software "Sawmill".  Using Sawmill, I get rather detailed information on any of the "Reasons" or "Actions" mail was stopped by SpamFilter including dropped connections.  The reports allow you to "Dig" very deeply into any field.  So, if I select, in my report, "Reasons" and then select "Dropped Connection"  I can then "Zoom" the report to view the log details (whew!) or, more useful, Source IP addresses or Form etc.  Sawmill has a sample report using my own logs (from May I believe) at http://www.sawmill.net/samples.html

EXAMPLE:  (This does not post to the fourm well but .... Zoom in of Dropped connections by source IP

Source IP Messages Bytes
1 216.244.114.10 18,072 23.4 % 173.00 k
2 66.181.192.65 1,610 2.1 % 0 b
3 66.181.200.5 831 1.1 % 0 b
4 220.117.246.30 117 0.1 % 0 b
5 221.146.4.161 110 0.1 % 0 b
6 220.117.251.253 107 0.1 % 0 b
7 24.98.128.183 92 0.1 % 0 b
8 216.153.145.213 87 0.1 % 628.00 k
9 220.117.244.127 86 0.1 % 0 b
10 216.113.188.96 86 0.1 % 914.00 k
  30553 other items 56,001 72.5 % 969.94 M
  Total 77,199 100 %   971.61 M

NOTE: Sawmill is using an earlier version of my Parsing Filter so the "Dropped Connection" field is not an option but my latest, and I believe the one they ship with, does.

I hope this is useful to you.

Regards,
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com
Back to Top
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Posted: 10 June 2005 at 3:16am

I will look into that, thanks Dan.

 

But now i have a new problem, i added some of the honeypot mail addy's to authedTOlist, and guess what happened; our ISP's mail relay server got blocked out.

So now the question becomes: how do i whitelist the relay's ip and still maintain full filtering on all mails it forwards?

Maybe a new function 'untrusted whitelist IP' or 'mail relay server IP' for just such an occasion is called for?

I saw someone else post somewhere that he needed such a function to pass mails thru from one domain to another, but can't find it now.

Any ideas?

 

Marco

 
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334 		Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 13 June 2005 at 2:46am
This is the same function I am looking for, and perhaps the way you worded it is better. To have the honeypost trust certain ips?
Back to Top
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Posted: 13 June 2005 at 3:24am

Roberto,

It doesnt 'seem' to be a very hard programming problem, simply have the honeypot filter match the ip's that are in a 'relay ip' tab, and if they match up, let the ip pass on the honeypot. (and maybe some more filters)

but maybe i'm seeing things too simple, correct me if i'm wrong.

 

Marco

 

*edit* just thought of something; for better statistics representation, have the relay ip's country be excluded from the statistics as well.

The relay buffers mails from all over the world, when the mail gateway is temporary unavailable, but is then placed in the statistics as beeing originated from the netherlands, in my case.

So, exluding mails that 'originate' on this relay would give more accurate statistics.

 

*edit2*

OR: in case the honeypot is triggered, compare the originating ip with the ones in the 'relay ip list' and if matched, drop the mail, but DO NOT blacklist the relay's ip.



Edited by Marco
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 13 June 2005 at 7:22pm
SpamFilter should really see the original IP of the sender when procesing emails. If SpamFilter handles emails that are being relayed by a "friendly" server, then things are bound to go wrong, not just with the honeypot file. Think about the SPF filter for example... If the IP of the server connecting to SpamFilter is not listed in the SPF DNS record of the sender, the email will be rejected. And if your secondary is forwarding emails to SpamFilter, that *will* cause a big issue.

The mains solutions that come to mind are to:
(1) place SpamFilter (or any other antispam software) in front of all the servers listed as MX records,
or (2) forward the email from the secondaries directly to your main SMTP server, bypassing the main spam filter.
or (3) install a second SpamFilter on a separate IP or separate server, configure it skip ALL IP-based tests (reverse-DNS, country, SPF, MAPS-RBL, IP blacklists, MX checks etc.), and have the secondary forward emails to this lesser-featured SpamFilter.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 4:11am

I don't know the exact configuration of the relay servers, since they are managed externally, by our ISP.

But as far as i know they act as gateways when our smtp server is responding, and the incoming mais are relayed 'as is' to our smtp host (the spamfilter).

However, when our smtp host is unavailable (due to a crash or overload) all incoming mails are forwarded to the secondary , and this secondary keeps trying to deliver the mails on regular intervals.

I have no control over the secondary, and cannot place a spamfilter in fron of it.

90% of all mails are delivered to the primary, but for some reason, mails get directed to the secondary as well even when the primary is up and running.(usually during the night).

(Maybe some of the spammers deliberately send mail to secondary server ip's)

I can live with the fact that all ip based checks will be worthless in this case, but i DO want the mails to be passed thru whatever filters that are still valid. (keywords, surbl, authedTOlist, bayes, honeypot)

All in all, spamfilter is allready doing a GREAT job, it catches 95% of all spam (even under the conditions described above)

I would really like to use the honeypot as well, since it WOULD actually catch some flies, and it allready caught some. All i'm asking for is the option to prevent *some* ip's from beeing added to the honeypotblacklist when its active.

Regards,

 

Marco

 

 



Edited by Marco
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 7:57am
Marco,
 
I completely understand your dilemma (as I am sure LogSat does also).  However, can you re-explain what invalid entries get into your honeypotIP list and what filter exactly puts them there?  Perhaps I can come up with a work around.  I also have backup servers BUT I have the luxury of being the administrator of them also so I have some additional control.
 
Regards,
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com
Back to Top
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 8:19am

Thanks for thinking along Dan, appreciate it.

ok, i did some researching on our MX entries (The ISP is also running the primay DNS) .

It is set up like this:

preference: 10 : mail.ourdomain.com

20: relay1.ISPdomain.net

20: relay2.ISPDomain.net

30: mail1.ourdomain.com

30: mail2.ourdomain.com

 

Only the mail.ourdomain.com is under my control.

I think things happen like this: the secondary mailservers of the ISP (mail1, mail2) are only receiving the inbound mails when the primary is unavailable. For some reason the relay's ip's are beeing put in the mail headers as beeing the originating ip. So when inbound mails got buffered and were using honeypot adresses, the relay's ip's got blacklisted.

Resuming; the honeypot is blacklisting the relay1, relay2 ip's, because of mail that is sent to us, gets buffered on mail1/mail2 , and is using honeypot adresses.

My first thought was to make a script that checks the honeypotblockededIP.txt file for presence of those 2 ip's and remove if found, But that isnt a very elegant solution, and would cost additional CPU load.

does this make sense to you?



Edited by Marco
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 8:45am
Originally posted by Marco Marco wrote:

Marco,
Forgive me ... I have not been thinking real clearly after our disatrous weekend but ...

I think things happen like this: the secondary mailservers of the ISP (mail1, mail2) are only receiving the inbound mails when the primary is unavailable. For some reason the relay's ip's are beeing put in the mail headers as beeing the originating ip. So when inbound mails got buffered and were using honeypot adresses, the relay's ip's got blacklisted.

The reason is that it IS the originating IP so that is correct.

Resuming; the honeypot is blacklisting the relay1, relay2 ip's, because of mail that is sent to us, gets buffered on mail1/mail2 , and is using honeypot adresses.

Understood.

My first thought was to make a script that checks the honeypotblockededIP.txt file for presence of those 2 ip's and remove if found, But that isnt a very elegant solution, and would cost additional CPU load.

Not elegant but EXACTY what I do.

Now,  Since my last post, I was emailing Roberto directly and realized what you were getting at ... and I have the same issue but dealt with it as above.  However, here is part of what I wrote to Roberto while thinking on this issue:

I think, but am not sure, what Marco and several others are asking for is a "TrustIP" list that would not allow relay but would prevent the honeypot from triggering if the IP was in the trust list.  So, rather than seeing "Bypassed all rules" scenario, you would see a "Bypassed SOME rules" situation.  Did I get this right?

Yet another stupid idea I had, and I think it is either impossible or real hard is to have a filter list that looks at the *next to the last* IP that was used to deliver the message.  If enabled, this list would contain a list of filters to use against the previous IP.  So, if the list looked like:

dnsbl
rdns

This would instruct the software to use *not* the connecting IP but use the IP before that for the above tests.   THis would add huge overhead I believe and would probably break every rule in the book BUT it would kill several birds with one stone.

Hows that for a hair brained thought?

Regards,
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com
Back to Top
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 9:57am

it would ROCK if possible!

It would make the filters 100% operational in my described situation.

Im also thinking now about backtracing to the origin, or at least the next after the originating ip, in case of spoofing.

If at all possible that would make spammer's lifes pretty miserable.

nah, i'm starting to rant now :)



Edited by Marco
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334 		Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 10:03am

adding in a "TrustIP" JUST for honeypot in this case would solve my problem too. I think in Marco's case and mine, we could write a script to parse the honeypot ip's list, but that isn't real efficient.

Just to be clear, we just want to be able to whitelist ips against the honeypot list...that's it.
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 10:05am

Marco,

Yet another part of a conversation I am having with LogSat:

One option that may help, but is rather tricky would be a method of testing for how many hops a message takes.   I am not sure I would trust this as more and more systems use several hops to deliver mail.  Our system does.  So determining a value for "Max Hops" could be an issue.

Aren't I just a Royal Pain in the you know what!

Regards,
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com
Back to Top
Marco View Drop Down
Senior Member
Senior Member
Avatar

Joined: 07 June 2005
Location: Netherlands
Status: Offline
Points: 137 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Marco Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2005 at 11:13am

No trouble in the sitting area yet :)

im thinking of a dedicated backtracing relay system in front of the spamfilter, with enough resources to do extensive tracing, right after it passes the mails on to the filter. Which in turn handles them as usual.

Not a toy i will be allowed to own though :/

but imagine such a system with automated hostmaster notification on spamming network users.... Not all of those would give a h00t ofcourse, but i bet quite a lot would take actions against the offending users..

Anyway, kspare said it, having a honeypot whitelist would fix our problems. Your suggestions are even better Dan, but i'm afraid it would take some serious effort in getting that to work, but i do hope logsat is willing to give it a try.

 

 

 

 

 
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334 		Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 16 June 2005 at 10:18am
Roberto? Can you enlighten us with your wisdom  
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104 		Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 24 June 2005 at 5:23pm
Kevin, Marco,

We were originally staying with the response given at http://logsat.com/spamfilter/forums/forum_posts.asp?TID=5217 #6068,
however we've revisited that...

We prepared a new beta that may solve your issues. Build 461 has the following release notes:

// New to VersionNumber = '2.5.2.461';
{TODO -cNew : Added RealtimeDiskLogging option in SpamFilter.ini file to have log being flushed to disk with every entry}
{TODO -cNew : Added DoNotAddIPToHoneypot option to SpamFilter.ini file to prevent certain trusted IPs from being blacklisted by the honeypot filter}
{TODO -cNew : Changed the logging on screen performance to increase reliability and have a smoother scroll}

We have not released it yet in the pre-release area of the website as it's an on-going work to add a per-domain filtering options (you'll see a non-working preview on the settings tab.

It is however otherwise fully functional and should be very stable. If you wish to try it to see if it will solve your problem, I'm sending you and Marco a download link by private message in this forum.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.191 seconds.

Copyright (c)2002-2021 LogSat Software LLC - Sales: sales@LogSat.com
Contact Us