Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Subject RegEx's New Thread
  FAQ FAQ  Forum Search   Register Register  Login Login

Subject RegEx's New Thread

 Post Reply Post Reply
Author
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Topic: Subject RegEx's New Thread
    Posted: 10 February 2004 at 7:02am

All,

 

I found a "baddy" which was causing false positives on my RegEx's.  If someone has a cleaner solution, let me know.  The problem was that my expressions were "Open Ended" so to speak.  This meant that the scanning continued on into the body.  I am not sure if this is a "Bug" or not.

My fix is to add  .*\n  to the end of the expressions to make sure the test ends at the end of the Subject line. 

Again ... Please ... Comments.  I am kinda fuzzy at this point ... not enough sleep!

Regards,

Dan S.

Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 February 2004 at 9:06am

All,

Here I go again!   Now I REALLY have them working correctly but I am going to run them past a couple of folks because I had to do some strange stuff to make sure that ONLY the Subject line was tested.   I am not going to post these for all spammers in the world to see and learn how to cheat them.

This points to a previous post about starting a mailing list.  My thought, is to have a totally separate BBS (which I can host or any one else for that matter) and only VERIFIED "good guys" will be able to log on. I will touch base with LogSat support to see how much of a burden it would be for them to verify the "good guyness" (can you tell I have been up too long?) , meaning they are SpamFilter customers.

Regards,

Dan S.

Back to Top
Brian View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote Brian Quote  Post ReplyReply Direct Link To This Post Posted: 10 February 2004 at 2:57pm

I was told to use .* sparingly and only if absolutely needed.  I like using the less-greedy +?.  This will match 1 or more times, but as few instances as possible.  I haven't tried using \n to close off the expression, but it makes sense.

If I wanted to catch =asdflja;lskjf=adsfadf= and nothing after it, I would use ((?i)=.+?=).  Changing the rexeg to ((?i)=.*=) catches not only the first set of equal signs, but if there is any more equal signs in the message, it will catch everyting up till the last equal sign in the message.  Modifying the regex once more to ((?i)=.*=.*\n) is better then .*, but unfortunately it will catch all after the second equal sign.  In this example that is not an issue, but it may be in others.  All in all, I prefer .+? to catch characters that may or may not be in between certain characters or phrases that I am trying to filter on.  I can see where .*\n works, but if not closed properly it can have disasterous results.

BTW: I second the notion for a more closed environment for regex discussions.  It would definitely help us novices get a better understanding of this powerful language used to filter in SF.  I'm not saying that Logsat needs to provide this, but I think maybe someone here may now of a place that is up and running now that maybe we could check out.

Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 February 2004 at 4:30pm

This will be VERY Brief because I am going on vapors here.   I did not post my "final"? RegEx (and am slightly uncomfortable doing so) but I had other issues and contacted LogSat directly to track down if I found a bug ... or I am just an Idiot.  After better than 30 hours with no sleep (Network maintenance) the "Idiot" choice may be the correct one!  However, Roberto indicated that he was following the thread and is going to track down the issue sometime tonight.

In response to your discussion, you are correct on the .* issue in most cases.  However, in one case, it is actually what I wanted (mainly to try to track the possible bug down).  In the other cases, using the ,* was "cheap and dirty" and required much less though on my part.  Once we track down the problem I encountered, I will spend a couple of days trying to create "real" expressions, rather than the ones I threw out to demonstrate that the Subject filtering does work.

As to a "Private" BBS,  I have already volunteered to set one up but really need to talk to Roberto about validating users.  I do not want to cause LogSat any inconvenience but from my viewpoint, it is imperative that the whole world doesn't get a free hand at seeing how to defeat our filters.

I hope we can put something together fairly soon because as you said, we have a fairly powerful tool in our hands and it would be nice to get the most out of it and this is an area where collaboration can really work to everyone's advantage!

Regards,

Dan S

Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 February 2004 at 8:51pm

All,

I am setting up a BBS in the next day or so for RegEx AND discussions of "wish Lists" so we don't clog up LogSats support site with a bunch of "out there" mod requests.  My thought was that the several users that I see making the most requests might hash things out and that way perhaps we could come up with a reasonably organized list of useful but not overly complex changes that we could then present in an orderly fashion.

 I have not figured out how I will verify users yet but I am sure SOMEONE has an idea ... right?  Also,  due to my recent work load, I may not be the best person to "moderate" it or generally administrate it but I will try to get the time in.  If any one else is interested in that "job", the BBS has a web admin so it is real simple to administrate.

Please let me know if this all seems like a good idea ... or should I just abandon the idea.

Regards,

Dan S.

Back to Top
Sean View Drop Down
Groupie
Groupie
Avatar

Joined: 21 February 2005
Location: United States
Status: Offline
Points: 81
Post Options Post Options   Thanks (0) Thanks(0)   Quote Sean Quote  Post ReplyReply Direct Link To This Post Posted: 11 February 2004 at 9:53am

Dan,

A while back I set up the unofficial SpamFilterISP Support forum. It can be easily reconfigured to require any approval needed. Any can also be block to hide all forums without login. However, depending on how we decide on "authenticating" users, there will always be the chance of a spammer getting the list. So these lead me to advance loggin of IPs, which can also be easily setup and monitored. This too could be spoofed at any given time by a spammer. Basically no matter how much you try to keep a spammer out, they will always find a way to get a hold of what they need, legally, or illegally.

Now What to do?

Whatever is setup needs to be known as  possible SPAM sender readable, so it should be encourage to have the users modify the List on their own to fit there needs, and only use the "published" list as reference.

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 11 February 2004 at 2:09pm
http://www.invisionpowerboard.com that's the forums i'd suggest. You can also set it up so that you have to be part of a different group just to be able to see a forum, that way you can't just sign up and see everything. I dunno how you will validate who is an actual user and who is a spammer though....
Back to Top
Sean View Drop Down
Groupie
Groupie
Avatar

Joined: 21 February 2005
Location: United States
Status: Offline
Points: 81
Post Options Post Options   Thanks (0) Thanks(0)   Quote Sean Quote  Post ReplyReply Direct Link To This Post Posted: 11 February 2004 at 4:33pm

The unofficial support forum that i createde a while back is run invisionboard. Which is nice out of the box, but can be changed easily to suit your needs. It can be seen below:

http://199.6.41.242:8080/SPAMFilterISP/forum/

Back to Top
JimMeredith View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote JimMeredith Quote  Post ReplyReply Direct Link To This Post Posted: 12 February 2004 at 1:18am

Great idea!

Whoever does it, and however it is done, count me in.  I can offer limited assistance.  Like you, Dan, I also have a full plate at work -- but I offer my efforts to "co-moderate" if needed, if doing so would help to share the workload a little bit and make this possible.  Let me know.

As for the concern about spammers getting in... the approach should be "cautious but not paranoid."  Your idea (Dan) about verifying email addresses with Roberto should provide adequate security, if he's OK with doing this.  There will always be a few spammers who waste their time trying to defeat our efforts, but short of performing a full background check on anyone prior to inviting their participation on this list, there is no practical way to avoid this.  By closing the list and requiring a minimal clearance of this type, most spammers will redirect their efforts to the SpamCop lists or NANAE or other places where they can learn about anti-spam efforts (and how to defeat them) without the cost or effort of getting an "invitation."

Thanks for your efforts.  Let me know if I can be of assistance.

Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 12 February 2004 at 9:07am

Sean,

I do not think we need to get overly paranoid so if you can secure it by user & pass, and if we can request that Roberto can verify that a email address is linked to a "customer in goog standing", that should be good enough.  Hackers will always be able to get past anything we do so that is the risk we take.  It is not life or death after all!

Please get mu email address from LogSat and we can just touch base on this.  If I do not have to set up a BBS and manage it, all the better for me.

Thanks and Regards,

Dan S.

Back to Top
George View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote George Quote  Post ReplyReply Direct Link To This Post Posted: 12 February 2004 at 7:46pm

As I have already stated my support for a forum like this, let me add that even though having Logsat verify the users, it would be easy to join the forum. Please note that $400.00 is a low price to pay for a determined spammer that wants to try to defeat the RegEx strings that are developed through the forum.

Not to sound paranoid or anything. :-)

Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.164 seconds.