Spam Filter ISP Support - RegEx & Incredimail

Print Page | Close Window

RegEx & Incredimail

Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=1303
Printed Date: 14 March 2025 at 6:41am

Topic: RegEx & Incredimail

Posted By: Guests
Subject: RegEx & Incredimail
Date Posted: 09 July 2003 at 12:20pm

When using the magic RegEx (<[!--]+[a-zA-Z0-9]{11,}) expression even for a few minutes I notice that it it blocking a lot of people using incredimail. I remember reading a previous post about this somewhere...

does anyone know of a way to exclude the incredimail comments so that it won't be blocked.?

Thanks.

Replies:

Posted By: Desperado
Date Posted: 09 July 2003 at 12:35pm

Please post a sample message that was blocked including the header and I can take a look at it. No promises though.

Dan S.

Posted By: Guests
Date Posted: 09 July 2003 at 1:15pm

This is an entire email message that was blocked.

at the bottom there is a

Received: from 24.236.126.4 by 192.10.10.224 (LogSat Software SMTP Server - Unlicensed Evaluation Copy) Wed, 9 Jul 2003 10:39:59 -0500
Received: (qmail 12960 invoked from network); 9 Jul 2003 15:39:53 -0000
Received: from unknown (HELO DsextonXP) (24.214.165.25)
by smtp3.knology.net with SMTP; 9 Jul 2003 15:39:53 -0000
Reply-To: <
From: "Dianne Sexton" <
To: "Becky Pierce" <
"Jennifer Joly \(E-mail\)" <
"Pam Herring \(E-mail\)" <
Subject: FW: Southern Folks
Date: Wed, 9 Jul 2003 10:42:18 -0500
Message-ID: < mailto:6F8EED291688D31188EA009027D102A92A2BAC@PCDI1" CLASS="ASPForums" TITLE="WARNING: URL created by poster. - 6F8EED291688D31188EA009027D102A92A2BAC@PCDI1 >
MIME-Version: 1.0
Content-Type: multipart/related;
boundary="----=_NextPart_000_0180_01C34606.8AF27740"
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1106
Importance: Normal
X-Server: LogSat Software SMTP Server - Unlicensed Evaluation Copy

This is a multi-part message in MIME format.

------=_NextPart_000_0180_01C34606.8AF27740
Content-Type: multipart/alternative;
boundary="----=_NextPart_001_0181_01C34606.8AF27740"

------=_NextPart_001_0181_01C34606.8AF27740
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

-----Original Message-----
From: Helen [ http://mailto:leroysbaby@elmore.rr.com" CLASS="ASPForums" TITLE="WARNING: URL created by poster. - http://mailto:leroysbaby@elmore.rr.com ]
Sent: Tuesday, July 08, 2003 2:43 PM
To: Wendy Anderson; Lisa Adams; Linda Lee; George Pace; Dollar; Dirty;
Dianne; Deidre Cannon; Betty Woodard; Beth Schultz; Beth Ann; Becky
Wilkinson; Bama
Subject: Fw: Southern Folks

Subject: Southern Folks

Gabriel came to the Lord and said, "I have to talk to you, I have some
Southern folks up here in Heaven who are causing some problems.

They are swinging on the Pearly Gates, my horn is missing, barbecue
sauce is all over their robes, ham hock, sparerib, and pig feet bone
are all over the streets of Gold.

Some folks are walking around with one wing.

They have been late taking their turn in keeping the stairway to
heaven clean.

There are watermelon seeds all over the clouds.

Some of them aren't even wearing their halos, saying it is messing
up their hair.

The Lord said, "I made them special, as I did you, my angel. Heaven
is home to all my children. If you really want to know about problems,
let's call the Devil.

The Devil answered the phone, "Hello? Dang, hold on."

The Devil returned to the phone and said, "Hello Lord, what can I do for
you?"

The Lord replied, "Tell me what kind of problems you are having down there."

The Devil said, "Wait one minute," and puts the Lord on hold.

After 5 minutes he returned to the phone, and said "Okay, I'm back.
What was the question?"

The Lord said, "What kind of problems are you having down there?"

The Devil said, "Man, I don't believe this..... hold on, Lord".

This time the Devil was gone for 15 minutes.

The Devil returned and said, "I'm sorry Lord, I can't talk right now. These
southerners done put the fire out, and are trying to install air
conditioning!"

I want think about it today, I 'll think about it tomorrow. After all
tomorrow is another day. Today I will Bead.

------=_NextPart_001_0181_01C34606.8AF27740
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">

<META content=3D"MSHTML 6.00.2800.1106" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY style=3D"BACKGROUND-POSITION: 0px 0px; FONT-SIZE: 12pt; =
FONT-FAMILY: "=20
background=3D"" scroll=3Dyes X-FVER=3D"3.0" ORGYPOS=3D"0">
<DIV> </DIV>
<DIV class=3DOutlookMessageHeader dir=3Dltr align=3Dleft><FONT =
face=3DTahoma=20
size=3D2>-----Original Message-----<BR><B>From:</B> Helen=20
[ http://mailto:leroysbaby@elmore.rr.com" CLASS="ASPForums" TITLE="WARNING: URL created by poster. - http://mailto:leroysbaby@elmore.rr.com ]<BR><B>Sent:</B> Tuesday, July 08, 2003 =
2:43=20
PM<BR><B>To:</B> Wendy Anderson; Lisa Adams; Linda Lee; George Pace; =
Dollar;=20
Dirty; Dianne; Deidre Cannon; Betty Woodard; Beth Schultz; Beth Ann; =
Becky=20
Wilkinson; Bama<BR><B>Subject:</B> Fw: Southern =
Folks<BR><BR></FONT></DIV>
<TABLE id=3DINCREDIMAINTABLE cellSpacing=3D0 cellPadding=3D2 =
width=3D"100%" border=3D0>
<TBODY>
<TR>
    <TD id=3DINCREDITEXTREGION=20
    style=3D"FONT-SIZE: 12pt; CURSOR: auto; FONT-FAMILY: Arial" =
width=3D"100%">
      <DIV><B>Subject:</B></I> Southern Folks</DIV>
      <DIV> </DIV><FONT face=3Darial,helvetica><FONT lang=3D0 =
face=3DArial size=3D2=20
      FAMILY=3D"SANSSERIF">Gabriel came to the Lord and said, "I have to =
talk to=20
      you, I have some<BR>Southern folks up here in Heaven who are =
causing some=20
      problems.</FONT><FONT lang=3D0 style=3D"BACKGROUND-COLOR: #ffffff" =
face=3DArial=20
      color=3D#000000 size=3D3 FAMILY=3D"SANSSERIF"> =
<BR><BR><BR></FONT><FONT lang=3D0=20
      style=3D"BACKGROUND-COLOR: #ffffff" face=3DArial color=3D#000000 =
size=3D2=20
      FAMILY=3D"SANSSERIF">They are swinging on the Pearly Gates, my =
horn is=20
      missing, barbecue<BR>sauce is all over their robes, ham hock, =
sparerib,=20
      and pig feet bone<BR>are all over the streets of Gold.</FONT><FONT =
lang=3D0=20
      style=3D"BACKGROUND-COLOR: #ffffff" face=3DArial color=3D#000000 =
size=3D3=20
      FAMILY=3D"SANSSERIF"> <BR><BR></FONT><FONT lang=3D0=20
      style=3D"BACKGROUND-COLOR: #ffffff" face=3DArial color=3D#000000 =
size=3D2=20
      FAMILY=3D"SANSSERIF">Some folks are walking around with one=20
      wing.<BR><BR>They have been late taking their turn in keeping the =
stairway=20
      to<BR>heaven clean.<BR><BR>There are watermelon seeds all over the =

      clouds.<BR><BR>Some of them aren't even wearing their halos, =
saying it is=20
      messing<BR>up their hair.<BR><BR>The Lord said, "I made them =
special, as I=20
      did you, my angel. Heaven<BR>is home to all my children. If you =
really=20
      want to know about problems,<BR>let's call the Devil.<BR><BR>The =
Devil=20
      answered the

Posted By: Desperado
Date Posted: 09 July 2003 at 2:32pm

The following will work BUT it does leave a small hole. If this RegEx is reported back to a Spammer, and they take the time to see what it is, they could "break through" the filter. Not very probable, I don't think.

(<[!--]+[a-hj-zA-HJ-Z0-9]{11,})

Try it and let me know please. If it works, great. If it stops doing it's intended job, not so great!

Dan S.

Posted By: Guests
Date Posted: 09 July 2003 at 2:37pm

Thanks, I'll give it a shot and let you know.
..can you give me a run down of how this one works?

Thanks for the help

Posted By: Guests
Date Posted: 09 July 2003 at 2:41pm

.. Nevermind, I see how it is different.

I'll let you know how it goes.

Posted By: Guests
Date Posted: 09 July 2003 at 2:44pm

This is a problem that will require one of two things.
1) A keyword whitelist if it would work.(needed new feature)
2) if,then, else process in the RegEx filter.

In order for either one of these two ideas to work, SpamFilter will have to process things by priority, ie. Process White list items first then the Black list items. According to Roberto, "..but the setting for no rev DNS instead says "quarantined it". There is no way currently to tell SpamFilter which is the predominant quarantine yes/no rule.

Whatever filter triggers the reject first will determine if the email will be quarantined or not."

Posted By: Desperado
Date Posted: 09 July 2003 at 3:25pm

For your own information, we have also found that some newsletters have tags that get nailed by that filter and we HAD further mods to help with them but took them out. The customers are simply going into the manager and sending them to themselves. The additional tag is as follows:

Now ... George made a post that I am going to respond to ... not here but from his post.

Dan S.

Posted By: Desperado
Date Posted: 09 July 2003 at 3:43pm

George,

Our fix does, in fact seem to work for the incredimail problem (see my post) because we are not seeing no rdns issues, just the comment tags.

Having said that, in our business model, we have made the decision that if there is no RDNS, it gets blocked ... period, no exceptions so for us, in a sense, that simplifies things. Also, my understanding is / was that white lists are processed first so I am not sure we have a problem there.

If all my assumptions are correct, my big "WISH CHANGE" would be a very simple one (depending on how the RegEx engine works). "Simply" get Booleans to work in a single expression. Example: ( (This expression) AND !(That Expression) ). The "!" meaning "NOT". All the documents I see on Regular Expressions don't mention ANYTHING about Booleans. However, I use them all the time in Perl so Perl must have an extended set of valid directives.

Bottom line ... AND , OR, NOT (!) would extend the capabilities of the Regular Expressions beyond belief!

Dan S.

Posted By: Guests
Date Posted: 09 July 2003 at 4:32pm

the adjusted regex code you gave me seems to be working well. So far it seems to have only cought junk mail and not incredimail.

Thanks again for your help.
If you have any other helpful info I'm all ears.

Thanks.

Posted By: Desperado
Date Posted: 09 July 2003 at 5:41pm

OK ... I am continuing to "Refine" this but I am using the following 3 in this order. I am working on a way to make them more accurate and simpler but this is what I have for the moment:

(<[!--]+[a-hj-zA-HJ-Z0-9]{11,})
(<[!--]+[a-zA-Z0-9]{22,})
(<[!--]+[a-zA-Z0-9]{2}(-->))

Dan S.

Posted By: Guests
Date Posted: 09 July 2003 at 6:06pm

Dan,
I agree with you and also put your modified RegEx tag in to try it out. RegEx needs to be more logical.

The problem I ran into in the the way the Black/White lists work was even though an email address is in the black list that was set to not quarantine, it still got quarantined because it had keywords in the keyword list and that list is set to quarantine.

This a problem when you get a flood of email to accounts that don't exist. I just wanted to block them out right and disconnect the sender with out having the emails quarantined.

Posted By: Desperado
Date Posted: 09 July 2003 at 7:05pm

George,

I am still working on the RegEx ... I posted an iteration of the modified one earlier but I am still trying to simplify it.

Now ... in response to "RegEx needs to be more logical" ... But then what would all the UNIX Geeks do in their spare time?

On your last point. Ahh ... I understand completely. I have a nasty situation where we host a domain (from an old ISP acquisition) that only has 2 valid addresses. We want to "dump" the domain and kill any MX records pointing to us but the 2 users are not interested in changing to our default domain. That single domain (out of the 400 + domains that we host) is responsible for 30% of the messages in quarantine. I thought I had a "work around" but it ended up not having the desired effect. I really think that, for now anyway, we have to live with the quarantining (is that a word?) of non-existent addresses. In LogSat's defense, all of us (perhaps you and myself especially) could "Wish List" them to death. I hope you agree that LogSat has a real good product here. At the traffic / account level that even our smallish ISP is at, the closest WORKING products we looked at, and I spent months on looking, were priced anywhere from $18K to $120K. I don't know about your situation, but that wasn't going to fly for us. Even at that level, there were areas that were less than perfect.

I have asked Roberto to give us a $$ quote for some specific mods the we really want. Everyone will benefit from the changes as we are not requesting a "special build" just for us. If we end up doing it, I think the major change I have requested will help both our situations ... not sure yet though.

BTW ... We are in CT / USA. Where are you located?

Dan S.

Posted By: Guests
Date Posted: 09 July 2003 at 8:24pm

Dan,
I saw your posting on the RegEx codes and have copied them to my keyword list which I will post this evening.

Yes "Quarantining" is a word when spelled this way. :)

You are correct about LogSat's product. It is by far the best product out there for the $$$. The fact that it will work with just about any SMTP server is the biggest plus. Most of the other spam filtering products I looked at were way too expensive and were limited to certian SMTP servers. Too many companies are trying to take advantage/$$$ of this problem. The ones I have to laugh at are the ones that send out SPAM advertising thier products.

I am located in Ca /USE. Roberto has my permission to give you my email address. Don't want to post it here since I am sure if I did I would end up getting Flamed by spammers.

Posted By: Desperado
Date Posted: 09 July 2003 at 8:43pm

George,

I asked Roberto to pass my address to you also. I want to discuss RexEx's some more without "Clogging" up the forum. Once I get the bugs out of the expressions I am using, I may post them but I do not want to be responsible for causing a problem due to my stupidity. Some of the expressions I am working on make me want to go back and "fix" the first 1 or 2 I have done. I am also trying to get clear stats on how effective they are with respect to eachother and the VERY few actual keywords I do have.

I have a grand total of 9 RegEx's and 8 actual words. I am truing for all Regular expressions and living with what else get by.

Dan S.

Print Page | Close Window