Starting with build v1.2.0.151 SpamFilter is able to scan the whole email content + subject header for RegEx (Regular Expression) keywords.

This allows very powerful keyword searches. Many spammers send html emails containing invalid (thus invisible) html tags or html comments in between letters to avoid normal keyword detection.

For example, the following html source:

<!--fxkbu8116c72f6-->SP<mynqhy2d9bswg-->AM 
    <!--ei2rq7erjldy3y-->MER<!--ywf1ph1zmgcik9-->

will actually display SPAMMER in an email client.

We've been using the following RegEx search string to, so far, successfully block a lot of this spam:

(<[!--]*[a-zA-Z0-9]{11,})

This is what the above expressions looks for (remember that SpamFilter requires a RegEx expression to be sorrounded by parenthesis () in order to distinguish it from regular keywords):

• <   look for an open tag start character, immediately followed by...
• [!--]*   this looks for zero or more occurrences of the  !--  characters indicating an html comment, immediately followd by...
• [a-zA-Z0-9]  any letter or digit....
• {11,}  repeated at least 11 times. This has to be a combination of only either letters or numbers. Any space, tab, single quote, double quote etc will break the sequence.

For example, <a href="aaaa.htm"> will not cause a trigger since there is a space immediately following the a before href.
We choose a minimum repetition of 11 since <blockquote> is a valid tag 10 characters long...

If anyone has comments, problems, or improvements with this "apparently magic" keyword search, please let us know!

Roberto Franceschetti
LogSat Software

Alan,

We had to make drastic changes in the code to support the new quarantine database and the web functionality. The code was not as stable as we would have liked, so we created and made public our beta test versions so that we could have more users test the application and report problems. Had we released an official release it would have been with several bugs, and would have many  many users complaining of crashes. We really did not want that.

After two weeks of testing we finally seem to have a much more stable product. Unless any major problems arise in the next few days, we are thinking of making this beta official by the end of this week.

Roberto Franceschetti
LogSat Software

Can RegEx work with a dictionary list?

The "invalid html tags" RegEx keyword has been working *almost* perfectly, but there have been a few situations where legitimate emails are being bounced by this rule.

Here's what appears to be happening.  If a message contains a forwarded message within its text, this forwarded message text is likely to include the original from/to email addresses.  In many cases, these email addresses are enclosed in <>'s.  For example:

To: <longusername@earthlink.net>

This matches the RegEx keyword criteria of [a-zA-Z0-9]{11,} so... bounce!

To get it working, I've changed the [!--]* portion of the RegEx keyword (zero or more occurrences of !--) to instead read [!--]+ (one or more occurrences).  This is still very effective, and has eliminated the bounces of legit messages... but is obviously not perfect as it doesn't offer protection from invalid html tags that are not comments.

RegEx is new to me, but I might try working with it later and coming up with some sort of logical NOT based on the occurrence of a @ within the string.  If someone more familiar with RegEx could just fire this out and post it here, that would be even better. :)

Jim