Print Page | Close Window

message size and checking speed

Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=550
Printed Date: 05 February 2025 at 1:57pm


Topic: message size and checking speed
Posted By: Guests
Subject: message size and checking speed
Date Posted: 15 May 2003 at 12:37pm

Roberto, i'm curious what to do with big size messages?
i found out that raw message size is in direct proportion with the speed of checking against blacklists...  for example i had 20 simults incoming connections with 84k message it took about 5+ minutes on my 2.5Ghz P4 CPU to parse it...

As u know i have 2 regexps in FROM: b/l and 2 in TO: b/l, also my FROM: b/l is 1.6 megs with overall 61000 entries, so.... any suggestions... or may be u have plans to optimize regexps performance somehow? may be the size of messages is the cause of those lockups?


Replies:
Posted By: LogSat
Date Posted: 15 May 2003 at 10:24pm

Message size should only matter if you also have huge keyword lists. If your keyword file is 10KB-20KB, it can take 5-10 seconds to scan for hundreds-thousands of keywords in a single email. Multiply that by the concurrent connections and the wait time will add up.

How large is your keyword file?

For the regexps, we tried to optimize the searches by using regex searches only for the line in your lists that has a regex expression. Any other text that is not regex will be searched using regular text search, which is much faster. To be honest we have not done much testing on local lists that are as big as yours, we'll do so shortly.

Roberto
LogSat Software


Print Page | Close Window