Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Mem Leak?
  FAQ FAQ  Forum Search   Register Register  Login Login

Mem Leak?

 Post Reply Post Reply Page  12>
Author
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Topic: Mem Leak?
    Posted: 08 August 2003 at 1:27pm
Hi, I'm running version 194 registered and it seems like there is a mem leak. After only 200 messages Spamfilter is useing 107mb ram and 133mb of virtual ram. Nothing fancy on the machine, just nt4, sp6a, iis4 and mysql.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2003 at 1:29pm

Kevin,

Do you have quarantined emails being displayed in the quarantine tab? If so, try to "Hide List", since that view does consume a lot of RAM.

Roberto Franceschetti
LogSat Software

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2003 at 1:32pm
I don't see how there could be, as I rebooted the server and never even logged into it until now to check to see how much ram it was using. I'm also running it as a service if that helps.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2003 at 3:16pm

We've had a couple of reports where the quarantine grid fills up by itself, without any user intervention, thus the question. If you have verified that it is indeed empty, could you please email us at support@logsat.com a copy of the spamfilter logs which start from when you last started the spamfilter service?

Roberto F.
LogSat Software

Back to Top
SteveW View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote SteveW Quote  Post ReplyReply Direct Link To This Post Posted: 11 August 2003 at 10:45am

Hi Kevin,

I am running version 190, as a severice on W2k SP3, MySQL, IIS and have also noticed that ther is a big memory leak when the quarantine list is not hidden.
It service as using 200+ megs and when I hid the list the memory usage dropped to 6 megs.
This is my test server and durring this testing there was no load on it over the last week.

I do not know if this helps.
- Steve

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 11 August 2003 at 11:46am
Yeah, there is definitly a problem, my backup mailserver, which gets about 40 emails a day, all of which are spam, consumed 400megs of mem over a 24 hour period. That's not good, I am having to restart the service ever 12 hours...
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 11 August 2003 at 5:20pm

Kevin,

What build are you running?  It is unclear to me why you are getting a memory leak.  I manage about 50,000 message a day and currently have 300,000 messages quarantines and Spam Filter is running at around 35MB of RAM Even if I let the Quarantine Grid populate, I do not have anywhere near the RAM usage you are reporting.

Dan S.

 

Back to Top
George View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote George Quote  Post ReplyReply Direct Link To This Post Posted: 11 August 2003 at 6:09pm

I typically have 50,000+ items in qurantine and use the grid many time a day and have never seen a memory leak nor the memory useage you are describing. This sounds like something else is contributing to the problem. Do you have a virus scanner running on the system? If so then you should exclude the spamfilter log folder since log files are always being written to.

 

g

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 11 August 2003 at 7:04pm
I'm running ver 194 registered. And yes I agree, it's doesn't make much sense.
Back to Top
George View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote George Quote  Post ReplyReply Direct Link To This Post Posted: 11 August 2003 at 8:37pm

While I haven't tried any of the builds after 190 you might want to try 195 and see if that helps.

Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2003 at 2:11am

Kevin,

Running build 195, with 307,000 messages in the Quarantine and under light load (10 or so connections) I have the following stats:

With out the Quarantine Grid:  10-12MB RAM

With Quarantine Grid: 225MB RAM

In both cases, the RAM usage fluctuates up and down with load but I do not see anything like a memory leak at this point on any of my installs.

Dan S.

 

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2003 at 2:14am

I just upgraded today to 195 and here are the results.

I have it running on 2 servers. and both are sitting at 50 megs.

 

One server has processed 17 messages (this is my backup smtp server)

The other server has processed about 100.

Not sure why it's using all that mem, but it's definitly slower than before. I am going to leave it running and see if it consumes anymore mem, although I can't see why it needs that much mem!

Oh, and because I know people will say it. If I refresh the q, the mem jumps up and as soon as I hide it, it goes back down, so I don't think that is the problem.

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2003 at 7:15pm

Steve,

The high memory usage while displaying the quarantined items is not actually a leak. Displaying all that data does actually require that much RAM. Unfortunately right now this is out of our control, the graphic grid control used by our compiler uses up all that RAM when displaying the items. Hiding the list releases all memory used, without any leaks.

We're looking for a replacement component.

Roberto F.
LogSat Software

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2003 at 7:20pm

Kevin,

I'm still not sure we received a definite answer on this. Do you see a usage of 100s of MB even after hitting the "Hide List" button? We do know about the high memory usage while displaying items in the list, please see Re: Mem Leak?  for more. But high ram while the grid is empty is not normal.

Roberto F
LogSat Software

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2003 at 7:39pm
Yeah, it does it eitherway. If I refresh the grid you can see the mem go up, and as soon as I hide it, it goes down. I have even tried clearing out the database so it has nothing quaruntined, and it still does it.
Back to Top
stevew View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote stevew Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2003 at 9:15pm

Ok, it not a "leak" but the usage is high for the size load and much higher than the amount used by the earlier version.

I have restarted it and it has been running for the last two days and it has processed 0 messages and has 0 quarantined and is using 30 megs, hidding does not change it.

Could it be the build of mysql or the ODBC driver, I am running the lastest version for mysql?

Any ideas as what to try?

Thanks
- Steve

 

Back to Top
George View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote George Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2003 at 11:42pm

It could be the ODBC driver causing the problem. I am using UDL to connect to my SAL server and don't see near the memory useage that everyone is seeing.

With Grid mem = 22,560K

Without Grid = 9600K

CPU 3-40%

Processes = 46

Database size = 152MB

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 12:08pm

Well guys, Dan and I spend 3.5 hours on this last night and we have a solution for the mysql guys!

We found that if you run mysql odbc driver 3.51 spamfilter would have a mem leak. Even with roberto's suggested settings:

(http://www.logsat.com/SpamFilter/images/MySQLODBC.gif)

My backup sever which processed 12 spam messages last night, sat at 80 megs. While my prod server which processed about 200 normally, would have been at around 100 megs, However, I uninstalled the 3.51 client and installed the 2.50 mysql odbc client and it worked like a charm, sits at 8 megs which is perfectly acceptable.

Hope this helps!

Kevin

 

 

Back to Top
Stevew View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote Stevew Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 2:05pm

Thanks I will try it and let you know.

- Steve

Back to Top
Sean View Drop Down
Groupie
Groupie
Avatar

Joined: 21 February 2005
Location: United States
Status: Offline
Points: 81
Post Options Post Options   Thanks (0) Thanks(0)   Quote Sean Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 5:15pm

This was the problem we were having in this post: http://www.logsat.com/spamfilter/forums/showmessage.asp?messageID=1451

I will try the 2.50 OBDC driver and see if it helps.

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 5:31pm

Good stuff, let us know how this works for you.

btw, my process was to delete any 3.51 dsn's uninstall the 3.51 odbc driver, then install 2.50 and it worked for me, however you should just have to install the 2.50 driver, delete the dsn and recreate it and re-configure spamfitler and you'll be good to go.

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 5:49pm

well we're not home free yet....

I have noticed since back reving to Mysql ODBC driver 2.50 when a message comes in and it is to be quarantined, I get the message "Exception occorred during RECIEVEMESSAGE: Could not convert varient of type (null) into type (integer)

at this point it disconnects and the message is not quaruntined. Is anyone else having this problem who has gone back to ver 2.50?

Back to Top
Sean View Drop Down
Groupie
Groupie
Avatar

Joined: 21 February 2005
Location: United States
Status: Offline
Points: 81
Post Options Post Options   Thanks (0) Thanks(0)   Quote Sean Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 5:54pm

I was just about to say that the 2.50 driver does not work. We are getting all kinds of execption errors such as yours. Also if we try to delete or deliver a message from quarentine view, nothing happens.

Reverted back to 3.51 driver for now.

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 7:12pm
I was just going to post that, after digging deeper I found numerous problems with the 2.50 driver. I have reverted back to 3.51 I will just have to restart the service every 24 hours for now. I do know that this problem has been recreated by Dan and possible Roberto. However it seems like it is limited to NT4.0 and not windows 2000.....
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 11:46pm

All,

I have 2 test beds set up and never having really used MySql before, this was a task. I have a autoresopnder endless loop set up and hit both servers with 200 messages / min

Both servers: SpamFilter build 195

Both Servers: ODBC Driver 3.5

After 2 hours

On the NT4 (SP6a) box, the server crashed and burnned in very short order.  I have so many errors comming from God knows where that I went into sensory overload trying to look at them

On the Win2K box ... well, it is still running and I have about 40meg being used by SpamFilter.

On the NT box, I changed my UDL to point to my Microsoft (how I hate that word) MS-SQL 2000 SP3(+) and it is running as I write this and looks like it will be fine.

So ... what the heck?  I just do not have enough experience with MySQL to go much further.  I have, in fact, sent all my test information over to Roberto and I believe he is setting a test up on and NT box but I do not want to speak for him.

Regards,

Dan S.

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2003 at 11:56pm

Kevin,

Now that you and George have found this to happen on NT4, we were finally able to reproduce the leak on an NT4 server with SpamFilter running as a service. What is odd is that the leak only happened once, we were not able to reproduce it again... Working with the debug data we collected, we may have found a possible problem. Non matter what we did, after the code change the leak was unreproduceable.

We've just uploaded beta build .196 in the registered user are. If you can give it a try, please let us know if a miracle happened and the leak is gone.

If not, can you please try running SpamFilter in standalone mode rather than as a service, and let us know if that leaks as well?

If so, can you please run SpamFilter from the command line with the follwoing switch:

SpamFilter -d

and let it run until you see 50-100MB leaking. When you close SpamFilter, the -d switch  will show a debug window to come up. Please email us the contents.

If the leak only occurs in the service, please add the -d switch in the registry at:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SpamFilter,

edit the ImagePath value and add the -d option at the end of the existing value, ex:

C:\\SpamFilter\\SpamFilterSvc.exe /startedbyscm:F9D641D8-40E26D57-SpamFilter -d

Roberto F.
LogSat Software

 

 

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 14 August 2003 at 12:23am

It leaked in standalone mode as well. I will install it on my backup server and give it a try right now and see how it does in a few hours.

 

Kevin

Back to Top
Sean View Drop Down
Groupie
Groupie
Avatar

Joined: 21 February 2005
Location: United States
Status: Offline
Points: 81
Post Options Post Options   Thanks (0) Thanks(0)   Quote Sean Quote  Post ReplyReply Direct Link To This Post Posted: 14 August 2003 at 8:40am
I hate to post this, but. Has anyone tried to see if the leak occurs while running MySQL on a seperate box from SpamFilter. If so was the Box running MySQL window 2K or NT 4.0. This should help determine if it is related to the MySQL OBDC driver running on NT 4.0. I'm sorry, I would test this myself, but aftering losing my Email server last week (hence lack of post latley from me), I am still in the process recreating all processes on the Server.
Back to Top
steveW View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote steveW Quote  Post ReplyReply Direct Link To This Post Posted: 14 August 2003 at 9:56am

Hi all,

It has been about 24 and after switching to the 2.5 driver the memory usage has remained at 9megs with the list hidden and no load.

Will upgrade to the latest version and see what happens.

Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 14 August 2003 at 1:15pm
Actually that is how I am running.....We know it's a problem between Spamfilter and the odbc driver on NT4
Back to Top
 Post Reply Post Reply Page  12>
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.303 seconds.