Unstable and freezing when using quarantene |
Post Reply 
|
| Author | |
morten44 
Groupie 
Joined: 07 March 2008 Status: Offline Points: 74 |
 Post Options
 Thanks(0)
 Quote Reply
 Topic: Unstable and freezing when using quarantenePosted: 06 September 2009 at 5:16pm |
|
Hi
We are running the latest version of spamfilter first on a windows 2000 server, then now later on Windows 2003.
We do use quarantene option and we store it for about 10 days.
Then it will delete mails every 30min.
I will guess we get about 75000 - 100.000 mails through our filter everyday.
I will guess about 25% goes to quarantene database. This is all Reverse DNS as we have seen that alot of big institutions etc do not use it and we need a tool to fish them up when the filter correctly put them in quarantene. The problems starts after about 3 weeks.
The program starts to freeze and it stops responding.
When we restart the server and start spamfilter, it freezes and only when we disable the quarantene database, then the program works fine.
We are using it with mysql 5.0
My question is:
Is there a known problem using quarantene database with the quantity of mails I am talking about?
Is it worth trying SQL instead of MySQL?
We do not use the quarantene database viewer within the application as we have before been adviced not to use this.
Its the 3rd time we have installed the system and it seems to be working fine for about 3 weeks, then we have no other options that turning off the quarantene database to secure that the system will run.
Any advice/comments are welcome
Kind Regards
Morten
|
|
 |
|
|
LogSat
Admin Group 
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 06 September 2009 at 9:54pm |
|
Morten,
There are no known problems relating SpamFilter with MySQL (not MSSQL) and the quarantine database. If you could please zip and provide us with 2 days' worth of SpamFilter logfiles for the day SpamFilter last crash and the day prior to that (we'd like two days in case there's events that lead up to the crash several hours before), we'll try to go thru them to see if we see any problems. If the logs are over 8MB in size (likely seeing your traffic), I'll be sending you via a PM the ftp login info for out FTP site.
|
|
|
|
|
morten44
Groupie
Joined: 07 March 2008 Status: Offline Points: 74 |
Post Options
Thanks(0)
Quote Reply
Posted: 07 September 2009 at 6:29am |
|
Hi Roberto Thanks for reply I have extracted a small part of the logfile that might show what the problem is. its now on your ftp server: logfile_morten44.txt
2 days of log files i 1GB!!!!!! so its a little hard to get it uploaded :)
Does this log file show you anything that I should look into? I can extract parts of logfiles and prob get 2 days down to 100MB, removing non important parts. WOuld that be needed or can you use what I have attached to anything?
|
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 07 September 2009 at 10:05am |
|
Morten,
Unfortunately the log entries you sent do contain several database errors (most of them due to the MySQL Server not being available), but do not show SpamFilter crashing. In regards to the logfile, if you compress the logfiles with WinZip (or Windows' built-in zip compression), the 1GB logfile sizes will likely shrink down to much less than 100MB making it much easier to transfer (but we ourselves will still have to debug 1GB worth of data....). In addition to errors in your logs about MySQL being unavailable, we also saw an unusual error occurring when cleaning up the quarantine. The logs report that this SQL statement is invalid: DELETE DISTINCTROW tblMsgs.* FROM tblMsgs LEFT JOIN tblQuarantine ON tblMsgs.MsgID = tblQuarantine.MsgID WHERE (tblQuarantine.MsgID IS NULL); Would you mind running that SQL statement on your SpamFilter database and let us know if it reports any errors?
|
|
|
|
|
morten44
Groupie
Joined: 07 March 2008 Status: Offline Points: 74 |
Post Options
Thanks(0)
Quote Reply
Posted: 07 September 2009 at 6:10pm |
|
Hi
Thanks for your answer.
Yes I can see that I had some issues with Mysql.
I started it and it stopped again straight away.
There where some errors in log file that size of log file was different than specified in .cnf file...
hm did not have a .cnf file, only .ini file. I created one and now the service starts.
Not sure why it worked fine for 1 month, then this happened. Anyway, I did run the command suggested and it did fail
We are currently not running ISP spamfilter on this mail server, due to the problem, but guess that should not cause the statement to fail?
Regarding the log files.
I will try to get them zipped and uploaded some point tomorrow as I can not get to do it at the moment I wonder why it fails?
Wonder if it ever has worked. Would you like some logfiles from when it worked fine as well, just to check if this statement always have failed?
Kind Regards
Morten
|
|
|
|
|
morten44
Groupie
Joined: 07 March 2008 Status: Offline Points: 74 |
Post Options
Thanks(0)
Quote Reply
Posted: 07 September 2009 at 7:50pm |
|
Hi Again.
I managed to upload 2 log files over 2 days. I think this is when we started to experience the problems first time. Kind Regards Morten |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 08 September 2009 at 11:36am |
|
Morten,
We've gone thru your files, and found out that the SQL statement that is giving an error: DELETE DISTINCTROW tblMsgs.* FROM tblMsgs LEFT JOIN tblQuarantine ON tblMsgs.MsgID = tblQuarantine.MsgID WHERE (tblQuarantine.MsgID IS NULL); is a "generic" SQL statement compatible with MSAccess being executed as a "last resort", since the correct statement that was to be executed for MySQL resulted in an error just prior to that. Seeing the other issues you were having with MySQL this is normal and is not something to worry about. Going back to the "freezing", from the logs we see that SpamFilter was shut down cleanly at: 07/24/09 00:09:18:343 -- Shutting down all threads. Please wait up to 15-20 seconds.... but something went wrong with the restart, as sometime between 00:14 and 00:20 the SpamFilter process was killed and SpamFilter restarted. We did not see any freezes outside that timeframe. When you say "freeze and stop responding", can you let us know more in detail what exactly you see occurring? As a side-note, in addition to the issues with MySQL, we saw several out-of-memory errors on the 23rd, occurring when SpamFilter attempted to reload the Bayesian database. Just to be on the safe side, can you please reset the Bayesian database? To do so, just stop SpamFilter, delete (or rename) the SpamFilter/corpus directory, and then restart SpamFilter? That should reset the corpus database for the bayesian filter in case it was corrupted.
|
|
|
|
|
morten44
Groupie
Joined: 07 March 2008 Status: Offline Points: 74 |
Post Options
Thanks(0)
Quote Reply
Posted: 08 September 2009 at 12:51pm |
|
Hi again.
Thanks for your answer. Regarding the error message (statement). OK I understad that this is not a problem then Regarding crashing. What happens is that at some point I could see that spamfilter was not sending forward any mails to Imail at all. I could not close Spamfilter, could not close service and could not even shut down windows. I had to get someone locally to the server to pull the plug and restart. When it came up again, windows was working fine, but as soon as Spamfilter tried to start it freezes like you see on the picture http://www.e-advice.dk/files/freezes.jpg I then managed from a remote server to set the spamfilter service to start manually. Got my friend to pull out the plug again, restart When I then go to Services and start, the same thing happens. Once it did start after hanging on the "freeze" screen for 5min, but then inbound connection quickly ran up to over 600 and it did not deliver mails again. In the end, I had to give up. I started our old mail server that before had spamfilter installed but crashed in the same way after some time in use. I deactivated the quarantene and that have been running fine ever since. So our aim is to find out where to go from here. We need to have quarantene database up and running again, as we have several users who need access to this due to "reverse dns issues" catching many real mails, so they need a way to get access to them. We have had the issue 2 times on 2 different servers, so we have to in a way conclude if we should stay using MYSQL or maybe try MSSQL instead if it turns out that issue is only with MYSQL. MYSQL is a standard 5.0 out of the box installation and is only serving spamfilter. I have one more crash date. If you need more logfiles to see if that can show something else, just tell me. Any Advice are welcome :) |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 08 September 2009 at 1:50pm |
|
Morten,
I'm afraid we won't be able to answer with 100% certainty here due to the several problem-factors that are involve. let me explain a few issues that when combined as in your case lead to uncertainty. 1. When SpamFilter starts up (as is the case in your screenshot), the GUI will appear frozen and won't refresh until SpamFilter is fully loaded. This can take a few minutes, especially if some of the black/white lists are large (in the order of 1MB or more). 2. If accessing SpamFilter's server via RDP (Remote Desktop/Terminal Services) on a slow connection (< 10Mb/s), and there are several (30/40+) concurrent connections, and you are viewing the Activity Log window, the screen refreshes caused by the activity log window may occur so fast that RDP is not able to keep up, causing the entire RDP session (not just SpamFilter) to appear frozen - mouse clicks and screen refreshes will be extremely slow on the entire RDP window. When this happens, connections may build up on SpamFilter as the software is unable to flush the screen causing internal queues to affect the email handling queues as well. We are aware of this flaw, as RDP issues should not affect email flow, and are working on fixes for it. 3. In the logs we received, MySQL connectivity was not working, causing practically every single email that was rejected to fail the database archiving procedure. When this happens, the quarantined emails are stored in a queue on disk. SpamFilter will attempt to re-archive this emails when the DB is back online, and during SpamFilter's startup, this archival process can use up more resources if there are tens of thousands (or hundreds) of queued emails to scan upon startup. These emails are stored in the \SpamFilter\Quarantine directory. This directory should be empty at all times, with the exception of the handful of files being processed at any instant. If there's tens of thousand or more of files in there, this indicates a database issue. 4. The bayesian database *appeared* corrupted, or there was a low memory issue on the server (corruption was however more likely). All of these things could have contributed to causing problems. This said, is the issue still occurring? Was MySQL repaired? There are no known issues with MySQL, but if the database itself becomes corrupted then yes, this will affect SpamFilter. Do you have a large amount of files in the \SpamFilter\Quarantine ? If the problems are still ongoing, are you able to allow us to remote into the server to have a better look?
|
|
|
|
|
morten44
Groupie
Joined: 07 March 2008 Status: Offline Points: 74 |
Post Options
Thanks(0)
Quote Reply
Posted: 09 September 2009 at 6:19am |
|
Hi
Thanks for your answer I checked the \SpamFilter\Quarantine and its so full that I can not even open the folder in Explore. It freezes and hangs for up to 30min and I have to restart the server. I then tried in a Command Prompt and when typing "dir" inside the quarantene folder, it freezes for about 15min and then it starts to list all the files. I have let it run for 2 hours now and still listing all the files so there must be several thousand/millions. Had to break the script and do a del *.* I guess it will take a couple of hours to get all files deleted. So conclution must be that this is big because mysql stopped responding. This beeing big causes spamfilter to "freeze" when starting and sometime for long time as maybe its trying to process what ever is inside this folder. The memory fault can be respolved by refreshing the database you mentioned. If we then can get mysql to run stable, that should resolve the problem. ITs not easy to get access as we do not use RDP but something called Netop for remote assistance. If this conclusion stand for now, I will go ahead and try that. If it fails again, I will come back to you so we can look into how we can get you remotely onto the server Kind Regards Morten |
|
|
|
Topic Options|
Post Reply
|
|
Tweet
|
| Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
This page was generated in 0.172 seconds.