logparse rrdtool |
Post Reply 
|
| Author | |
danego 
Newbie 
Joined: 22 March 2011 Location: NL Status: Offline Points: 9 |
 Post Options
 Thanks(0)
 Quote Reply
 Topic: logparse rrdtoolPosted: 08 June 2016 at 5:25pm |
|
Hi,
I have written a vbs script to get some information from the logfile and put it in rrdtool. Then we generate an image with rrdtool. At the moment I collect #connections, rejected mails, forwarded mails, quarantined mails. You can schedule the script with the windows task scheduler. I run the script every 5 minutes to get the data in rrdtool. The image then shows the # of messages per minute (this is the average of the 5 minutes) The script writes a tempfile in the logfile folder to keep up where it left off. It detects a log rotate and cleans up the tempfile from yesterdays logfile before it starts with the new day. The script is almost done, was just curious if anyone is interested. (needs more comment and I bet it's not monkey proof yet :) Regards, Emiel. |
|
 |
|
|
danego
Newbie
Joined: 22 March 2011 Location: NL Status: Offline Points: 9 |
Post Options
Thanks(0)
Quote Reply
Posted: 09 June 2016 at 10:18am |
|
|
|
LogSat
Admin Group 
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 09 June 2016 at 5:54pm |
|
Great job danego! We ourselves here at LogSat could use your script on our server :-) Plus 1 for us for a copy of it!
|
|
|
|
|
yapadu
Senior Member 
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 13 June 2016 at 9:26pm |
|
That is certainly impressive looking, I would love to get a copy of that as well.
|
|
|
--------------------------------------------------------------
I am a user of SF, not an employee. Use any advice offered at your own risk. |
|
|
|
|
danego
Newbie
Joined: 22 March 2011 Location: NL Status: Offline Points: 9 |
Post Options
Thanks(0)
Quote Reply
Posted: 16 June 2016 at 4:27pm |
|
Ok here is the download link of the script with an extra textfile with the rrdtool commands used.
You also need rrdtool for windows : There is an rrdtool.exe in of the the folders. I have copied our current running script. (posted the latest day and week graph on www.danego.net/spamfilter) Always curious if someone else but me gets it up and running without any problems.  Be aware you need to run it at least three times with 5 minutes intervals before you see something appear in the graphs. It counts the different values from the log and adds this to the rrdtool database. rrdtool recalculates this to a value per second and during the graph generation it gets multiplied by 60 to get a count per minute. So the value is per minute from a 5 minute average. |
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 19 June 2016 at 1:13am |
|
Thanks for the code. Thought I would share my findings so far.
Just running it, VBS crashes on this line: EpochUTC = DateDiff("s", "01/01/1970 00:00:00", Left(DateAdd("n", UTC_Offset, Now()),Len(DateAdd("n", UTC_Offset, Now()))-2) & "00") I think you are trying to get the date (with UTC offset taken into consideration) and then remove the seconds? I converted it to this to make it run on my system: EpochUTC = DateDiff("s", "01/01/1970 00:00:00", FormatDateTime(DateAdd("n", UTC_Offset, Now()),2) & " " & FormatDateTime(DateAdd("n", UTC_Offset, Now()),4)) That gets past the error. The link to rrdtool does not contain any .exe file that I can find. I found a compiled copy on this page, way at the bottom (direct link). That has a compiled version and some associated files it needs. I just put them all into the same directory as your VBS and it runs now. Now I'm working on a problem where the VBS assumes the logs are all in the format yyyymmdd.log but spamfilter has an option to rotate the logs more frequently and may contain a number after, like this yyyymmdd-x.log and the script does not pick up on that. Edited by yapadu - 19 June 2016 at 1:32am |
|
|
--------------------------------------------------------------
I am a user of SF, not an employee. Use any advice offered at your own risk. |
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 19 June 2016 at 9:50am |
|
Have had some success on this end, thanks for the code.
I've modified it so it does not need to run every 5 minutes, it can calculate the timestamp needed from the logs themselves. The result is it can now also process historical data (assuming you start the RRD DB back far enough). Now I am looking for a way to merge the databases from multiple servers to get one complete picture of everything. |
|
|
--------------------------------------------------------------
I am a user of SF, not an employee. Use any advice offered at your own risk. |
|
|
|
|
danego
Newbie
Joined: 22 March 2011 Location: NL Status: Offline Points: 9 |
Post Options
Thanks(0)
Quote Reply
Posted: 26 June 2016 at 5:18pm |
|
Hi, I was on a short break last week.
Good to read that you modified the script to your situation and that you found it useful. I never had the need to change the logrotation so I was unaware of this feature in logsat in my post I also had a url to the rrdtool website to download the exe. When I read what you have modified it shouldn't be that hard to combine multiple servers? It's a more or less simple add up of some numbers :) One thing I want to add to the script is the number of files in the queue folder. This should be very low but if there is a problem with your mailserver this queue folder could go skyhigh
|
|
|
|
Topic Options|
Post Reply
|
|
Tweet
|
| Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
This page was generated in 0.102 seconds.