Spam Filter ISP Forums : logparse rrdtool https://www.logsat.com/spamfilter/forums/ Mon, 15 Jun 2026 01:23:07 +0000 Sun, 26 Jun 2016 17:18:50 +0000 http://blogs.law.harvard.edu/tech/rss Web Wiz Forums 11.04 360 https://www.logsat.com/spamfilter/forums/RSS_post_feed.asp?TID=7136 <![CDATA[Spam Filter ISP Forums]]> https://www.logsat.com/spamfilter/forums/forum_images/web_wiz_forums.png https://www.logsat.com/spamfilter/forums/ <![CDATA[logparse rrdtool : Hi, I was on a short break last...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14834&title=logparse-rrdtool#14834 Author: danego
Subject: 7136
Posted: 26 June 2016 at 5:18pm

Hi, I was on a short break last week.
Good to read that you modified the script to your situation and that you found it useful.
 
I never had the need to change the logrotation so I was unaware of this feature in logsatSmile
 
in my post I also had a url to the rrdtool website to download the exe.
 
When I read what you have modified it shouldn't be that hard to combine multiple servers?
It's a more or less simple add up of some numbers :)
 
One thing I want to add to the script is the number of files in the queue folder.
This should be very low but if there is a problem with your mailserver this queue folder could go skyhighSmile
]]> Sun, 26 Jun 2016 17:18:50 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14834&title=logparse-rrdtool#14834 <![CDATA[logparse rrdtool : Have had some success on this...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14833&title=logparse-rrdtool#14833 Author: yapadu
Subject: 7136
Posted: 19 June 2016 at 9:50am

Have had some success on this end, thanks for the code.

I've modified it so it does not need to run every 5 minutes, it can calculate the timestamp needed from the logs themselves.

The result is it can now also process historical data (assuming you start the RRD DB back far enough).

Now I am looking for a way to merge the databases from multiple servers to get one complete picture of everything.
]]> Sun, 19 Jun 2016 09:50:10 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14833&title=logparse-rrdtool#14833 <![CDATA[logparse rrdtool : Thanks for the code. Thought I...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14831&title=logparse-rrdtool#14831 Author: yapadu
Subject: 7136
Posted: 19 June 2016 at 1:13am

Thanks for the code.  Thought I would share my findings so far.

Just running it, VBS crashes on this line:

EpochUTC = DateDiff("s", "01/01/1970 00:00:00", Left(DateAdd("n", UTC_Offset, Now()),Len(DateAdd("n", UTC_Offset, Now()))-2) & "00")

I think you are trying to get the date (with UTC offset taken into consideration) and then remove the seconds?

I converted it to this to make it run on my system:

EpochUTC = DateDiff("s", "01/01/1970 00:00:00", FormatDateTime(DateAdd("n", UTC_Offset, Now()),2) & " " & FormatDateTime(DateAdd("n", UTC_Offset, Now()),4))
That gets past the error.  The link to rrdtool does not contain any .exe file that I can find.

I found a compiled copy on this page, way at the bottom (direct link).  That has a compiled version and some associated files it needs.  I just put them all into the same directory as your VBS and it runs now.

Now I'm working on a problem where the VBS assumes the logs are all in the format yyyymmdd.log but spamfilter has an option to rotate the logs more frequently and may contain a number after, like this yyyymmdd-x.log and the script does not pick up on that.


Edited by yapadu - 19 June 2016 at 1:32am]]> Sun, 19 Jun 2016 01:13:02 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14831&title=logparse-rrdtool#14831 <![CDATA[logparse rrdtool : Ok here is the download link...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14830&title=logparse-rrdtool#14830 Author: danego
Subject: 7136
Posted: 16 June 2016 at 4:27pm

Ok here is the download link of the script with an extra textfile with the rrdtool commands used.
 
http://www.danego.net/spamfilter/logsatparser_v1.zip
 
You also need rrdtool for windows :
 
http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.2.30-win32-perl510.zip

There is an rrdtool.exe in of the the folders.
 
 
I have copied our current running script.
(posted the latest day and week graph on www.danego.net/spamfilter)
 
Always curious if someone else but me gets it up and running without any problems.Smile
 
Be aware you need to run it at least three times with 5 minutes intervals before you see something appear in the graphs.
 
It counts the different values from the log and adds this to the rrdtool database.
rrdtool recalculates this to a value per second and during the graph generation it gets multiplied by 60 to get a count per minute.
So the value is per minute from a 5 minute average.
 
 
 
]]> Thu, 16 Jun 2016 16:27:07 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14830&title=logparse-rrdtool#14830 <![CDATA[logparse rrdtool : That is certainly impressive looking,...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14829&title=logparse-rrdtool#14829 Author: yapadu
Subject: 7136
Posted: 13 June 2016 at 9:26pm

That is certainly impressive looking, I would love to get a copy of that as well.
]]> Mon, 13 Jun 2016 21:26:50 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14829&title=logparse-rrdtool#14829 <![CDATA[logparse rrdtool : Great job danego! We ourselves...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14828&title=logparse-rrdtool#14828 Author: LogSat
Subject: 7136
Posted: 09 June 2016 at 5:54pm

Great job danego! We ourselves here at LogSat could use your script on our server :-) Plus 1 for us for a copy of it!]]> Thu, 09 Jun 2016 17:54:50 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14828&title=logparse-rrdtool#14828 <![CDATA[logparse rrdtool : Here is an image with an example...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14827&title=logparse-rrdtool#14827 Author: danego
Subject: 7136
Posted: 09 June 2016 at 10:18am

Here is an image with an example of this
 
http://danego.net/spamfilter/
]]> Thu, 09 Jun 2016 10:18:18 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14827&title=logparse-rrdtool#14827 <![CDATA[logparse rrdtool : Hi,I have written a vbs script...]]> https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14826&title=logparse-rrdtool#14826 Author: danego
Subject: 7136
Posted: 08 June 2016 at 5:25pm

Hi,
 
I have written a vbs script to get some information from the logfile and put it in rrdtool.
Then we generate an image with rrdtool.
 
At the moment I collect #connections, rejected mails, forwarded mails, quarantined mails.
 
You can schedule the script with the windows task scheduler.
I run the script every 5 minutes to get the data in rrdtool.
The image then shows the # of messages per minute (this is the average of the 5 minutes)
 
The script writes a tempfile in the logfile folder to keep up where it left off.
It detects a log rotate and cleans up the tempfile from yesterdays logfile before it starts with the new day.
 
The script is almost done, was just curious if anyone is interested.
(needs more comment and I bet it's not monkey proof yet :)
 
Regards,
Emiel.
 
]]> Wed, 08 Jun 2016 17:25:28 +0000 https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=7136&PID=14826&title=logparse-rrdtool#14826