SFDB HTTP Errors
Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5853
Printed Date: 05 February 2025 at 11:07am
Topic: SFDB HTTP Errors
Posted By: mikek
Subject: SFDB HTTP Errors
Date Posted: 31 October 2006 at 10:01am
|
Hi Roberto I'm seeing quite a lot of HTTP Errors lately when SF queries and updates the SFDB. I counted 6549 successful queries/updates today and 1992 HTTP Errors. Would you like me to send you our log file or is it a known problem? Cheers, Mike |
Replies:
Posted By: mikek
Date Posted: 31 October 2006 at 10:04am
| "SFDB_Weberror has reached it's limit, SFDB checks are paused temporarily" was output 812 times today in the log... :-( |
Posted By: LogSat
Date Posted: 31 October 2006 at 10:06am
|
mikek, What build of SpamFilter are you using? Are they timeout problems? If so, we may have added too short of a timeout on the new 605 build to fix a problem with the previous versions. We're actually seing this on our own server as well, but with a much lower percentage than you. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: mikek
Date Posted: 31 October 2006 at 10:08am
| maybe the timeouts are too short when the SFDB Server is a bit busy... or on trans-atlantic internet connections as ours (22 hops, ~150ms RTT)... |
Posted By: mikek
Date Posted: 31 October 2006 at 10:08am
| using build 605, and yes, the usual message is "read timeout"... |
Posted By: Marco
Date Posted: 01 November 2006 at 4:54am
|
I think maybe a couple of SFDB mirrors across the net are called for. Perhaps some of the major SFI users (ISP's) would/should volunteer? Afterall, major ISP's have the most to gain by having a good SFDB lookup system. SFDB can only go one direction: bigger, with more query's to handle with every new customer logsat gets. Roberto, maybe it's wise to test the water to see who would cooperate. It's certainly wise to plan ahead on this one. Afterall, resources are allways limited. ------------- Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams |
Posted By: LogSat
Date Posted: 01 November 2006 at 8:47am
|
Actually CPU on the server we use for the SFDB is practically never above 10% peaks, and the average load is only 4%-5%. Mirorring won't help as the load is not the problem. The issue is a bit different, and has to do with locking issues in SQL server when inserting new records. We're looking into this now to see what can be done on the server's end to minimize these timeouts. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: LogSat
Date Posted: 01 November 2006 at 8:21pm
|
Update. This is one of the times where a person could take a server and throw it out of the window. We just found out that on our IIS server, the Microsoft's ADOVBS.INC file, which contains all the ADO-related constants used by IIS and Active Server Pages, is missing the following line: Const adAsyncExecute = &H00000010 That happens to be the constant we use (or better, we *THOUGHT* we were using) to allow the SFDBUpload queries to execute asynchronously. In simple terms, this means that once the web page on our SFDB web server that uploads new SFDB records is called by SpamFilter, the ASP code tells our database server to process the stored procedure, and to then return immediately. The key piece here is "then return immediately". This is done by using the adAsyncExecute constant in the ASP code. But since that constant was missing from the Microsoft's file, the ASP pages were waiting for the SQL statement to complete before terminating. This is why we have all been seing timeout errors recently, as while usually the query takes milliseconds to run, sometimes it does take a few seconds if the records are locked. At 20:08:00 we added the missing entry, so the timeout errors should hopefully stop, or at least greatly reduce. We'll be monitoring the situation. In the meantime I'll have a camomille to cool down the nerves... We apologize to everyone for the problems! ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: mikek
Date Posted: 02 November 2006 at 2:14am
|
Thanks for the update, I will observe the logs today and report later... BTW: as an ASP programmer myself, I can give you the following tip: instead of including adovbs.inc just place the following line at the top of your script, or in global.asa: <!--METADATA TYPE="typelib" NAME="Microsoft ActiveX Data Objects 2.8 Library" UUID="{2A75196C-D9EB-4129-B803-931327F72D5C}" VERSION="2.8"--> |
Posted By: mikek
Date Posted: 02 November 2006 at 6:34am
|
A bit better, but not much. Statistics for the last 12 hours: Successful queries: 6666 HTTP Errors: 1005 SFDB checks are paused temporarily: 509 |
Posted By: LogSat
Date Posted: 02 November 2006 at 7:32am
|
Yes, we see the same on our end. Looks like the timeout is not caused by the query, as we've verified that the calls are now indeed non-blocking, and they were not before. Back to the drawing board. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: mikek
Date Posted: 02 November 2006 at 7:40am
| ok, here's a wild guess: check the ASP -> Requests Queued with perfmon and see if there are a lot of queued requests... |
Posted By: dcook
Date Posted: 03 November 2006 at 12:15pm
|
I too have had 980 of these errors in thae last 6 days --- HTTP Error in SFDBUploadIP check:Socket Error # 10054 -- Connection reset by peer. I searched under socket error to find them. ------------- Dwight www.vividmix.com |
Posted By: Lee
Date Posted: 11 November 2006 at 9:53pm
|
I am also seeing quite a few of the same errors. Lee 11/11/06 21:51:52:656 -- (2133) HTTP Error in SFDBUploadIP check:HTTP/1.1 500 Internal Server Error 11/11/06 21:51:52:826 -- (2133) SFDB - Added 71.251.35.252 - Response: 11/11/06 21:51:53:006 -- (2133) Disconnect |
Posted By: StevenJohns
Date Posted: 29 November 2006 at 6:09pm
|
Roberto,
Looks like this isn't quite fixed yet... I'm getting quite a few of these... 11/29/06 01:13:55:531 -- (5536) HTTP Error in DoSFDBCheck:Read Timeout |
Posted By: StevenJohns
Date Posted: 29 November 2006 at 6:11pm
|
Actually,
I'm also getting a few of these....
11/29/06 02:51:13:390 -- (7136) HTTP Error in SFDBUploadIP check:HTTP/1.1 500 Internal Server Error 11/29/06 06:23:10:125 -- (9204) HTTP Error in GetSFDBStats:Connect timed out.
|
Posted By: Wayne
Date Posted: 02 December 2008 at 10:31am
Push up 
Roberto, is there something new about this problem? Because we've got 1'410 "HTTP Error in GetSFDBStats:Connect timed out" in 24h. Thx Wayne ------------- SF4.5.0.1-beta |
Posted By: Desperado
Date Posted: 02 December 2008 at 12:09pm
|
Hmmm ... not sure if this means anything or helps but .... in 5 days:
1 of these: HTTP Error in GetSFDBStats:HTTP/1.1 503 Service Unavailable
3 of these: HTTP Error in SFDBUploadIP check:HTTP/1.1 503 Service Unavailable
All events happened around the same time ... within a 1 minute period. ------------- The Desperado Dan Seligmann. Work: http://www.mags.net Personal: http://www.desperado.com |
Posted By: LogSat
Date Posted: 02 December 2008 at 11:00pm
|
Wayne,
SpamFilter will poll stats from our SFDC and SFDB databases every two minutes to display the count of IPs and of spam hashes being blacklisted at any time. Twice every two minutes indicates 1440 queries every 24 hours, which is pretty close to the number of errors you see. If you see those errors exactly 2 minutes apart, it means that your SpamFilter server is not able to communicate with our website (specifically http://sfdb.logsat.com) to retrieve that information. Please also note that if your server is not able to make HTTP requests to that same URL, SpamFilter will not be able to use our SFDB and SFDC filters, which are very effective in stopping spam. Can you please ensure that the server running SpamFilter is able to make those web connections? Proxy servers and/or web filtering appliances/applications in your network could be preventing that from occurring. You can verify that this process works by going to the "Settings" - "Configuration" tab in SpamFilter, and then clicking on the "SFDB Settings" button. In the window that appears, you should see in the bottom right a value for "Current IPs listed in the SFDB" in the order of 100,000-500,000. ------------- Roberto Franceschetti http://www.logsat.com" rel="nofollow - LogSat Software http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP |
Posted By: Wayne
Date Posted: 03 December 2008 at 4:52am
|
Roberto
Thank you for your input, the blame was the protection profile at the spamfilter rule of our firewall 
Sorry for that! Kind regards Wayne ------------- SF4.5.0.1-beta |