SFDB HTTP Errors |
Post Reply 
|
| Author | |
mikek 
Senior Member 
Joined: 22 February 2005 Location: Switzerland Status: Offline Points: 133 |
 Post Options
 Thanks(0)
 Quote Reply
 Topic: SFDB HTTP ErrorsPosted: 31 October 2006 at 10:01am |
|
Hi Roberto
I'm seeing quite a lot of HTTP Errors lately when SF queries and updates the SFDB. I counted 6549 successful queries/updates today and 1992 HTTP Errors. Would you like me to send you our log file or is it a known problem? Cheers, Mike |
|
 |
|
|
mikek
Senior Member
Joined: 22 February 2005 Location: Switzerland Status: Offline Points: 133 |
Post Options
Thanks(0)
Quote Reply
Posted: 31 October 2006 at 10:04am |
|
"SFDB_Weberror has reached it's limit, SFDB checks are paused temporarily" was output 812 times today in the log... :-(
|
|
|
|
|
LogSat
Admin Group 
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 31 October 2006 at 10:06am |
|
mikek,
What build of SpamFilter are you using? Are they timeout problems? If so, we may have added too short of a timeout on the new 605 build to fix a problem with the previous versions. We're actually seing this on our own server as well, but with a much lower percentage than you. |
|
|
|
|
mikek
Senior Member
Joined: 22 February 2005 Location: Switzerland Status: Offline Points: 133 |
Post Options
Thanks(0)
Quote Reply
Posted: 31 October 2006 at 10:08am |
|
maybe the timeouts are too short when the SFDB Server is a bit busy... or on trans-atlantic internet connections as ours (22 hops, ~150ms RTT)...
|
|
|
|
|
mikek
Senior Member
Joined: 22 February 2005 Location: Switzerland Status: Offline Points: 133 |
Post Options
Thanks(0)
Quote Reply
Posted: 31 October 2006 at 10:08am |
|
using build 605, and yes, the usual message is "read timeout"...
Edited by mikek |
|
|
|
|
Marco
Senior Member
Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137 |
Post Options
Thanks(0)
Quote Reply
Posted: 01 November 2006 at 4:54am |
|
I think maybe a couple of SFDB mirrors across the net are called for. Perhaps some of the major SFI users (ISP's) would/should volunteer? Afterall, major ISP's have the most to gain by having a good SFDB lookup system. SFDB can only go one direction: bigger, with more query's to handle with every new customer logsat gets. Roberto, maybe it's wise to test the water to see who would cooperate. It's certainly wise to plan ahead on this one. Afterall, resources are allways limited. |
|
|
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 01 November 2006 at 8:47am |
|
Actually CPU on the server we use for the SFDB is practically never above 10% peaks, and the average load is only 4%-5%. Mirorring won't help as the load is not the problem.
The issue is a bit different, and has to do with locking issues in SQL server when inserting new records. We're looking into this now to see what can be done on the server's end to minimize these timeouts. |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 01 November 2006 at 8:21pm |
|
Update.
This is one of the times where a person could take a server and throw it out of the window. We just found out that on our IIS server, the Microsoft's ADOVBS.INC file, which contains all the ADO-related constants used by IIS and Active Server Pages, is missing the following line: Const adAsyncExecute = &H00000010 That happens to be the constant we use (or better, we *THOUGHT* we were using) to allow the SFDBUpload queries to execute asynchronously. In simple terms, this means that once the web page on our SFDB web server that uploads new SFDB records is called by SpamFilter, the ASP code tells our database server to process the stored procedure, and to then return immediately. The key piece here is "then return immediately". This is done by using the adAsyncExecute constant in the ASP code. But since that constant was missing from the Microsoft's file, the ASP pages were waiting for the SQL statement to complete before terminating. This is why we have all been seing timeout errors recently, as while usually the query takes milliseconds to run, sometimes it does take a few seconds if the records are locked. At 20:08:00 we added the missing entry, so the timeout errors should hopefully stop, or at least greatly reduce. We'll be monitoring the situation. In the meantime I'll have a camomille to cool down the nerves... We apologize to everyone for the problems! Edited by LogSat |
|
|
|
|
mikek
Senior Member
Joined: 22 February 2005 Location: Switzerland Status: Offline Points: 133 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 November 2006 at 2:14am |
|
Thanks for the update, I will observe the logs today and report later...
BTW: as an ASP programmer myself, I can give you the following tip: instead of including adovbs.inc just place the following line at the top of your script, or in global.asa: <!--METADATA TYPE="typelib" NAME="Microsoft ActiveX Data Objects 2.8 Library" UUID="{2A75196C-D9EB-4129-B803-931327F72D5C}" VERSION="2.8"--> |
|
|
|
|
mikek
Senior Member
Joined: 22 February 2005 Location: Switzerland Status: Offline Points: 133 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 November 2006 at 6:34am |
|
A bit better, but not much.
Statistics for the last 12 hours: Successful queries: 6666 HTTP Errors: 1005 SFDB checks are paused temporarily: 509 |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 November 2006 at 7:32am |
|
Yes, we see the same on our end. Looks like the timeout is not caused by the query, as we've verified that the calls are now indeed non-blocking, and they were not before. Back to the drawing board.
|
|
|
|
|
mikek
Senior Member
Joined: 22 February 2005 Location: Switzerland Status: Offline Points: 133 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 November 2006 at 7:40am |
|
ok, here's a wild guess: check the ASP -> Requests Queued with perfmon and see if there are a lot of queued requests...
|
|
|
|
|
dcook
Senior Member
Joined: 31 January 2005 Location: United States Status: Offline Points: 174 |
Post Options
Thanks(0)
Quote Reply
Posted: 03 November 2006 at 12:15pm |
|
I too have had 980 of these errors in thae last 6 days --- HTTP Error in SFDBUploadIP check:Socket Error # 10054 -- Connection reset by peer.
I searched under socket error to find them. |
|
|
Dwight
www.vividmix.com |
|
|
|
|
Lee
Groupie 
Joined: 04 February 2005 Location: United States Status: Offline Points: 50 |
Post Options
Thanks(0)
Quote Reply
Posted: 11 November 2006 at 9:53pm |
|
I am also seeing quite a few of the same errors.
Lee 11/11/06 21:51:52:656 -- (2133) HTTP Error in SFDBUploadIP check:HTTP/1.1 500 Internal Server Error 11/11/06 21:51:52:826 -- (2133) SFDB - Added 71.251.35.252 - Response: 11/11/06 21:51:53:006 -- (2133) Disconnect |
|
|
|
|
StevenJohns
Senior Member
Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
Quote Reply
Posted: 29 November 2006 at 6:09pm |
|
Roberto,
Looks like this isn't quite fixed yet... I'm getting quite a few of these... 11/29/06 01:13:55:531 -- (5536) HTTP Error in DoSFDBCheck:Read Timeout |
|
|
|
|
StevenJohns
Senior Member
Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
Quote Reply
Posted: 29 November 2006 at 6:11pm |
|
Actually,
I'm also getting a few of these....
11/29/06 02:51:13:390 -- (7136) HTTP Error in SFDBUploadIP check:HTTP/1.1 500 Internal Server Error 11/29/06 06:23:10:125 -- (9204) HTTP Error in GetSFDBStats:Connect timed out.
|
|
|
|
|
Wayne
Groupie
Joined: 29 August 2006 Location: Switzerland Status: Offline Points: 60 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 December 2008 at 10:31am |
|
Push up
Roberto, is there something new about this problem? Because we've got 1'410 "HTTP Error in GetSFDBStats:Connect timed out" in 24h. Thx Wayne |
|
|
SF4.5.0.1-beta
|
|
|
|
|
Desperado
Senior Member
Joined: 27 January 2005 Location: United States Status: Offline Points: 1143 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 December 2008 at 12:09pm |
|
Hmmm ... not sure if this means anything or helps but .... in 5 days:
1 of these: HTTP Error in GetSFDBStats:HTTP/1.1 503 Service Unavailable
3 of these: HTTP Error in SFDBUploadIP check:HTTP/1.1 503 Service Unavailable
All events happened around the same time ... within a 1 minute period.
|
|
|
The Desperado
Dan Seligmann. Work: http://www.mags.net Personal: http://www.desperado.com |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 December 2008 at 11:00pm |
|
Wayne,
SpamFilter will poll stats from our SFDC and SFDB databases every two minutes to display the count of IPs and of spam hashes being blacklisted at any time. Twice every two minutes indicates 1440 queries every 24 hours, which is pretty close to the number of errors you see. If you see those errors exactly 2 minutes apart, it means that your SpamFilter server is not able to communicate with our website (specifically http://sfdb.logsat.com) to retrieve that information. Please also note that if your server is not able to make HTTP requests to that same URL, SpamFilter will not be able to use our SFDB and SFDC filters, which are very effective in stopping spam. Can you please ensure that the server running SpamFilter is able to make those web connections? Proxy servers and/or web filtering appliances/applications in your network could be preventing that from occurring. You can verify that this process works by going to the "Settings" - "Configuration" tab in SpamFilter, and then clicking on the "SFDB Settings" button. In the window that appears, you should see in the bottom right a value for "Current IPs listed in the SFDB" in the order of 100,000-500,000. |
|
|
|
|
Wayne
Groupie
Joined: 29 August 2006 Location: Switzerland Status: Offline Points: 60 |
Post Options
Thanks(0)
Quote Reply
Posted: 03 December 2008 at 4:52am |
|
Roberto
Thank you for your input, the blame was the protection profile at the spamfilter rule of our firewall 
Sorry for that! Kind regards Wayne |
|
|
SF4.5.0.1-beta
|
|
|
|
Topic Options|
Post Reply
|
|
Tweet
|
| Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
This page was generated in 0.270 seconds.