SF stripped attachments - bug or feature? |
Post Reply 
|
| Author | |
yapadu 
Senior Member 
Joined: 12 May 2005 Status: Offline Points: 297 |
 Post Options
 Thanks(0)
 Quote Reply
 Topic: SF stripped attachments - bug or feature?Posted: 11 November 2009 at 9:13pm |
|
We had a new client sign up for our spamfilter service.
After about 1 week, they contact our support and say that an invoice (HTML attachment) from a vendor is being stripped off. Not placed into quarantine, but stripped off and the rest of the message delivered. I am not aware of any feature like this, and we have no virus scanner on the server itself other than the one included in spamfilter. We conduct some tests, and don't find any problem to our email addresses. The logs show the messages flowing through our server, and no issues shown in the logs either. I ask the new client to put the MX records back to their old MX records which takes SF out of the equation. Sure enough, after about 36 hours they contact us and say the messages are being delivered with the attachments in tact. How can this be, and how can we track the problem down. We have lost the client now, so probably don't have them available to do testing. They have provided us with one of the sample attachments but we can't reproduce the issue. Anyone else ever have stripped attachments? |
|
 |
|
|
LogSat
Admin Group 
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 12 November 2009 at 11:58pm |
|
yapadu,
SpamFilter will not strip attachments when forwarding emails, and we are not aware of any issues of this ever happening in the past. If you are able to reproduce this we'll definitely look into it of course, as there is always a possibility of a bug causing unexpected behavior. Also, if you can zip and email us the section of SpamFilter's activity logfile around the time the email in question was received, we can scan it for errors.
|
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 17 November 2009 at 7:35am |
|
The customer with the missing attachments came back, and we were able to be included on the list of recipients (at two different domains).
We have so far received three messages from the specific sender with the problem, and sure enough there is no attachment. Is there any way to enable some type of advanced debugging for a specific domain or sender or anything and trap what comes in/out? We can not reproduce the problem ourselves yet, so we can't simulate the problem on demand. |
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 17 November 2009 at 9:03am |
|
I have been going through the logs, looking at the incoming messages from said sender. No errors reported during the processing of the messages.
I do find this to be of interest: (4016) EMail from email@exmple.com to "email1@abc.com" was queued. Size: 11 KB, 11264 bytes So, 11264 bytes in size was processed at this point by spamfilter. Here is the delivery summery to the next server: 1NACf6-0002zK-GZ => postmaster <email1@abc.com> F=<email@example.com> R=virtual_user T=virtual_localdelivery S=1324 Only 1324 bytes received from the server spamfilter passed the message onto. When I pull the message from the mailbox the message size is exactly 1324 bytes. It is starting to look like it is spamfilter who is eating part of the message for some reason. Here is what remains of the message (I have three with virtually the same content) excluding some headers. Subject: ABC INVOICE# 0961756 Content-type: multipart/mixed; boundary="ABC____29737919012111160901" X-Server: LogSat Software SMTP Server X-SF-RX-Return-Path: <ORDERS@ABC.COM> X-SF-HELO-Domain: SM.ABC.COM X-SF-Originating-IP: 48.69.60.107 This message contains an attachment in MIME format, which is not legible to your E-mail application. --ABC____29737919012111160901 Content-type: multipart/alternative; boundary="ABC____29737919012111160902" --ABC____29737919012111160902 --ABC____29737919012111160902 Content-type: text/html; charset=Windows-1252 Edited by yapadu - 17 November 2009 at 9:05am |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 17 November 2009 at 10:42pm |
|
yapadu,
We received the logs, and as you said, this is going to be an interesting issue... The only way we're going to be able to see what is happening is by actually "seeing" what the original email looks like. SpamFilter does have a "Debug" tab to capture SMTP traffic, but it doesn't capture *all* of the email's body, so it's not going to help here. What would help would be to install either WireShark (Etherreal) on the SpamFilter server (or any other packet capture utility), and capture all SMTP traffic from the remote server (I'll send you the IP common to the 2 emails in the logs via email). If you'd like, you could also provide our address to the vendor to copy us as well on the emails. We can then run the packet capture on our end and try to get to the bottom of this. Please let us know in advance if you'd like for us to run the trace so we can setup up our end before they send the emails.
|
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 25 November 2009 at 8:52pm |
|
Roberto,
I have sent you two text files that were compiled from wireshark, the first shows the incoming conversation between servers of the delivery. The second shows the delivery of the message to the SMTP server. You can clearly see that the attachment was received when delivered to spamfilter, and when spamfilter delivers it the attachment portion of the message is gone. |
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 25 November 2009 at 9:05pm |
|
I see the problem, I see the problem!
I am so excited! Here is a snippet of the incoming message. --ABC____30689919010211250902 Content-type: text/html; charset=Windows-1252 . --ABC____30689919010211250902-- --ABC____30689919010211250901 Here is the last part of the sending communication. --ABC____30689919010211250902 Content-type: text/html; charset=Windows-1252 . 250 OK id=1NDSsR-0007cd-78 QUIT The message is cut off, as the body of the message when incoming contains a single period in between <CRLF> so spamfilter assumes that is the end of the message when incoming. In actual fact the sending server still sends additional data. That's gotta be it... Edited by yapadu - 25 November 2009 at 9:05pm |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 28 November 2009 at 1:27am |
|
We received the files, and were finally able to pinpoint the issue thanks to your wireshark capture. Please see the first screenshot showing the incoming file to SpamFilter. What you see is not a period between the <CR><LF> separators. Wireshark shows that it is actually a NULL character (byte value of 0). This is a control character that, per RFC, should not be used in the mail body.  SpamFilter does receive the email successfully, and however this is where the 2nd part of the problem begins (the first part is the remote server sending an invalid NULL character....). Please now see the second screenshot showing the outgoing email from SpamFilter to your destination SMTP server. When SpamFilter forwards the email, it tries to "fix" the invalid NULL character and replaces it, of all things, with the "0x2e" character, which happens to be the period. Since this period, (un)luck has it, is surrounded by the <CR><LF> sequence, this just happens to indicate the end-of-email sequence and thus your destination SMTP server obviously stops receiving the email.  This has got to be one of the most unlucky sequence of events we've seen... I'll try to see if there's anything we can do to either prevent SpamFilter form autofixing the invalid NULL characters, or to at least replace them with a different, non-dot, character...!
|
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 29 November 2009 at 7:49pm |
|
It was suspected from the start this one was going to be difficult to isolate. It certainly took some doing but I'm glad the problem has at least been identified and hope you are able to come up with a solution for this issue.
|
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 December 2009 at 10:34pm |
|
yapadu,
We've finally *just* been able to reproduce this. Thank you a LOT for the patience and the help. I can confirm everything is a suspected. SpamFilter is replacing the NULL character with the ".", and its placement just happened to be between the <CR><LF>.<CR><LF> sequence causing the email to be terminated. While we come up with a permanent fix, the good news is that there is actually a very easy workaround that so far seems to avoid the problem. In the SpamFilter.ini file, simply change the following setting to "1" (there is not even the need to restart SpamFilter after the change...) SpoolQueueFilesToMemory=1 This forces SpamFilter to spool the incoming emails to disk rather than handling them in RAM. The spooling process actually causes SpamFilter to load the message a bit differently when forwarding it, avoiding the NULL character to be replaced. |
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 December 2009 at 11:28pm |
|
Hi Roberto,
Hate to make things more complex, but check out this snippet from my ini file: ;Set this to 0 to prevent queued emails to be spooled to memory, and force spooling to disk. While less efficient, spooling to disk helps allow existing antivirus software to detect and block some infected email files SpoolQueueFilesToMemory=1 The server is already spooling to disk it seems. :-( |
|
|
|
|
yapadu
Senior Member
Joined: 12 May 2005 Status: Offline Points: 297 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 December 2009 at 11:29pm |
|
Hmmm...
After reading the setting in the INI file, I think you mean to set it to 0 right? The ini comment says 0 will spool to disk. Since it is already 1 and the bug is happening you must have meant 0... I'll try that. |
|
|
|
|
LogSat
Admin Group
Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
Quote Reply
Posted: 02 December 2009 at 11:33pm |
|
Sorry... bug in the posting! I copied/pasted the setting that *breaks* the email, so yes, the correct one to use that fixes the issue is:
SpoolQueueFilesToMemory=0 ..and a side-note, we already have a fix ready. TWO single lines of code were causing the problem. We'll be testing this for a few days and will make the new build available to everyone as soon as we're sure there's no side-effects.
|
|
|
|
Topic Options|
Post Reply
|
|
Tweet
|
| Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
This page was generated in 0.338 seconds.