Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Corrupt spam notification emails
  FAQ FAQ  Forum Search   Register Register  Login Login

Corrupt spam notification emails

 Post Reply Post Reply
Author
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Topic: Corrupt spam notification emails
    Posted: 02 April 2007 at 5:04am
Hello all,

I've developed our spam notification emails, so that recipients can accept & whitelist particular emails in their quarantine by clicking the relavent link without having to log into anything.

Anyhow, some of these email notification are being detected as corrupt, either Outlook can't open some of them, or the backup software (Veritas) reports them as corrupt/bad during the backup run.

I thought the problem was due to line length and thought i'd fixed it, but since my latest update there has since been another corrupt email detected.

To take a look at the RAW email as it is stored in the database visit http://mx1.uksubnet.net/spam2/getmsg.asp?msgid=3392812

This pulls the email from the SF database so what you see is exactly what is sent. You'll see that none of the lines in this email are over 76 characters long, so what else could the problem be?



Edited by lyndonje
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 5:31am
Hi

Your structure is:

1. multipart/related
1.1 multipart/alternative
1.1.1 text/plain
1.1.2 text/html
1.1.3 image/gif

I don't know if this is the problem, but this structure would make more sense in my opinion:

1. multipart/alternative
1.1 text/plain
1.2 multipart/related
1.2.1 text/html
1.2.2 image/gif

And with both structures, shouldn't the "multipart/alternative" and the "multipart/related" parts be using unique boundaries?

Cheers

Mike


Edited by mikek
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 6:40am
Hi Mike,

How much experience do you have with MIME emails? To be honest I have none, this is the first MIME email I've created from code.

I only ask because I got my references from
http://mailformat.dan.info/headers/mime.html

Which I think infers this is how the encoding should be - however I understand your logic.

What do you think?

Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 6:48am
I have some experience with MIME E-Mails... (wrote my own webmail client for our webserver)...

Anyway: here's an example I just googled, which explains to pros and cons of the different cascades and confirms my theory that the boundaries must be unique...

http://segate.sunet.se/cgi-bin/wa?A2=ind9903&L=mhtml& ;P=2248
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 7:24am
Hi Mike,

I've followed your link, and it looks like my formatting is the same as in example 9.1 (Multipart/alternative inside Multipart/related).

I can't see where it mentions any con's in using 9.1?

Not saying your wrong - because I don't know either way, I'm just a little confused :)
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 7:35am
Mike, just to clarify, what were you refering to when you mentioned the boundaries needed to be unique?

I know they need to be, but thought mine would be? How do you know if they are/arn't unique?
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 8:14am
Originally posted by lyndonje lyndonje wrote:

Mike, just to clarify, what were you refering to when you mentioned the boundaries needed to be unique?

I know they need to be, but thought mine would be? How do you know if they are/arn't unique?


The "multipart/alternative" and the "multipart/related" parts have to use different boundary strings...
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 8:25am
Below is a snippet from the email in question (found at the URL mentioned in the original post)

Content-Type: multipart/related;
    type="multipart/alternative";
    boundary="----=_NextPart_1175274000_371753209_31011983"

------=_NextPart_1175274000_371753209_31011983
Content-Type: multipart/alternative;
    boundary="----=_NextPart_1175274000_371753210_31011983"

Is this where you are refering to? In my email they are unique unless you are refering to somewhere else?

Still confused....

Thanks.
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 8:34am
of course, you're right... looks like I need a new pair of glasses...
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2007 at 8:54am
No problem. Do you have any other thoughts?
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 03 April 2007 at 3:48am
Here are some more messages that were detected as corrupt in last nights backup:

Message 1
Message 2
Message 3
Message 4
Message 5



Edited by lyndonje
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 03 April 2007 at 4:07pm
Getting an "access denied" when trying to view the emails...
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 04 April 2007 at 3:40am
Hi R,

Are you following the links or copying them? They'll only work it the referer passed is www.logsat.com.

If for some reason this can't work for you let me know.

Thanks,
Lyndon.
Back to Top
__M__ View Drop Down
Groupie
Groupie


Joined: 30 August 2006
Location: Australia
Status: Offline
Points: 75
Post Options Post Options   Thanks (0) Thanks(0)   Quote __M__ Quote  Post ReplyReply Direct Link To This Post Posted: 04 April 2007 at 4:50am
Lyndon, unfortunately I'm not able to be of assistance with diagnosing your problem however I think what you are doing sounds great and I'd like to know a bit more about how your achieving this when you get it all running.

Top work.

Regards, Mike

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 04 April 2007 at 11:19am
Got it. I use SSL when browsing the forums, and thus your referrer check blocked me. Looking at the messages, I'll let you know if we spot anything.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 04 April 2007 at 1:20pm

We are doing something similar but I just created a table in SQL with 3 fields:

 email | auth | date

When I generate a notification email  a link is sent in the email the customer receives like this:

Dear x,

  You currently have 46 messages in the quarantine db. We encourage you to check to make sure valid mail has not been stopped. Please click the link below to be taken to you spam administration menu.

http://spam_web/default.asp?email=joeblow@mydomain.com&a uth=H&Q@!tR

When the user clicks this link the asp program checks the SQL table to see if the username & auth code match and if they do it logs them into the web spam admin area to view quarantines, modify whitelists,etc..

The 7 digit code is just a random generator script I found. Everytime emails are sent out I update the auth code and date so every link the customers receives is unique.

 



Edited by WebGuyz
http://www.webguyz.net
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 05 April 2007 at 3:48am
Yeah, similar. The problem I see with that is its another step the user has to take to find out if anything genuine has been stopped. They may take the time follow the link only to find it was all spam anyway.

The email I send lists the sender address & subject. If there is nothing of interest in the email the user can simply delete/ignore it. The emails listed in the notification will then automatically be deleted after 7 days, and even if they arn't deleted they won't be mentioned in future notifications. In the tblQuarantine table I've added a notified field, and an auth field. The asp generates a random auth code for each message and sets the notified flag to true when a message has been included in a notification. Therefore the SQL statement in the auto_notify.asp page only pulls messages where deliver, expired and notifed are all false.

The link in the email contains the MsgID, Auth Code and Quarantine ID. When the link for a particular email is followed, an ASP page is run that makes sure the MsgID, Auth Code & Quarantine ID all match. If so its sets the deliver flag to true for SF to whitelist and forward. If all three codes dont match, or an email has already been delivered or deleted, an error is displayed indicating the likely cause. Only problem is some of these are corrupt.... and I don't know why!


Edited by lyndonje
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 05 April 2007 at 8:59am

But they would have to take time to read the email and sometimes your just not sure from the subject. Some of our older customers get tons of email everyday.

To make it more manageable I ripped the guts out of a Webmail package for use in the spam admin and now when the customer see the quarantine list they have a small graphic on each line, when they click on the graphic, the actual email appears in a popup window in html, instead of a jumble of text that makes no sense and is hard to read.

 

http://www.webguyz.net
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 11 April 2007 at 7:36am
Anybody have anymore ideas on this? Roberto have you managed to take a look?

Just thinking on... incase the email generated and saved in the database isn't corrupt, and the email is being somehow corrupt during transit, does anybody know of a way to see the full and raw source of an email in an exchange mailbox via outlook and compare that with the data saved in the SF database?
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 11 April 2007 at 6:12pm
lyndon,

I've been fiddling around with your samples for several days now, trying to figure out "why" they were considered corrupted. The one I've been concentrating on is the email in your "Message 1" above. To be honest, I really can't find anything wrong with it... Everything looks as it should.

Yes, looking to see what happens in transit is a good idea. If you're using Outlook 2003, there is finally a way to view the email's original, unmodified source. Look at http://www.outlook-tips.net/howto/view_source.htm for the registry entry to change.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 12 April 2007 at 4:15am
Hi Roberto,

Got my hopes up there! But as is often the case, if you're using Exchange this doesn't work, or so the page says:

Quote This works on mail obtained from Internet mail transports, not Exchange server mailboxes.


Just had another idea to give us an indication on whether they are being corrupt in transit... notifications that have already been sent to the respective recipeints that have been detected as corrupt are still stored in the database. I can change the EmailTo field of previously known corrupt emails to myself, and set deliver to 1 and expire to 0. Let them come through to me. If all are again detected as corrupt, its unlikely to be a problem in transit, however if some arn't detected as corrupt, it could be transit?

What do you think?
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 12 April 2007 at 4:36am
So far it doesnt seem like a transit issue....

I've selected 10 notification emails that have previously been detected by the backup as corrupt. I set their emailto field to myself, and deliver to 1 and expire to 0.

Only 8 of the 10 got into my Inbox. The other two would not sync from the exchange server (I'm using Cached Exchange Mode in OL2003). If I had cahced exchange mode disabled, I presume I would see the emails in my Inbox, but not be able to open them, as this is the symptom some emails have shown on other systems so I'm guessing this is the difference there. I then tried a further two times at having these two particular emails sent through but every time there was a sync issue. These two messages are:
The sync error is
Quote 09:26:34     The following message had an error and synchronization of it was skipped (0x8004011b):


I'll wait to see what tonights backup says about the other 8 emails....


Edited by lyndonje
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 12 April 2007 at 6:33am
Another thing I've just tried... The above two mentioned emails which Outlook won't even display, I've set the EmailTo to a POP3 account and downloaded with Outlook Express - both emails downloaded and displayed fine? Would this indicate a problem with these emails and exchange?
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 13 April 2007 at 3:33am
FYI the 8 other notifications were all reported as corrupt again by the backup, so doesn't look like they're being corrupt in transit...?
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.289 seconds.