Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Maybe exclude aol.com and other IP’s from
  FAQ FAQ  Forum Search   Register Register  Login Login

Maybe exclude aol.com and other IP’s from

 Post Reply Post Reply
Author
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Topic: Maybe exclude aol.com and other IP’s from
    Posted: 20 May 2006 at 10:31am

Hi,

  I think its not practical to block aol.com, hotmail.com, gmail.com valid IP blocks in the SFDB bacause of the large volume of mail and the potential of anyone of them being blacklisted and added to the SFDB causes more issues than its worth.

Anyone else have an opinion?

http://www.webguyz.net
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 20 May 2006 at 4:38pm

webguy,

I had previously requested this in a direct message to LogSat:


"Can we come up with a way for SF Users to remove blocks in the SFDB from an IP other that the mail server IP?  I managed to get a hotmail server on the list with a mistake and want to clear it.  I can see this happening semi-often.

Also,  How about a SFDB whitelist such as aol has?"
 
I think something should be looked at but I am not sure what.  As an ISP, we went through a lot to get whitelisted by aol but have to  deal with nearly hourly reports from them.  Hotmail ... well they won't play so  I am not real happy with them anyway.  Other thoughts?
 
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
Roman View Drop Down
Newbie
Newbie


Joined: 04 November 2005
Location: Russian Federation
Status: Offline
Points: 32
Post Options Post Options   Thanks (0) Thanks(0)   Quote Roman Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 11:51am
my 2 cents...

May be it could be reasonable to (optionally?) skip SFDB checks for IPs which PASS SPF test?
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 12:46pm

Hmmmm   I suspect that that would drastically reduce the effectiveness of SFDB.  Need to think more on that.  Besides,  With AOL, you need now to define "pass"  as their own SPF record is loose:

TXT Record 1:   spf2.0/pra ip4:152.163.225.0/24 ip4:205.188.139.0/24 ip4:205.188.144.0/24 ip4:205.188.156.0/23 ip4:205.188.159.0/24 ip4:64.12.136.0/23 ip4:64.12.138.0/24 ptr:mx.aol.com ?all

TXT Record 2:   v=spf1 ip4:152.163.225.0/24 ip4:205.188.139.0/24 ip4:205.188.144.0/24 ip4:205.188.156.0/23 ip4:205.188.159.0/24 ip4:64.12.136.0/23 ip4:64.12.138.0/24 ptr:mx.aol.com ?all

Here are some general thoughts ...
1)  Get the RFC for mail servers out of the stone age
2)  Get the "Open Source" servers to correct their reject codes (4xx VS 5xx) or stop receiving mail from them (LISTEN UP POSTFIX USERS)
3)  Get AOL, Hotmail (msn), Yahoo & gmail to agree on something ... ANYTHING!
4)  Get ISP's to start charging for email delivery FROM domains that hammer them.  The US Post Office makes $$$ on bulk mail.  Shouldn't we?
5)  Charge OUR customers for bulk mailing and verify their opt in / opt out methods.  We do and it works well and we make $$ and the world doesn't get hammered with obfuscated, BS or forged crap from our IP's.
6)  Really crack down (somehow) on anything that is obfuscated
7)  Get valid newsletter providers to "Do it right"
8)  Hang or shoot anybody that doesn't adhere to all of the above.

The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
sgeorge View Drop Down
Senior Member
Senior Member


Joined: 23 August 2005
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote sgeorge Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 1:23pm
Dan, on that note of uselessly loose SPF records, I was just thinking today that it would be great to be able to override a domain's useless SPF rules.  Some may have they're reasons for not getting their acts together yet, but with all of the PayPal scams out there these days (for one example), I'm amazed that they still have not been able to publish a -all SPF record to date for themselves.

Stephen
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 2:46pm

Stephen,

Or worse ... I see a lot of "allow everything" records.

The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 6:48pm
The SFDB has the capability of "whitelisting" IP addresses. The process on how this is to be done is yet to be defined. We have thought about allowing licensed users whitelist IPs from the registered user area, but that could lead users whitelisting spammer's IP (either on purpose or by mistake...).
Allowing the owenrs of the blacklisted IPs to whitelist themselves is also a bad idea...

We are open to suggestions!
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 7:47pm

I believe that the major players post their mailserver blocks somewhere on their websites. I think that only regional ISP's should qualify to have thier mail server IP blocks whitelisted. Maybe a list of regionals should be drawn up and people could volunteer to find a source of info for one or more of the ISP's mail server and forward the URL to SF support and they could verify it and add them. That way we could break up the workload load and feed the info to one source who has the power to whitelist without that person having to do all the legwork. A vote for some of the regionals are:

aol.com
msn.com
yahoo.com
gmail.com
earthlink.com
mindspring.com
netscape.net
netzero.com

http://www.webguyz.net
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 8:42pm
Webguyz,
 
What you are saying is you "Trust" the "big boys".  Well, I don't.  If we are to take your tact, I would weight it against their SPF record (mentioned earlier but someone).
 
aol.com  Loose SPF with a ?all
msn.com  Also loose with hotmail includes and a ~all
yahoo.com  No SPF at all
gmail.com  Your kidding right?   Redirect to google.com with a ptr ?all  I can spoof that is a heart beat
earthlink.com No SPF at all
mindspring.com  No SPF at all
netscape.net  No SPF at all
netzero.com  Loose SPF with a ?all
 
What does this say about their commitment to anti-spam?  Of the above list, aol is the ONLY one we have had any real success communicating with about abuse/spam either to or from our network.  They have worked with us very closely as we do have customers on our network doing huge mailings.  Also, we went to Chicago for a "Email deliverability Boot Camp" and aol seemed to have the brightest people there and had the most reasonably implementable (is that a word?) ideas. msn/hotmail simply wanted to make money guaranteeing mail deliverability (big surprise there)  and Yahoo .... let's just say that their name describes my opinion of their representative.
 
Aren't I just a cranky, cynical SOB? 
 
SO ... my thought ... A list of "Trusted" Mail servers should only include servers with SPF records and should be SEMI-Whitelisted.  Meaning, do not add to the SFDB until "X" number of submissions come in and expire them faster ... say 1-4 hours,  Kinda like aol does their black-listing.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 9:11pm

I didn't say anything about SPF. I'm saying get the actual IP address of their outgoing mail servers.

 

http://www.webguyz.net
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 23 May 2006 at 9:26pm

Webguys,

I understand that BUT, I do not want to simply trust their mail servers.  Inclusion of an SPF record to a domain shows at least an attempt to join the anti-spam community so I am using SPF as a starting point to start trusting them.  If a mail provider makes no attempt to prevent their addresses from being spoofed, how can I believe they will stop their users from spamming?  And if they do not communicate easily to another ISP concerning spam issues ... same problem.  That was all my point was.

The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
Roman View Drop Down
Newbie
Newbie


Joined: 04 November 2005
Location: Russian Federation
Status: Offline
Points: 32
Post Options Post Options   Thanks (0) Thanks(0)   Quote Roman Quote  Post ReplyReply Direct Link To This Post Posted: 24 May 2006 at 10:10am
ok, let me explain my thoughts about SPF and the topic.

First of all, the easiest (and may be the only right) way to obtain foriegn ISP's mailservers IPs is (or should be) SPF. You may only wish that you know all their MS IPs, but for large organizations this list is very big and dynamicly changing. So (uless you use SPF) you would have to constantly _manually_ monitor them. With clear SPF _pass_ answer you must be sure that certain IP is allowed to mail certain DomainFROM.

Therefore, WebGuyz, I'm not sure that we could "get the actual IP address of their outgoing mail servers" if they are lazy enough to not support even correct SPF record.

Otherwise, Dan is right about all the problems. My original idea was to (optionally) reward "-all" records, but rules "ip4:0.0.0.0/0" could ruin this too...

Back to Top
Roman View Drop Down
Newbie
Newbie


Joined: 04 November 2005
Location: Russian Federation
Status: Offline
Points: 32
Post Options Post Options   Thanks (0) Thanks(0)   Quote Roman Quote  Post ReplyReply Direct Link To This Post Posted: 24 May 2006 at 11:00am
and some offtopic thoughts about SPF.

The situation with major IPSs went absolutely crazy. The fact is that (almost) all of them use "?all" (or "~all") records. So de facto I should use "softfail" and "neutral" responses to mark spam. I know thats not good, but that is the only way stop junk from google and hotmail (the 1st and the 3rd in my spam domains top).

The 2nd place for spam is yahoo... I don't know what to say. May be DomainKey is absolutely perfect, but why the hell not to use SPF? I can only remember russian KGB joke: "if you want to sabotage dissident organization - lead it".
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 24 May 2006 at 11:23am

Roman,

Your comment "So de facto I should use "softfail" and "neutral" responses to mark spam. I know thats not good, but ...."  Is a problem that I flip/flop on.  The SPF proposal states that you should allow these but can use the results to "score" Spam.  I do allow them but get hammered as a result.  I really wish that ISP's (such as ourselves) would take responsibility for their mail server IP's.  Is AOL going to try to convince me that they don't know for sure what IP's they are using?  Or ... is it a hedge ... I mean, they are one of the bigest pushers of SPF and their own record is ?all  Something wrong here.

The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
sgeorge View Drop Down
Senior Member
Senior Member


Joined: 23 August 2005
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote sgeorge Quote  Post ReplyReply Direct Link To This Post Posted: 24 May 2006 at 12:08pm
In the spirit of derailing this topic train further off course... 

I would disagree with the policy of universally blocking softfail and/or neutral spf results.  It may give you great results in blocking spam, but it punishes anyone who happens to have a legitimate reason for creating softfail and neutral spf rules.

What's this?  Could there actually be a practical reason for "softfail" and "neutral".  I would say that there is...  In fact, I do use it - to the least extent that I have to.  My domain's spf 1 record looks something like this:
v=spf1 +mx:mail.mydomain.com ?mail.sharedhostingcompany.com -all


...The story is that email that legitimately can be from "...@mydomain.com" comes from two sources... 1.) my private mail server, or 2.) my web site - which is on a shared web host.

This shared hosting company is completely inept when it comes to tracking down customers of theirs that send out spam.  And yhe last thing I need is for one of these spamming customers to get smart enough to fake a peer customer's email address - knowing that the peer's spf record is likely to have a "pass" rule for all email originating from the shared hosting company.  Effectively, my spf record would assert that spam alleged to be from @mydomain.com really is.

So, yes, I find that making a neutral rule for my shared web host helps me distance myself from peer spammers, while preventing the outgoing email from my web site by getting kicked by my -all.

My thought is is this...  the more we push our spf verification criteria past design, the less likely we are to get the big boys to play nice.  True, they will get the complaints from their customers when mail isn't delivered.  But what do you really think their response is going to be?  Make a -all record?  I bet that a +all is more likely...

Stephen



Edited by sgeorge
Back to Top
WebGuyz View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 24 May 2006 at 5:50pm

SPF is not a good choice because of the reasons listed above. But why not list all the IP blocks owned by aol.com, yahoo.com, etc.

These can be found by checking my logs for some current IP addresses and verifying them in a tool like Sam Spade and where I can get a list of the IP blocks owned by the ISP.

Don't lose site of the fact that we are not whitelisting them by adding the ip blocks to SFDB, we are just relying on all the other SF tests as we have in the past to avoid false positives. My stats show that the highest stopping filter on my server is SFDB and thats great, but I don't want someones filter updating SFDB with an IP causing me to stop being able to receive from the largest ISP's. This has only happened once, but I can see it happening again.

SF is tuned over time to filter spam for your users, Adding SFDB to the mix is good but some of that filter uniqueness (is that a word?) is lost and by blacklisting some of the big guys  I am at the mercy of someones unique spam fighting tactics.

Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 24 May 2006 at 7:00pm
OK ... I am going to take a different tact here.  WHY are the "Big Boys" getting on the SFDB in the first place? As Webguyz stated (And yes that is a word!), "I am at the mercy of someones unique spam fighting tactics."  I see that at the most accurate reason for the blacklisting of the above mentioned "Big Boys".  HOWEVER,  I use the SFDB feature of only accepting SFDB uploads that I feel are the least subjective  to,  for the most part, solve that issue.  Consider the following and note that this is MY opinion and MY method only:
 
SFDB Checks for IPs blocked by selected filters:
  1. Domain is in local blacklist file - NO  Someone may not want mail from NetZero
  2. EmailFrom is in local blacklist file - YES  Since this is a full email address, somewhat safer that 1 above
  3. Reverse DNS not found - YES I set this test so I should accept others
  4. Empty Main From - NO Non RFC and we get valid messages with empty FROMs
  5. Mail From and Mail To are equal - NO equal To & From common in our application
  6. Mail From and Mail To Domain equal - NO  Same as above
  7. Exceeded max number of RCPT TO - YES I find this to be a real good filter
  8. IP address from a blacklisted country - NO we work with loads of off shore folks
  9. Email To is in local blacklist file - YES This should be fairly safe
  10. EmailTO in not in AuthorizedTOEmail list - YES This has very few false triggers
  11. No relay allowed or % in FROM - YES  Obviously
  12. IP Found in MAPS search - NO If it is in a MAP I use, it will block anyway.  I do not want to be at the mercy of another SF User using a ridiculously aggressive or inaccurate MAPS list
  13. Keywords found in content - NO  Some people WANT or even NEED to enlarge their body parts!
  14. Statistical filter match - NO I do not want to trust other users statistics as I do not know what kind of business they are handling
  15. SPF - YES For now, I am trusting this one
  16. Invalid sender domain MX record - YES I personally like having a valid return path
  17. Virus found in email - NO  Usually a victim not a culprit
  18. URL in email found in SURBL search - YES  I like the statistics of this filter
  19. Detected spam signature in embedded image - YES  But the Jury is still out on this.
  20. SFDB - NO but I do not think this is tested anyway.
Thoughts and opinions always welcome.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
sgeorge View Drop Down
Senior Member
Senior Member


Joined: 23 August 2005
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote sgeorge Quote  Post ReplyReply Direct Link To This Post Posted: 25 May 2006 at 1:10pm
Originally posted by Desperado Desperado wrote:

13. Keywords found in content - NO  Some people WANT or even NEED to enlarge their body parts!


...Interesting, but WAY more than I needed to know about you. 

Stephen
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 25 May 2006 at 1:19pm

Stephen,

Roger that!

The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
Roman View Drop Down
Newbie
Newbie


Joined: 04 November 2005
Location: Russian Federation
Status: Offline
Points: 32
Post Options Post Options   Thanks (0) Thanks(0)   Quote Roman Quote  Post ReplyReply Direct Link To This Post Posted: 31 May 2006 at 11:57am
Stephen, there is no "legitimate reason for creating softfail and neutral spf rules" (in long term perspective)!

First of all (if I remember correctly) origionally "?" is for "testing" and "~" is for domains "where not all clients are upgraded to use correct smtp relays yet". But "smart" guys decided: "I want to participate in antispam and I want 100% deliverability, but I am very lazy... What can I do? Oh! There goes "~/?" !". This sick method is even allowed by http://www.openspf.org/whitepaper.pdf . So we have the illusion that big ISPs support antispam policy, but de facto it is ABSOLUTELY USELESS (if follow original standards). Well, if the biggest ISPs use this sick method I'm constrained to block "?/~". Anyway, what should "a:some ?all" mean? As I can see it mean "host "some" is ok to send mail, and I am in testing phase for all other IPs in the world". When are you gonna finish your endless global testing??? Sorry, this is defenetly not right.

About your particular case. Spam became the huge problem because of insecure smtp standard. But now with current "best practicies" (where the hell new RFC is ?!?) it is a problem only besause man's laziness and incompetence - spammers can only relay their crap through WRONGLY configured servers. So administration task became very clear - do everything RIGHT and block everything WRONG. By assuming that your hosting company could fail in RIGHT authentication of customers and therefore using "?" you actually say: "my hoster (presumably) has WRONG configuration in his servers, so I should also implement WRONG configuration on my side". Then please don't be disappointed when I block you.

If you use anyone's relay, please, make it "pass", make it RIGHT. Yes, if he would fail in authentication (or anything else) - he will be blocked because he did something WRONG. And then he will have a chance to do it RIGHT.
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 31 May 2006 at 12:18pm
Roman,
 
That was a little reminiscent of "Who's on first?"
 
In principal,  I really do agree with your statements and I get very irritated at servers and DNS that are configured wrong or sloppily (Wow - Spell check says that's a word).  However, sometimes, it is not laziness but extreme pressure from our remote customers who don't give a rat's you know what about "right" or "wrong" and just want their mail to go through.  Also, some of our own sales force want their Sprint PCS to work and their IP is dynamic and even crosses the /24 barrier so what do we do there?  I have changed my SPF record over and over again to make up for that.  We tried to get the stupid "Smart Phones" to use smpt auth and it fails about 30% of the time (??).  I do not have a good answer but I do have a -all in my SPF.  I just have to live with the problems it causes if I am claiming to be an anti-Spam crusader.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
Roman View Drop Down
Newbie
Newbie


Joined: 04 November 2005
Location: Russian Federation
Status: Offline
Points: 32
Post Options Post Options   Thanks (0) Thanks(0)   Quote Roman Quote  Post ReplyReply Direct Link To This Post Posted: 31 May 2006 at 12:44pm
Dan, about your SFDB usage policy (as long as opinions are welcomed :) . Consider this:

The main idea of SFDB is to mark WRONG relays. Let's look at 1st point. People usually (I mean IMHO. Myself.) block some domain, let's say bonbon.net not because they are/were subscribed to their mailing lists, but because spam/viruses fakes the return addr.. In other words I block some domain only if
1. SpamHits number is more than <some predefined number>
2. The possibility of real business with it's owners is zero (and I have never seen their legitimate mail ever)
3. It has no SPF record (actually my custom error for this rule sounds like: "550 Your domain %Domain% does not have SPF record and/or is Blacklisted...")
So when I mark some server for "some@bonbon.net" there is no chance that I'd mark the REAL bonbon server - it always will be some stupid ADSL user. So it's pretty safe to use (not much or less than others).

The same applies to 5, 6 (in my config - never => I use this rule => only spammers will be marked).
May be true also for 13 and 14 (not 100% sure, needs research).
17 - always bad|stupid boy - must be marked.
20 - well, very strange for me to see this rule at all - "autoinflate" possible.
3, 15, 16 you can do it yourself as 12 - see no reason.

Edited by Roman
Back to Top
Roman View Drop Down
Newbie
Newbie


Joined: 04 November 2005
Location: Russian Federation
Status: Offline
Points: 32
Post Options Post Options   Thanks (0) Thanks(0)   Quote Roman Quote  Post ReplyReply Direct Link To This Post Posted: 31 May 2006 at 12:54pm
"but extreme pressure from our remote customers who don't give a rat's" - that's another point: WE NEED RFC.

Well, I can only suggest to move obsolete (good adj. for SmartPhones :) equipment to subdomain with wide or no SPF...

Anyway, as long as "big guys" use "?~all" I see no difference between "~?-all" at all.

Edited by Roman
Back to Top
sgeorge View Drop Down
Senior Member
Senior Member


Joined: 23 August 2005
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote sgeorge Quote  Post ReplyReply Direct Link To This Post Posted: 31 May 2006 at 1:46pm
Roman, I understand that you can have your own unique interpretation of SPF results in order to curb spam, but your interpretation doesn't match up to what's in the spec.  True, neutral and softfail results are to be used in the testing phase, but that is not their only usage.

From http://new.openspf.org/svn/project/specs/rfc4408.html#op-res ult:
2.5.2.  Neutral

The domain owner has explicitly stated that he cannot or does not want to assert whether or not the IP address is authorized. A "Neutral" result MUST be treated exactly like the "None" result; the distinction exists only for informational purposes. Treating "Neutral" more harshly than "None" would discourage domain owners from testing the use of SPF records (see Section 9.1 (Sending Domains)).

...

2.5.5.  SoftFail

A "SoftFail" result should be treated as somewhere between a "Fail" and a "Neutral". The domain believes the host is not authorized but is not willing to make that strong of a statement. Receiving software SHOULD NOT reject the message based solely on this result, but MAY subject the message to closer scrutiny than normal.

The domain owner wants to discourage the use of this host and thus desires limited feedback when a "SoftFail" result occurs. For example, the recipient's Mail User Agent (MUA) could highlight the "SoftFail" status, or the receiving MTA could give the sender a message using a technique called "greylisting" whereby the MTA can issue an SMTP reply code of 451 (4.3.0 DSN code) with a note the first time the message is received, but accept it the second time.

...Neutrals aren't just used by lazy admins.  I think that my case - where I used a shared hosting company that has both good and bad customers is a perfect example of where a customer such as myself would hesitate to stamp certain emails as a "pass", when same smtp server sends unsolicited email for abusive customers.

I've tested my web hosting company's smtp server, and confirmed that it is not an open relay.  By getting on AOL's FBL (Feedback loop), I was able to receive a copy of all spam sent out through my hosting company.  The mail headers helped me identify that the outgoing spam is coming from actual customers from the web hosting company, who are using their web sites to send out spam.

Notice that I do use a -all in my SPF record, it is only email originating from the shared web hosting company that I use that I mark as neutral.
v=spf1 +mx:mail.mydomain.com ?mail.sharedhostingcompany.com -all


If I were to avoid any neutral or softfail results, as you suggest, then I would only have the following options - none of which are acceptable to me:
SPF Record                                                    Consequence
+mail.sharedhostingcompany.com -all   Spammers on my hosting company's
                                       site can send mail on my domain's
                                       behalf. My SPF record asserts that
                                       the email is from me.
-mail.sharedhostingcompany.com -all   All email from my web hosting
                                       company's web site (including the
                                       email that I send) is marked as
                                       forgery.
(Deleting my SPF record)        I lose all benefits from SPF
                                 validation, in efforts to make sure
                                 that you can receive my mail.

I do agree with you - that neutrals and softfails are widely overused by lazy administrators.  Unfortunately, the Big Boys (Hotmail, Google, etc..) are some of the worst offenders.  But as you consider keeping your SPF acceptance policy more aggressive, I hope you consider that my isolated use for neutral results from one of my outgoing email servers is appropriate.

Stephen

Edited by sgeorge
Back to Top
Roman View Drop Down
Newbie
Newbie


Joined: 04 November 2005
Location: Russian Federation
Status: Offline
Points: 32
Post Options Post Options   Thanks (0) Thanks(0)   Quote Roman Quote  Post ReplyReply Direct Link To This Post Posted: 31 May 2006 at 3:01pm
Stephen, as you wrote "neutral ... stated that [domain owner] cannot or does not want to assert whether or not the IP address is authorized". If you explicitly use some relay for your mail (I think) you should authorize that IP for this task. If that relay will fail to separate you from spamming (and faking your authorization) customer, well, may be it is not a very good relay. I’m afraid his IP (sooner or later) will be blacklisted if not for wrong SPF but for something else.

My point is: if you authorize some server to relay your mail then you should do it explicitly. You should not inherit foreign errors in your own configuration.

I know, acc. to original meaning of SPF you are (maybe) right. But current practice of big ISPs makes it wrong (ok, only for me maybe).

Well, I have a suggestion for Roberto which could help us all:

To make an option to treat “~all”, “?all” (and maybe “+all”) as “-all”, but process explicit “neutral/softfail” responses as it does now.

In other words I’d restrict all SPF statements which says “pass|neutral the world”.
Back to Top
sgeorge View Drop Down
Senior Member
Senior Member


Joined: 23 August 2005
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote sgeorge Quote  Post ReplyReply Direct Link To This Post Posted: 01 June 2006 at 11:38am
Roman, along the idea of prospective enhancements in SpamFilter’s SPF filtering options, wouldn’t it be nice to be able to explicitly override an external domain’s SPF record with one of your own making?

This would allow finer-grain control to tightening lazy SPF records.  We could make better use of the lazy SPF records from the Big Guys, without rejecting any and all responses of neutral and softfails.

If one could copy and paste Gmail’s existing SPF record into SpamFilter, changing the useless ?all to a –all, for example, one could stand to stop a lot of spam.

I suppose though, that the right rejection code would have to be used.  Returning an error due to the “Sender’s” policy would not be accurate.

Stephen

Back to Top
vrspock View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote vrspock Quote  Post ReplyReply Direct Link To This Post Posted: 15 June 2006 at 4:59pm
I like the idea of having an SPF override to force a -all on some of these loose records.  Not sure though what sort of worms that would let out of the can with regards to when they add/remove mail servers from their pool.
Back to Top
MartinC View Drop Down
Newbie
Newbie


Joined: 29 July 2005
Status: Offline
Points: 25
Post Options Post Options   Thanks (0) Thanks(0)   Quote MartinC Quote  Post ReplyReply Direct Link To This Post Posted: 05 September 2006 at 10:12am

interesting thread.

we switched to having soft fails blocked due to the amount coming from hotmail that got through on this check.

it may not be perfect but we had 100s getting through of hard to block spam... with the config change, all are blocked.

it would be nice if the soft fail option was configurable per domain... or something we could override so we can get the benefits of both options and a bit more granularity.

Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.486 seconds.