Submit Spam Feature |
Post Reply | Page 12> |
Author | |
jerbo128
Senior Member Joined: 06 March 2006 Status: Offline Points: 178 |
Post Options
Thanks(0)
Posted: 24 April 2006 at 12:10pm |
I know that this has been brought up before - I am not trying to beat a dead dog... I have some users that despite how "tight" I make the filter, they continue to receive spam that the filter has missed. I would really like to see a feature where I (as an administrator) can submit spam that was not caught by the server. Right now, our users have been trained to copy email headers and forward messages to spam@domain.com. We then go through those emails and manually blacklist ips, and add keywords to our list. No only is this VERY time consuming, but it is not very effective in the big picture. If anyone else has ideas - I would like to hear from you. Jerbo128 |
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
While wishing for features I'll bring up the dead horse I keep flogging It would be great to have a spamassassin plugin (similar to anti-virus plugin) Currently I'm using Mdaemon (only for Spamassassin, all other filtering disabled) behind my SFI and it catches that last little bit that manages to get by SF and keeps me from having to learn regex I currently have it set up to test for the Autowhitelist flag recently incorporated into SFI and if it exists then it bypasses spamassassin check. Of course I can't teach SF about the spam but I can at least make sure my customers never see it. Would be nice to have it all in one. I think thats the last thing SF is missing in an otherwise fantastic product. |
|
http://www.webguyz.net
|
|
Marco
Senior Member Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137 |
Post Options
Thanks(0)
|
SUppose a user reports spam to spam@domain.com. , and your mailserver would relay mail to this specific adress back to the SPF server, which in turn would recognise the TO adress, and act on it. In effect 'certain adresses' could be considered 'inbound' for the SPF server. This also opens the opportunity for internet users to report spam to your SPF system. If the report is presented to the SPF system in a certain format, so that the SPF engine can use the data within to effectively issue a block on the reported spammer... What do you think Roberto?
|
|
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
This has indeed been brought up before. I'm afraid we're going to "stall" for a bit more and see how the new SFDB filter performs. Submitting spam has challenges, and what Marco mentions as a problem is indeed one... Another alternative is to store the sender's info in a database for some time to allow easy recovery/blacklisting via a web interface by the users. In any case there's several considerations to make as this in not going to be simple for us, for the admins and for the end-users...
Thus the "stalling" on our end ! |
|
Dingo
Newbie Joined: 06 April 2006 Status: Offline Points: 1 |
Post Options
Thanks(0)
|
What tests does the bayesian filter learn from ?
Could i simply forward a spam back through SF that i edit to intentionaly fail a test. If this could be a workaround, what test would be best to cause the bayesian filter to learn it and would not result in my IP etc being banned ? Regards Dave |
|
sgeorge
Senior Member Joined: 23 August 2005 Status: Offline Points: 178 |
Post Options
Thanks(0)
|
A good idea indeed Dave, but I know that Robert has considered that idea before... I remember his response being that even by just forwarding a message to SpamFilter for the Bayesian filter to learn, you add a whole new set of keywords (mostly headers) to the message. The result is that the filter is not as likely to block that message as it really should.
Jerbo, I currently use the same method that you do (in fact, I have a spam@mydomain.com address too). It is time consuming – and honestly, I can only really act on about 20% of the spam that our users submit (If I added a keyword entry for every spam subject I came across, my keyword file would be in MB, not KB..). In any case, for those who are interested in trapping that spam that still gets through, you may want to check out pcmatt’s post on this thread: http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID= 5314&KW=pcmatt Stephen |
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
For that last bit of spam we wrote some tools using some email COM components. Basically we strip out any header info that SF adds and remove the last received from address and feed it through but add a single ^ to the subject line and our keyword filter is looking for that ^ and will reject the mail. The only difference between the original message and the one we resend is ^ in the Subject and of course we have to use our IP address as we can not spoof that. Jury is still out on how this effective it is but if we had an option in SF to have a directory where only known spam was placed I could easily make the message look 100% like the original stripping out anything added by SF and removing out ip from the Recieved line. Lots of work but I really hate spam. |
|
http://www.webguyz.net
|
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
What I would like is to have a seperate DB where all "good" mail is archived to, exactly the same way as the quarantine DB works. Then, when a user gets an email that is spam, they forward it to spam@domain.com . When this mail arrives, we could easily figure which origional email this is, and as we have the origional...we could retrain the bysian based on the origional email.
|
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
forgot to add..... ANY email filtering program that cannot be retrained is not that great. IF a spam gets past the filters, then the filters arn't working properly and they need to be retrained. ?This can only be done when a user gets spam and sends it back to the server to be retrained. Question...Logsat...what do you suggest users do with spam emails that your program failed to detect????
|
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
This topic has been discussed in the past. The method you describe to retrain the bayesian filter is certainly possible. It is to note however that the Bayesian filter stops a very small percentage of spam when compared to the other filters.
We opted to concentrate our resources this past year on developing new filters like the spam-image filter, and the extremely successful SFDB (SpamFilter Distributed Database). These filters block a huge amount spam when compared to the Bayesian filter. We felt that rather "slightly" improve the Bayesian filter, SpamFilter would be more successful if we developed new filtering tecniques. |
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
Logsat, I understand, however teo things spring to mind. Firstly, we are still receiving emails that get through all of the current filters and have no way to teach SF that these are indeed spam emails. Secondly, can you please explain exactly how the SFDB works, as a HUGE customer (Tesco ... the largest supermarket in the UK) seems to have their mail server's IP listed in SFDB, but not in any other DB as in spamcop, spamhause, ordb etc. Also, how can we get this IP delisted??
Cheers |
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
If it's listed it's... probably because they are sending unsolicited emails. If you login the registered area on our website, you'll be able to enter an IP address and perform a lookup in the SFDB to see how many reports there are for that IP, and for what reason it was reported.
Our SFDB is updated in realtime, so the split second they would start an email campaign to their subscriber list, should they emails to unsolicited recipients, they will be blacklisted. If the campaign stops, they will be delisted automatically in either 6 or 24 hours, depending on the amount of spam they had. If it's not listed in other services... well, that may just mean the SFDB is better than others |
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
ok, no problem....exactly where would this registered area login page be??? I can't find it.
|
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
The link is found from the download page. More specifically, it's:
http://www.logsat.com/sfi-login.asp |
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
got it..cheers
|
|
Marco
Senior Member Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137 |
Post Options
Thanks(0)
|
SFDB went straight to the top after it's implementation. It simply is the best performing filter (dare i say it) on the web....
How it works: if a (settable) number of SF users declare a certain mail as beeing spam, ALL SF members reject the mail in question. So it's a joint effort / joint intelect type of filter. ANY spam that is not caught by SFDB filter, is because of 'your own' doing, and SFDB is fed with that.
Edited by Marco |
|
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
I'm sorry, but I don't share you'r feelings...... SFDB blacklisted Tesco, the largest supermarket in the UK.....well donw...NOT !!!!! |
|
Marco
Senior Member Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137 |
Post Options
Thanks(0)
|
refresh and see what i added.. then you'll undersdtand sorry, I didnt describe it correctly, it should be; *my* SF will block the sender since 'N' number of users declared the mail as beeing spam... Apparantly, a number of the SF users think the tesco mails are spam, and your system believes it, since 'n' number of users say so. Edited by Marco |
|
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
Marco, I know how it works, and I think it has at least one fatal flaw in it's design. If 10,000 users sign up for a mailing list, then some time later, 100 of then descide that they no longer want it....invariably they don't bother to unsubscribe, they just mark it as spam. Using this scenario, lazy/stupid/ignorant users can quite easily report an email as being spam, whereas in fact it isn't. The problem here is that if the company in question uses a single mail server IP for both the mailing list and the normal company mail (as Tesco do), then all of the normal email (Tesco use email to place orders with their suppliers, which can be for hundreds of thousands of pounds) will also be blocked. Therefore, the failing here is that SFDB has no confirmation that the email that is being submitted is in fact spam. I would suggest that it is very likely that the number of users declaring mail as spam is far too small. Just my opinion... |
|
Marco
Senior Member Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137 |
Post Options
Thanks(0)
|
As far as i know, your spam/mail system is 'only' one user in the SFDB chain, so, no matter how many of your internal users decalre the mailer as spammer, your system will issue the sending IP ONCE to SFDB. So, if 'n' number of antiispamsystems say it is spam (read: ISP's/businesses), your system will adhere to that, no matter HOW OFTEN those ISP/businesses report the sending IP as spammer, it will only keep the ip alive in the SFDB. you know the fun thing? you can bypass the SFDB alltogether, or increase the number, so that the chance of a falce positive gets smaller. the setting "network reliability' number is the one you need to set to zero, or raise...
|
|
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
Excellent...didn't know what that setting was...cheers. any suggestion on a suitable number...it defaults to 3.....maybe a bit low |
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
Had mine set to 3 but then a Hotmail server started getting rejected by SFDB so I bumped mine up to 6. Seems to be pretty good and haven't seen any more major ISP servers stopped. |
|
http://www.webguyz.net
|
|
Marco
Senior Member Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137 |
Post Options
Thanks(0)
|
yeah, 3 might be a bit too low for your organisation, try 10 and see what happens, if your false positives go away, work your way down by one a day/week untill they return. The lower the number the more 'aggressive' the filter works, the higher the number, the lesser affective it becomes. You need to find out what works best for you by trial and error.
|
|
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
|
Marco
Senior Member Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137 |
Post Options
Thanks(0)
|
p.s. you can allways whitelist this tesco domain, if you think it ok all your internal users can receive their mails.
|
|
Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
Marco,
Thanks for your explanation, you are right on the money. We actually slightly increased the lower limit on the SFDB lookups on our database server itself. This means that our SFDB lookups won't provide a positive match unless there are at least nnn other reports. I'd like to keep the number private so that lurker spammers don't see it. If you wish to know just PM/email us, we'll provide the value. StevenJohns, if Tesco was listed in the SFDB, it's because a rather large number of recipients have complained. And while I have no proof, I'm guessing it's quite larger than 100... Please note that there have to be a certain number of *different* providers/companies running SpamFilter that have received emails from Tesco, not just one company. For example, if 100 users at LogSat.com complained that it was spam, the report would be only one, from logsat.com. However, the 100 users from LogSat.com would report the spam to MAPS RBL services, like sorbs or spamcop. In this case, if these service blacklist Tesco, and multiple SpamFilter administrators use these RBLs, then yes, each SpamFilter installation would send a report. In this case, the SFDB would then contain the same information as the RBL. If users believe this is wrong, or do not wish to use the SFDB for IPs that were blacklisted due to RBLs, the SFDB lookups can be configured to not use RBL (or any other filter you wish). |
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
Roberto, Now I'm confused... An earlier post suggested that the "Network Reliability" number was the minimum number of "spam complaints" that an IP had to have before SF DB would block the IP. Then, (as the default is 3), Webguyz suggested 6, Marco suggested 10 as being a suitable number, but you are suggesting that the tesco IP might have more than 100 "complaints". Please can you have a look at your database and give me some examples of how many "complaints" some IP's typically get. I understand that you may not wish prying spammers to know this, so feel free to PM it to me. This will give me a good idea as to what a realistic number should be.
Cheers |
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
As I mentioned earlier, you can always login the registered user area of the website, and perform detailed lookups on the SFDB database to see more details on what is being blocked and for what reason.
Let me explain the "100 complaints". Consider the following example: SpamFilter is shipped with default RBL servers. One of them is spamcop's bl.spamcop.net. Tesco sends 10,000 emails, and 100 customers complain and report Tesco to spamcop. Spamcop will then blacklist Tesco in its blacklist. Hundreds of SpamFilter administrators are using spamcop, so when they receive an email form Tesco, they will report it as spam to the SFDB. If more than "nn" different SpamFilter's report that IP to the SFDB, then the SFDB will list the IP. Please note that Tesco would have to send emails to users who's provider runs SpamFilter (and is using spamcop), in order for SpamFilter to report the IP. |
|
sgeorge
Senior Member Joined: 23 August 2005 Status: Offline Points: 178 |
Post Options
Thanks(0)
|
Steven, maybe I can try to explain...
The network reliability number is the number of registered SpamFilter installations that have reported one or more blocked emails from a particular email. Let's try a hypothetical example...
Stephen |
|
StevenJohns
Senior Member Joined: 03 August 2006 Status: Offline Points: 119 |
Post Options
Thanks(0)
|
Roberto, got the PM..thanks. sgeorge, thanks for the explanation. I think that I need to whitelist the tesco domain for certain local domains. I know that I can whitelist certain sender email addresses for specific recipients, but can I whitelist *@tesco.com for *@mydomain.com ?? if so, how??
Cheers |
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
... there's actually a bug in SpamFilter that helps in your request. Please see the thread at http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID= 4970.
It will also explain why the bug won't be fixed |
|
Post Reply | Page 12> |
Tweet
|
Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
This page was generated in 0.359 seconds.