Submit Spam Feature
Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: https://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5588
Printed Date: 26 December 2024 at 7:31pm
Topic: Submit Spam Feature
Posted By: jerbo128
Subject: Submit Spam Feature
Date Posted: 24 April 2006 at 12:10pm
I know that this has been brought up before - I am not trying to beat a dead dog...
I have some users that despite how "tight" I make the filter, they continue to receive spam that the filter has missed.
I would really like to see a feature where I (as an administrator) can submit spam that was not caught by the server. Right now, our users have been trained to copy email headers and forward messages to mailto:spam@domain.com - spam@domain.com . We then go through those emails and manually blacklist ips, and add keywords to our list. No only is this VERY time consuming, but it is not very effective in the big picture.
If anyone else has ideas - I would like to hear from you.
Jerbo128
|
Replies:
Posted By: WebGuyz
Date Posted: 24 April 2006 at 3:04pm
While wishing for features I'll bring up the dead horse I keep flogging
It would be great to have a spamassassin plugin (similar to anti-virus plugin) Currently I'm using Mdaemon (only for Spamassassin, all other filtering disabled) behind my SFI and it catches that last little bit that manages to get by SF and keeps me from having to learn regex
I currently have it set up to test for the Autowhitelist flag recently incorporated into SFI and if it exists then it bypasses spamassassin check. Of course I can't teach SF about the spam but I can at least make sure my customers never see it.
Would be nice to have it all in one. I think thats the last thing SF is missing in an otherwise fantastic product.
------------- http://www.webguyz.net
|
Posted By: Marco
Date Posted: 25 April 2006 at 9:41am
SUppose a user reports spam to mailto:spam@domain.com - spam@domain.com . , and your mailserver would relay mail to this specific adress back to the SPF server, which in turn would recognise the TO adress, and act on it.
In effect 'certain adresses' could be considered 'inbound' for the SPF server. This also opens the opportunity for internet users to report spam to your SPF system.
If the report is presented to the SPF system in a certain format, so that the SPF engine can use the data within to effectively issue a block on the reported spammer...
What do you think Roberto?
------------- Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
Posted By: LogSat
Date Posted: 25 April 2006 at 10:56pm
This has indeed been brought up before. I'm afraid we're going to "stall" for a bit more and see how the new SFDB filter performs. Submitting spam has challenges, and what Marco mentions as a problem is indeed one... Another alternative is to store the sender's info in a database for some time to allow easy recovery/blacklisting via a web interface by the users. In any case there's several considerations to make as this in not going to be simple for us, for the admins and for the end-users...
Thus the "stalling" on our end !
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: Dingo
Date Posted: 17 May 2006 at 6:45am
What tests does the bayesian filter learn from ?
Could i simply forward a spam back through SF that i edit to intentionaly fail a test. If this could be a workaround, what test would be best to cause the bayesian filter to learn it and would not result in my IP etc being banned ?
Regards
Dave
|
Posted By: sgeorge
Date Posted: 17 May 2006 at 10:35am
A good idea indeed Dave, but I know that Robert has considered that idea before... I remember his response being that even by just forwarding a message to SpamFilter for the Bayesian filter to learn, you add a whole new set of keywords (mostly headers) to the message. The result is that the filter is not as likely to block that message as it really should.
Jerbo, I currently use the same method that you do (in fact, I have a spam@mydomain.com address too). It is time consuming – and honestly, I can only really act on about 20% of the spam that our users submit (If I added a keyword entry for every spam subject I came across, my keyword file would be in MB, not KB..).
In any case, for those who are interested in trapping that spam that still gets through, you may want to check out pcmatt’s post on this thread: http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5314&KW=pcmatt - http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID= 5314&KW=pcmatt
Stephen
|
Posted By: WebGuyz
Date Posted: 17 May 2006 at 10:46am
For that last bit of spam we wrote some tools using some email COM components. Basically we strip out any header info that SF adds and remove the last received from address and feed it through but add a single ^ to the subject line and our keyword filter is looking for that ^ and will reject the mail. The only difference between the original message and the one we resend is ^ in the Subject and of course we have to use our IP address as we can not spoof that. Jury is still out on how this effective it is but if we had an option in SF to have a directory where only known spam was placed I could easily make the message look 100% like the original stripping out anything added by SF and removing out ip from the Recieved line.
Lots of work but I really hate spam.
------------- http://www.webguyz.net
|
Posted By: StevenJohns
Date Posted: 10 August 2006 at 3:56pm
What I would like is to have a seperate DB where all "good" mail is archived to, exactly the same way as the quarantine DB works. Then, when a user gets an email that is spam, they forward it to mailto:spam@domain.com - spam@domain.com . When this mail arrives, we could easily figure which origional email this is, and as we have the origional...we could retrain the bysian based on the origional email.
|
Posted By: StevenJohns
Date Posted: 10 August 2006 at 3:59pm
forgot to add.....
ANY email filtering program that cannot be retrained is not that great.
IF a spam gets past the filters, then the filters arn't working properly and they need to be retrained. ?This can only be done when a user gets spam and sends it back to the server to be retrained.
Question...Logsat...what do you suggest users do with spam emails that your program failed to detect????
|
Posted By: LogSat
Date Posted: 10 August 2006 at 11:18pm
This topic has been discussed in the past. The method you describe to retrain the bayesian filter is certainly possible. It is to note however that the Bayesian filter stops a very small percentage of spam when compared to the other filters. We opted to concentrate our resources this past year on developing new filters like the spam-image filter, and the extremely successful SFDB (SpamFilter Distributed Database). These filters block a huge amount spam when compared to the Bayesian filter. We felt that rather "slightly" improve the Bayesian filter, SpamFilter would be more successful if we developed new filtering tecniques.
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: StevenJohns
Date Posted: 11 August 2006 at 4:20am
Logsat,
I understand, however teo things spring to mind. Firstly, we are still receiving emails that get through all of the current filters and have no way to teach SF that these are indeed spam emails.
Secondly, can you please explain exactly how the SFDB works, as a HUGE customer (Tesco ... the largest supermarket in the UK) seems to have their mail server's IP listed in SFDB, but not in any other DB as in spamcop, spamhause, ordb etc.
Also, how can we get this IP delisted??
Cheers
|
Posted By: LogSat
Date Posted: 11 August 2006 at 7:46am
If it's listed it's... probably because they are sending unsolicited emails. If you login the registered area on our website, you'll be able to enter an IP address and perform a lookup in the SFDB to see how many reports there are for that IP, and for what reason it was reported.
Our SFDB is updated in realtime, so the split second they would start an email campaign to their subscriber list, should they emails to unsolicited recipients, they will be blacklisted. If the campaign stops, they will be delisted automatically in either 6 or 24 hours, depending on the amount of spam they had.
If it's not listed in other services... well, that may just mean the SFDB is better than others
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: StevenJohns
Date Posted: 11 August 2006 at 8:01am
ok, no problem....exactly where would this registered area login page be??? I can't find it.
|
Posted By: LogSat
Date Posted: 11 August 2006 at 4:04pm
The link is found from the download page. More specifically, it's:
http://www.logsat.com/sfi-login.asp - http://www.logsat.com/sfi-login.asp
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: StevenJohns
Date Posted: 11 August 2006 at 4:06pm
Posted By: Marco
Date Posted: 16 August 2006 at 9:42am
SFDB went straight to the top after it's implementation. It simply is the best performing filter (dare i say it) on the web....
How it works: if a (settable) number of SF users declare a certain mail as beeing spam, ALL SF members reject the mail in question. So it's a joint effort / joint intelect type of filter. ANY spam that is not caught by SFDB filter, is because of 'your own' doing, and SFDB is fed with that.
------------- Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
Posted By: StevenJohns
Date Posted: 16 August 2006 at 9:45am
I'm sorry, but I don't share you'r feelings......
SFDB blacklisted Tesco, the largest supermarket in the UK.....well donw...NOT !!!!!
|
Posted By: Marco
Date Posted: 16 August 2006 at 9:48am
refresh and see what i added.. then you'll undersdtand
sorry, I didnt describe it correctly, it should be;
*my* SF will block the sender since 'N' number of users declared the mail as beeing spam...
Apparantly, a number of the SF users think the tesco mails are spam, and your system believes it, since 'n' number of users say so.
------------- Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
Posted By: StevenJohns
Date Posted: 16 August 2006 at 10:06am
Marco,
I know how it works, and I think it has at least one fatal flaw in it's design.
If 10,000 users sign up for a mailing list, then some time later, 100 of then descide that they no longer want it....invariably they don't bother to unsubscribe, they just mark it as spam.
Using this scenario, lazy/stupid/ignorant users can quite easily report an email as being spam, whereas in fact it isn't. The problem here is that if the company in question uses a single mail server IP for both the mailing list and the normal company mail (as Tesco do), then all of the normal email (Tesco use email to place orders with their suppliers, which can be for hundreds of thousands of pounds) will also be blocked.
Therefore, the failing here is that SFDB has no confirmation that the email that is being submitted is in fact spam.
I would suggest that it is very likely that the number of users declaring mail as spam is far too small.
Just my opinion...
|
Posted By: Marco
Date Posted: 16 August 2006 at 10:17am
As far as i know, your spam/mail system is 'only' one user in the SFDB chain, so, no matter how many of your internal users decalre the mailer as spammer, your system will issue the sending IP ONCE to SFDB.
So, if 'n' number of antiispamsystems say it is spam (read: ISP's/businesses), your system will adhere to that, no matter HOW OFTEN those ISP/businesses report the sending IP as spammer, it will only keep the ip alive in the SFDB.
you know the fun thing? you can bypass the SFDB alltogether, or increase the number, so that the chance of a falce positive gets smaller.
the setting "network reliability' number is the one you need to set to zero, or raise...
------------- Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
Posted By: StevenJohns
Date Posted: 16 August 2006 at 10:53am
Excellent...didn't know what that setting was...cheers.
any suggestion on a suitable number...it defaults to 3.....maybe a bit low
|
Posted By: WebGuyz
Date Posted: 16 August 2006 at 11:09am
Had mine set to 3 but then a Hotmail server started getting rejected by SFDB so I bumped mine up to 6. Seems to be pretty good and haven't seen any more major ISP servers stopped.
------------- http://www.webguyz.net
|
Posted By: Marco
Date Posted: 16 August 2006 at 11:10am
yeah, 3 might be a bit too low for your organisation, try 10 and see what happens, if your false positives go away, work your way down by one a day/week untill they return. The lower the number the more 'aggressive' the filter works, the higher the number, the lesser affective it becomes.
You need to find out what works best for you by trial and error.
------------- Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
Posted By: Marco
Date Posted: 16 August 2006 at 11:33am
p.s. you can allways whitelist this tesco domain, if you think it ok all your internal users can receive their mails.
------------- Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams
|
Posted By: LogSat
Date Posted: 16 August 2006 at 4:26pm
Marco,
Thanks for your explanation, you are right on the money. We actually slightly increased the lower limit on the SFDB lookups on our database server itself. This means that our SFDB lookups won't provide a positive match unless there are at least nnn other reports. I'd like to keep the number private so that lurker spammers don't see it. If you wish to know just PM/email us, we'll provide the value.
StevenJohns, if Tesco was listed in the SFDB, it's because a rather large number of recipients have complained. And while I have no proof, I'm guessing it's quite larger than 100... Please note that there have to be a certain number of *different* providers/companies running SpamFilter that have received emails from Tesco, not just one company. For example, if 100 users at LogSat.com complained that it was spam, the report would be only one, from logsat.com. However, the 100 users from LogSat.com would report the spam to MAPS RBL services, like sorbs or spamcop. In this case, if these service blacklist Tesco, and multiple SpamFilter administrators use these RBLs, then yes, each SpamFilter installation would send a report. In this case, the SFDB would then contain the same information as the RBL. If users believe this is wrong, or do not wish to use the SFDB for IPs that were blacklisted due to RBLs, the SFDB lookups can be configured to not use RBL (or any other filter you wish).
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: StevenJohns
Date Posted: 16 August 2006 at 4:49pm
Roberto,
Now I'm confused...
An earlier post suggested that the "Network Reliability" number was the minimum number of "spam complaints" that an IP had to have before SF DB would block the IP. Then, (as the default is 3), Webguyz suggested 6, Marco suggested 10 as being a suitable number, but you are suggesting that the tesco IP might have more than 100 "complaints".
Please can you have a look at your database and give me some examples of how many "complaints" some IP's typically get. I understand that you may not wish prying spammers to know this, so feel free to PM it to me. This will give me a good idea as to what a realistic number should be.
Cheers
|
Posted By: LogSat
Date Posted: 16 August 2006 at 5:04pm
As I mentioned earlier, you can always login the registered user area of the website, and perform detailed lookups on the SFDB database to see more details on what is being blocked and for what reason.
Let me explain the "100 complaints".
Consider the following example:
SpamFilter is shipped with default RBL servers. One of them is spamcop's bl.spamcop.net. Tesco sends 10,000 emails, and 100 customers complain and report Tesco to spamcop. Spamcop will then blacklist Tesco in its blacklist. Hundreds of SpamFilter administrators are using spamcop, so when they receive an email form Tesco, they will report it as spam to the SFDB. If more than "nn" different SpamFilter's report that IP to the SFDB, then the SFDB will list the IP.
Please note that Tesco would have to send emails to users who's provider runs SpamFilter (and is using spamcop), in order for SpamFilter to report the IP.
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: sgeorge
Date Posted: 16 August 2006 at 5:08pm
Steven, maybe I can try to explain...
The network reliability number is the number of registered SpamFilter installations that have reported one or more blocked emails from a particular email.
Let's try a hypothetical example...
- Your network reliability level is set to 3
- my domain, which uses the SFDB, blocks 20 messages from 123.123.123.123. my domain submits 123.123.123.123 to the SFDB, and the SFDB accepts 1 and only 1 submission for 123.123.123.123 from me.
- Joe's domain, also on the SFDB, blocks 5 messages from 123.123.123.123; that becomes the 2nd submission for 123.123.123.123 on the SFDB.
- Bob's domain, also on the SFDB, blocks 5 message from 123.123.123.123 as well;that becomes the 2nd submission for 123.123.123.123 on the SFDB.
- You are receiving a message from 123.123.123.123. Your SpamFilter setup is trying to determine if it should accept the connection. When you query the SFDB, it says 123.123.123.123 has been blocked 3 times. Since this is greater than or equal to your network reliability limit, you block the message from 123.123.123.123.
If I'm not making sense, then that's just me being myself.
Stephen
|
Posted By: StevenJohns
Date Posted: 16 August 2006 at 5:46pm
Roberto, got the PM..thanks.
sgeorge, thanks for the explanation.
I think that I need to whitelist the tesco domain for certain local domains. I know that I can whitelist certain sender email addresses for specific recipients, but can I whitelist mailto:*@tesco.com - *@tesco.com for mailto:*@mydomain.com - *@mydomain.com ?? if so, how??
Cheers
|
Posted By: LogSat
Date Posted: 16 August 2006 at 11:30pm
... there's actually a bug in SpamFilter that helps in your request. Please see the thread at http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=4970 - http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID= 4970.
It will also explain why the bug won't be fixed
------------- Roberto Franceschetti
http://www.logsat.com" rel="nofollow - LogSat Software
http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP
|
Posted By: StevenJohns
Date Posted: 17 August 2006 at 3:49am
excellent....sorted.
Thanks everyone for clearing this up.
|
Posted By: MartinC
Date Posted: 05 September 2006 at 12:03pm
so what is good level to start using the SFDB option?
we currently don't have it enabled but I'm certainly interested in trying it.
3 sounds dodgy from this thread.
10 or more?
and presumeably people are choosing the options which to check on like keywords, bayesian, invalid sender etc.
|
Posted By: WebGuyz
Date Posted: 05 September 2006 at 3:58pm
Really depends on your mail mix. Start with 6 and see how it goes. We tried 4 but ended up blocking hotmail.com and comcast.net which were in the SFDB.
------------- http://www.webguyz.net
|
|