Keywords... Help Collaborate a decent list. |
Post Reply |
Author | |
Sean
Groupie Joined: 21 February 2005 Location: United States Status: Offline Points: 81 |
Post Options
Thanks(0)
Posted: 19 May 2003 at 12:36pm |
I thought It would be nice, since most of us recieve similar SPAM, that we collaborate on a Keyword list. I don't know the best way to go about this, so suggestion are helpful. Basically if we share our thoughts on the keywords we use, and RegEX then maybe we can help each other by finding errors, and prevent legit emails from being blocked. As we all know the MAPS filter works great, but still blocks quite a few legit emails. So the way around this is to use the MAPS filter less and use the Keywords Filter more. Please let me know what you think about this.
|
|
James Taylor
Guest Group |
Post Options
Thanks(0)
|
Ok, so if you want a repository, I'll set up, merge the lists and make it available via ftp. I suppose that we need catagories? sex html gambling How should the list(s) be organized so that we can understand and edit them?
|
|
George
Guest Group |
Post Options
Thanks(0)
|
One question, who is going to determine what is spam. Not all people will agree that what one person sees as spam is spam to them. Don't get me wrong on this. I block 260+ keywords and my list is growing. I have a very aggressive anti SPAM policy on my network. If it weren’t for the overhead issues, a system that works like the black hole sites would be great. This would allow for a central database of banned keywords. This biggest problem we are facing now is the HTML bogus code that is used to breakup the keywords. I would like to see some help from the Regex guru's out there in creating ways to block this problem. What is needed is a way (if,else, then type of expressions) to allow valid HTML code to pass while stopping the many bogus codes used. Any ideas would be much appreciated. |
|
Sean
Groupie Joined: 21 February 2005 Location: United States Status: Offline Points: 81 |
Post Options
Thanks(0)
|
A general Keyword List list doesn't exactly mean every user would want to use it, or use it as is. the nice thing is any user could edit it, just using the General Keyword Filter Lsit as a basic Starting point. As I read through other post on this forum it seems to be a common question of new users, on "suggestions" and "examples" of other users keyword List. Also it is a good point that many of us have no clue what RegEx are, or how to write them. which is why a general list would be great. The point of who decides what should be block is deteremined by all the support users of the list, basically saying that if you find an error, because of the latest release containing the word "a" for some reason (possible in error), an it is blocking emails that are legit then a user suggest to have that remove in the next release, this doesnt mean they can remove it in the current realease they have. In Short, the General List would just provide an up-to-date example that other users have found to work well. |
|
Sean
Groupie Joined: 21 February 2005 Location: United States Status: Offline Points: 81 |
Post Options
Thanks(0)
|
I guess we should get started. If you have a good example, because as of 5/21/2003 2:00PM EST the list is empty, Please post it here. For now the General Keywords Filter list can be found here: http://199.6.41.242:8080/SPAMFilterISP/index.htm
|
|
Sean
Groupie Joined: 21 February 2005 Location: United States Status: Offline Points: 81 |
Post Options
Thanks(0)
|
George Can you post you running keywords list here, so we can use that as the basis of our project?
|
|
George
Guest Group |
Post Options
Thanks(0)
|
This is my current list of banned keywords. It is long and grows daily as I get spammed. I hate SPAM!!!!!!! The last 8 are to block the new virus that poses as an email from Microsoft support. List starts below. <font color="#ffffff"> |
|
Sean
Groupie Joined: 21 February 2005 Location: United States Status: Offline Points: 81 |
Post Options
Thanks(0)
|
Thanks! I added it! that will give us a great start.
|
|
Desperado
Senior Member Joined: 27 January 2005 Location: United States Status: Offline Points: 1143 |
Post Options
Thanks(0)
|
OK ... I tend to get long winded but I have been at this anti-spam problem for over 2 years and it has been one of my "pet projects". Please excuse the fact that I can't spell to save my life. Here is my take on the whole filter list problem. As a commercial ISP, we find that some people actualy WANT to enlarge their penis or partake in odd (to me anyway) sexual practices. We also have a lot of realestate companies so the whole "finance your home" issue is a problem. Unless you are administrating an Enterprise network, and are in the position of censoring your users, ANY filter list has the problem of "throwing out the baby with the bath water". We are looking at "self learning" bezier filters that score the percent PROBABILITY that the mail is SPAM. If, at some date, a "hook" is developed as part of the spamfilterISP software that can take advantage of this kind of filter, AND the end user can customize their level of blocking, then filtering becomes more reliable. There are antispam packages out there that use this method ... we looked at several of them. They are ALL in the $80K - $100K price range! By just using our preferred dnsbl lists and our own dnsbl, AND testing for RDNS, with the SpamFilter ISP software, we are tossing about 80% of our inbound mail as junk. So far, we have had VERY few messages that our customers actually wanted and my own inbox has dropped from my usual 8-9 hundred SPAM messages in a 24 hour period to about a total of 12! Bottom line ... any attempts we make to use filter LISTS, causes us lots of tech support issues and the Spammers are constantly changing theit tactics so the filters break. We find that helping the operators of dnsbl lists is more productive ... for our application as I said. For your personal or Enterprise networks, keep pounding away at the filter lists. Dan Seligmann Mags Net, LLC |
|
George
Guest Group |
Post Options
Thanks(0)
|
You being a commercial ISP, you should already know that 98% of these kinds of bulk mail ads are scams. Anyone foolish enough to think that they will actually get a bigger penis or real mortgage re-finance loan from some spammer that is sending thousands of spam from some foreign country using a fake email address is just plain stupid. I find it crazy that with all the news of the scams and fraud anyone thinks this crap is real. The only ones that profit from SPAM are the spammers who turn around and sell the email address's that are verified by dummies who actually clink on the links.
The word "Spam" as applied to Email means Unsolicited Bulk Email ("UBE"). Unsolicited means that the Recipient has not granted verifiable permission for the message to be sent. Bulk means that the message is sent as part of a larger collection of messages, all having substantively identical content. Technical Definition: An electronic message is "spam" IF: (1) the recipient's personal identity and context are irrelevant because the message is equally applicable to many other potential recipients; AND (2) the recipient has not verifiably granted deliberate, explicit, and still-revocable permission for it to be sent; AND (3) the transmission and reception of the message appears to the recipient to give a disproportionate benefit to the sender. |
|
George
Guest Group |
Post Options
Thanks(0)
|
Just a follow up. Out of 75792 Email attempts, 5377 emails were forwarded in the last week. |
|
Sean
Groupie Joined: 21 February 2005 Location: United States Status: Offline Points: 81 |
Post Options
Thanks(0)
|
I'm agreeing with both of you here... SPAM is an annoyance, but as stated, some people actually do want it, odd or not, its true, people actually sign up to recieve some of this stuff, however as George said most is not legit, and by keyword filtering you run the risk of blocking both legit and non-legit. It's the price you pay to help combat this problem. Now as for RBL list, these can tend to be a problem, they at time block entire subnets, which leads to a loss of a great amount of legit emails, why block an enitire domain because a Spamer used their mail server, chances are the SPAM could have just been a spoofed IP cause the RBL to include the address. There is so much involved in blocking SPAM, its a major project. You have to be willing to do so though. Now for another problem. In a running environment is the only way to test to see if your filters work, problem being it creates a large amount of time and resources to sift through the quarentine. One major enhancement I must suggest to be pushed for is "Tagging". with this, it will place a message in the Subject. (EX. "*POSSIBLE SPAM CAUGHT BY FILTER* - enlarge your pen1s t0day"). this would allow the Admin to pick up and generate their Black list, as well as ensure all emails are still going through. Don't get me wrong the quarentine is great, but it is time consumming. By The Way LogSat... Awesome Product!!!!!! Keep up the good work! |
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
Hi Sean, We may add subject tagging in the very near future. However you reason for having it may disappear after using the new quarantine we are developing. It's database-based, and we are developing both ASP and PHP interfaces that will allow the end users to sift thru their quarantine items, without bothering the administrators. The will be able to see/preview/force-send/delete their quarantined items. Access to the SpamFilter server will not be needed (security....). All that's required is that the web server be able to communicate with the back-end database. We're aiming to support the following for now: MS Access (low traffic admins may have the need for this), MS-SQL, Oracle and MySQL. We should be able to release a beta build very soon, within days. Roberto Franceschetti |
|
George
Guest Group |
Post Options
Thanks(0)
|
Roberto, That being said, due to the many different email servers in use, it would not work to use the mail server login accounts due to incompatibility problems. So this leaves an opt-in model that would have to be verified by the admin on the system. Is this close or do you have something else in mind? |
|
Desperado
Senior Member Joined: 27 January 2005 Location: United States Status: Offline Points: 1143 |
Post Options
Thanks(0)
|
MOST, not ALL email servers support poppassd. This would be a clean interface to the pop servers password file. Of course, MS exchange would be an issue. Most popservers I have worked with have similar ways to get at the pop passwords but I wouldn't know how to integrate them. For myself, being that we have so many accounts, and adding usernames and passwords one at a time would be a problem, I can visualize a web form where the customer requests access to the Spam Management. The web form can then automatically email the customer back (if someone was trying to spoof, the actual user will get the email) and if the user than replies back, an administrator merely clicks on an "approved" link and the form adds the username (the users email address) and his password into a database. We do this for several of the secure services we provide and it takes about 5 minutes a day out of my life to manage it. The first 1 or 2 days would be the only "log jam". Just an idea.
Dan S.
|
|
Abdu
Guest Group |
Post Options
Thanks(0)
|
Why are you filtering out <font color="#ffffff">? These are also valid ones. Basically you're blocking any html message that has a comment. <!--a ... |
|
Post Reply | |
Tweet
|
Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
This page was generated in 0.398 seconds.