Idea to reduce database usage |
Post Reply |
Author | |
kspare
Senior Member Joined: 26 January 2005 Location: Canada Status: Offline Points: 334 |
Post Options
Thanks(0)
Posted: 28 September 2005 at 6:57pm |
This will have it's ups and downs, but it's more the idea i'm looking to pursue right now. When you connect to an smtp server it is possible to verify if the user exists, why couldn't we write a script or integrate this into spamfilter so that it could run maint once a night to check against the database and against users servers to see if users exist? Spamfilter already knows where to send email, either direct or to an alternative gateway so that info is already there, it's just the process. comments? |
|
keizersozay
Groupie Joined: 26 January 2005 Location: United States Status: Offline Points: 77 |
Post Options
Thanks(0)
|
I'm not sure if I understand you but let me take a shot. You are suggesting that since SpamFilter sends email to our email server it can learn which email addresses are real since the email server (exchange or whatever) will either accept it or not. Then at night (or whenever) SpamFilter can go through and figure out which email addresses are actually active and only accept email for those? If that is the case I don't think it will work for a few reasons. I and others here use another filter (Trend IMSS) behind SpamFilter for various reasons, so before the email gets to our email server it passes though another filter and SpamFilter never gets a response from the email server about the validity of the email address. Also, it would have to be a real time solution because when we add new email addresses, those people need to be able to get email immediately. If I told them it would take 24 hours or 'till the next day I will be flipping burgers before long. With that being said I do like you idea. I had suggested in the past that SpamFilter have an optional LDAP connection to a server you specify (a domain controller) and for every email it could do a small query to see if the email address is real. This was shot down because it gets away from the main duty of SpamFilter and it was suggested that I script all valid email addresses out of our environment and have them automatically added to our approved sender list. This would work, but I haven't figured out how to do it yet. Again, I’m not even sure if I understand your question correctly, so if I got it wrong just ignore all this. Thanks |
|
Kim
Guest Group |
Post Options
Thanks(0)
|
What is Trend IMSS used for? /Kim (got interested....) |
|
keizersozay
Groupie Joined: 26 January 2005 Location: United States Status: Offline Points: 77 |
Post Options
Thanks(0)
|
The Trend Interscan Messaging Security Suite is used for several things. It does have some spam filter ability but that isn’t really too good unless you buy the addition spam piece and SpamFilter beats it hands down so we just use it for creating disclaimers and some other policy type rules. It also does all our virus checking, which works very well. |
|
kspare
Senior Member Joined: 26 January 2005 Location: Canada Status: Offline Points: 334 |
Post Options
Thanks(0)
|
You kinda missed what i was getting at. Midnight maint would do this, it would run through the database and collect email addy's. For each email addy it would attempt to send an email to that address, when you connect to an smtp server, after helo the recipient data comes through, and at this point the email server will accept or decline the message. If the users smtp server declines the message it could be flagged for deletion. Does that make a little more sense? |
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
Would be just easier to write a script to get the userlist for everyone and put it into the AuthorizedTo list on a regular basis. We poll our 2 mailservers every 10 minutes and download a list of all users (about 6200) and put them into the authorizedTo list. We poll this frequently so as new customers come on board and start adding email users they are automatically added in 10 minutes. Amazing how much stuff is stopped by the AuthorizedTo list. Most mail servers should allow some type of scripting to get this info. We use Merak and Imail. Even if you use ldap you should be able to query the mailserver and build a list of valid email addresses and overwrite the AuthorizedTo list on a regular basis. |
|
http://www.webguyz.net
|
|
keizersozay
Groupie Joined: 26 January 2005 Location: United States Status: Offline Points: 77 |
Post Options
Thanks(0)
|
can anyone offer some help on scripting valid email addresses out of an exchange/ldap environment? Thanks. |
|
kspare
Senior Member Joined: 26 January 2005 Location: Canada Status: Offline Points: 334 |
Post Options
Thanks(0)
|
No, not all of the mail the comes in is destined for our mail server, we provide a spam relay service for many customers so we don't always have direct access to their servers.
|
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
We have 3 customers who have Exchange servers and we provide relay services for. We wrote a web interface that allows their admin to enter their users login/pasword info into a table and have a script that polls that info every 10 minutes (same script as mentioned above) and also adds that to the AuthorizedTo list. This is helpful to us because we charge per user per month. Customers don't bug us every time they want to add or remove a user, they just update it themselves and in 10 minutes its active. We run a report of these users at the end of the month and bill accordingly. For those who relay for others, do you just charge a flat fee per month?? |
|
http://www.webguyz.net
|
|
kspare
Senior Member Joined: 26 January 2005 Location: Canada Status: Offline Points: 334 |
Post Options
Thanks(0)
|
Well for now all the customers are also customers whom I manage their exchange server so it's not a big deal for billing, however that is an interesting way to look after billing! Currently we do email based entirely on domains, which has been successful, it is based on the honor system as far as billing goes. |
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
Another added benefit of doing this is that these exchange users can now log into the quarantine db with their own login/password to manage their own filtered mail. Most know to check once a day and we keep it for 48 hours. We also have these exchange users use us as a smarthost so that we could get a list of everyones outgoing emails (as well as our local users) . We add to the AutoWhitelistDelivery.txt any emails that our valid customers send out. We do this with scripts we wrote to poll our dedicated outgoing mail server every 10 minutes and create valid SENDER|RECEIVER pairs for all emails that are sent out. The premise being that if they are sending an email they must be wanting a reply back. Our AutoWhitelistDelivery.txt file is 1.3 meg big right now and has about 28,000 entries. This has cut down false positives down dramatically!!
|
|
http://www.webguyz.net
|
|
Alan
Groupie Joined: 06 May 2005 Location: United States Status: Offline Points: 43 |
Post Options
Thanks(0)
|
Hey WebGuyz,
I am a bit disappointed that the new LDAP feature in SF in not used for incoming. But I see that you have another method to do something similar. Could you provide some assitance or script to help with your method of automatically extracting LDAP emails from Exchange. I am being hit hart by a spammer spoofing my domain with literally tens of thousands of bounces just over the past two days. You can contact me via the forum private messaging if you want to keep confidential. Thanks for any help you can provide. |
|
ImInAfrica
Groupie Joined: 27 June 2006 Location: FL, USA Status: Offline Points: 60 |
Post Options
Thanks(0)
|
> can anyone offer some help on scripting valid email addresses out of an exchange/ldap environment?
What if SF builds a list out of rejected email addresses and caches those that were rejected for specified period of time as blacklisted? Example: - SF recieves an email for user1@domain.com - SF accepts it and tries to forward it to the specified server. - The recieving server rejects with "No User found". - SF then caches the email user1@domain.com - Further email to user1@domain.com are rejected for 24 hours (or 2 hours, or whatever is specified in the config file). Pretty similar to what milter-ahead does in the *nix world. Best Amir |
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
Alan, Take a look at this ADSI script below. If you can get it to work then maybe it can be modified to get a listing of Exchange mailboxes. I don't have Exchange but if its in the ballpark then maybe it can be a starting point. I can help you with the ASP, but not the Exchange testing. You may have to run this on the Exchange server.(hopefully you have to have IIS on that server) |
|
http://www.webguyz.net
|
|
Alan
Groupie Joined: 06 May 2005 Location: United States Status: Offline Points: 43 |
Post Options
Thanks(0)
|
Thanks. I was able to find a solution for addresses via LDAP on Ex55/NT4.
|
|
LogSat
Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4104 |
Post Options
Thanks(0)
|
As an FYI, we are evaluating whether to allow LDAP/Active Directory checks for incoming emails, to validate existing recipients.
The issue is that doing so could cause a Denial of Service attack on the LDAP/AD servers, as massive emails can result in massive amounts of queries to the directory servers, potentially bringing them down. |
|
WebGuyz
Senior Member Joined: 09 May 2005 Location: United States Status: Offline Points: 348 |
Post Options
Thanks(0)
|
The blacklist cache would stop repeat dictionary attackers and if you cached users who have already been LDAP authorised (and held them in memory cache for a settable time) then the load would minimal on the LDAP (or whatever) server. |
|
http://www.webguyz.net
|
|
Post Reply | |
Tweet
|
Forum Jump | Forum Permissions You cannot post new topics in this forum You cannot reply to topics in this forum You cannot delete your posts in this forum You cannot edit your posts in this forum You cannot create polls in this forum You cannot vote in polls in this forum |
This page was generated in 0.262 seconds.