Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - SMTPAUTH HACKED
  FAQ FAQ  Forum Search   Register Register  Login Login

SMTPAUTH HACKED

 Post Reply Post Reply
Author
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Topic: SMTPAUTH HACKED
    Posted: 26 March 2014 at 7:45am
Hi all,

our configuration is:

SFE1: standard spamfilter 
SFEAUTH: for smtpauth mode (unix authentication)
HMAIL SERVER: mail server 

SFE1 forward external messages to hmail domains.
SFEAUTH: sends messages in authentication mode to external account.
They forward messages to hmail server.

Yesterday our systems were attacked by massive spam messages.

In SFE1 log we can see:

"Bypassed all rules for: snyderrobertw@yahoo.com from trueaccount@ourdomain.xx ( User authenticated with AUTH LOGIN)
03/24/14 23.59.18.983 -- (3660) Received RCPT TO: e9367@comcast.net
03/24/14 23.59.18.983 -- (3660) Bypassed all rules for: e9367@comcast.net from xonaly@ourdomain.xx ( User authenticated with AUTH LOGIN)".

trueaccount@ourdomain.xx is a smtpauth account (is present in unix list pwd)
xonaly@ourdomain.xx  is NOT a smtpauth account. It doesn't exist at all.

The same problem occurs even if we delete the trueaccount@ourdomain.xx account from unix pwd list and with several someaccount@ourdomain.xx non-existent account.

It seems that the unix authentication would be hacked.

Please, could you help?

Many thanks.
Regards.
SUPPORTO TECNICO



Edited by alfaproject - 26 March 2014 at 10:13am
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 26 March 2014 at 8:14am
I tested via telnet the SMTP AUTH on both of your SpamFilter servers, and incorrect passwords are being correctly rejected. Are you certain that you have removed the affected user(s) from the same Unix passed file that SpamFilter is configured to use? The path to that file is in the "Settings - User Authentication" tab in SpamFilter, and appears when selecting the "Unix Passwd File" tab.

If the issue persists, could you please zip us the following so we can take a look:

• SpamFilter's activity logfile for today

• Your SpamFilter.ini and the PASSWD Unix password files

• The \SpamFilter\Domains directory structure (if the files containing any of your blacklists/whitelists are outside that directory tree, please include those as well.


If the zipped file is over 8MB in size, please try to upload the file to our repository at:

https://cloud.logsat.com/public.php?service=files&t=ec1a5cd65f702ae7d9cacfe951fe063a


I'll provide you the password for the above URL via a separate PM.


In the meantime, if most of these AUTH LOGIN attempts are originating from a single IP of a specific subnet, you could also go to the "Settings - Debug View" tab in SpamFilter. There you can enter an IP or a subnet over which you can capture some of the SMTP traffic. If you can capture some of the successful SMTP AUTH events there, we would be able to see what the spammer is doing to gain access to the user's account(s). If you do that, could you please also include the output from that capture in the zip file? 

Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 26 March 2014 at 11:09am
Hi,
yes, I'm sure: I deleted the affected user from Unix Password File.

I'll send you the files required.
Please, could you provide me the pwd for your cloud url?

I'll put a folder containing:

- 2 example email messages showing  the problem;
- yesterday SFE1 zipped log file (today, I have to stop forwarding service bacause the problem is going on...);
- the spamfilter.ini and unix-password files;
- the domains folder structure image.

These AUTH LOGIN attempts are originating from multiple different IP, so I can capture some example, only.
I'll try to do it and I'll send you a zipped file.

MaNy thanks.
Regards.

SUPPORT TECNICO

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 26 March 2014 at 12:27pm
We received the files. The strange thing in the logs is that the username being used to authenticate is not being logged. It should appear right after the entry "User authenticated with AUTH LOGIN".

The SMTP trace capture is at this point very important to see what is happening. As you said there are many different IPs sending these emails. You can try entering:

1*

in the Debug View's "IP to monitor". This will capture traffic from any IP that start with a "1", so it won't capture *everything* (which would likely cause performance issue with SpamFilter) but it should capture enough traffic to eventually catch one connection that did the SMTP AUTH command. Could you please email us the contents of that output when you captured some of it?

If you'd like and you prefer to perform a real packet capture with WireShark for a few minutes, we can also analyze the WireShark .pcap file to troubleshoot the problem. Just make sure to save the original raw data when saving with Wireshark, and not just the text output.

You can email us directly at support @ logsat.com if you prefer.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 26 March 2014 at 12:30pm
Hi,
meanwhile, I have another question related to our problem.
In log file there are several rows like this:
Socket Error # 10054 -- Connection reset by peer. - forwarding to:10.0.0.2 - message queued - trueaccount@ourdomain.xx
03/25/14 00.01.21.446 -- (2792) Read Timeout - forwarding to:10.0.0.2 - message queued  trueaccount@ourdomain.xx.

What does it mean?
I mean: the message has been forward to the recipient? Is that message in some sfe folder?
Because, I notice that several messages still have not been received from final recipients.

Many thanks.
Regards.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 26 March 2014 at 12:48pm
If the forwarding attempt was disconnected, the messages should be queued in the \SpamFilter\queue folder, and SpamFilter will retry sending them every 20-60 minutes (depending on your retry settings).

I have an updated while we debug your logs. The only account that is used to send emails using SMTP AUTH is the "a.dan______@______.___". That account is indeed being logged after the "User authenticated with AUTH LOGIN: a.dan________@_____". I had missed that as the spammers are sending multiple emails in a single connection, and only the first one of the series will log the AUTH username, leaving all the others empty.

Have you restarted SpamFilter after removing that account from your Unix password file? I ask as the user only needs to authenticate once during an SMTP session. Once authenticated at the beginning of it, they don't need to authenticate again untile they disconnect and then re-connect. For example there was an SMTP session that lasted over 4 hours:

03/25/14 03.50.06.377 -- (2972) Connection from: 37.11.194.210  -  Originating country : N/A
03/25/14 07.55.25.804 -- (2972) Disconnect

that resulted in a *lot* of emails sent from that authenticated user (always the same). If you close and restart SpamFilter after changing the Unix password file you would force everyone to reconnect and thus re-authenticate immediately. 


Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 27 March 2014 at 4:57am
Hi,
yes, I've restarted SFE many times after unix file changing.
At the moment I've stopped hmail to send messagges from local to external address to allow SFE to process more than 100.000 queue messages.
As soon as possible, I'll send you a debug view report, or Wireshark one.

Many thanks.
Regards.
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 28 March 2014 at 7:35am
Hi,

in SFE queue folder there are a lot of .rcpt file with this structure:
FROM: <ourtrueaddress@our_domain.xx> (the one involved in the problem)
some crearly non-existent email address.

Can I delete these messages without causing any other problems to SFE1 queue delivery?

Many thanks
Regards.
ALFAPROJECT
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 28 March 2014 at 7:56am
The files in the queue are always in pair. One with either a nnnnnnn.tmp or a with a nnnnnnn.~tmp extension, and one with nnnnnnn.tmp.rcpt extension. The .tmp contains the message itself, and the .tmp.rcpt contains the to/from email addresses. 

If you just delete the .tmp.rcpt file, SpamFilter will cleanup after itself next time the queue is processed by removing the orphaned .tmp file. If possible however you should delete both files to prevent SpamFilter from doing extra work and possibly slowing it down while processing those orphaned files.

Do you have any SMTP or packet captures you'd like for us to analyze to see how this issue occurred?
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 28 March 2014 at 12:44pm
Thanks a lot for your reply.
At the moment SFE1 has 60.000 queue messages to deliver.
I think I'll be able to perform a packet capture on next days.
I'll send you the results.

Many thanks. 
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 31 March 2014 at 10:33am
Hi,

from SFE queue messages I extracted some IPs from which spam was sent and I put them in SFE BL list.
Now, I've activated the debug view to monitor 1* ip range.
How can we protect the unix pwd list?
Is there something else to activate to avoid the problem in object?

Many thanks.

Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 31 March 2014 at 11:08am
Hi,

some other questions:
- can I reduce the SFE-AUTH single connection time?
- can I configure SSL port on SFE and SFE-AUTH? If so, how to do it?
- in SFE-AUTH the BL IPs list checking is bypassed by the user authentication?

Many thanks.
Regards.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 31 March 2014 at 2:02pm
SMTP AUTH is one of the commands that can be transmitted over SMTP, so there are no configuration settings like timeouts or SSL that can be configured just for that. 

SSL can certainly be enabled for SpamFilter, but you would not be able for example to force SMTP AUTH to use the SSL port you configured. SMTP AUTH would still be available on the non-SSL port.

I'll update the post shortly to confirm or not whether the SMTP AUTH whitelisting will bypass the BL IP checking - I'm not certain at the moment without testing this in the lab. This has never been an issue in the past so it's not a common question.

We'll wait for your debug report as I'm hoping it will have the info we need to determine what is happening.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 31 March 2014 at 7:46pm
We've confirmed that the users who login with SMTP AUTH will be whitelisted bypassing any BL IP checking.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2014 at 11:25am
Hi,

I'm mailing you the wireskark report.
It seems there is nothing strange in it.

Please, tell me if you need other details.

Many thanks.
Regards.
Back to Top
alfaproject View Drop Down
Newbie
Newbie


Joined: 10 January 2008
Location: milan
Status: Offline
Points: 12
Post Options Post Options   Thanks (0) Thanks(0)   Quote alfaproject Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2014 at 11:29am
Sorry,

the file dimension is too big to send you by email.

Could I put it on your cloud folder?
Please, could you give me access credentials?

Thnaks.
Regards.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 02 April 2014 at 12:30pm
Sure - I had sent you a private message on the forum a few days ago with the password and URL for our cloud area. I just re-sent it to you. If you check your profile here on the form account you should be able to see the PM notifications.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 05 May 2014 at 11:44am
Was there any resolution to this. We use Unix SMTP Auth and are interested if there is any problem with it.
http://www.webguyz.net
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 05 May 2014 at 12:30pm
No issues had been found. A user appeared to have their SMTP password compromised, and a spammer was using that account to relay email. They kept an open/connected SMTP session which was using to send the spam. Deleting the user from the Unix passed file will prevent the user from performing further SMTP AUTH logins, but the current SMTP session is not disconnected, so they kept spamming for a while. There were also tens of thousands of emails in the SpamFilter queue, so the outbound traffic actually continued (emptying the queue) even after the spammer was unable to login any further.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 07 May 2014 at 7:30am
Is there any way to kill a session short of restarting SFI? What settings in global options would force these type spam attacks to have to authenticate more often.

We monitor the SFI log and if we see anyone sending more then 100 emails in 2 minutes we remove that email address from the auth password file, but in a case like this a lot of spam would still would get thru. We need to have them try to authenticate more frequently so they would stopped.



http://www.webguyz.net
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4104
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 07 May 2014 at 5:01pm
You can see what sessions are active from the "Connections" tab in SpamFilter. From there clicking on the "X" on the last column ("Kill") will disconnect that session immediately.

There are currently no parameters to limit the number of emails an authenticated user can send. You have a very valid argument for adding an option to impose a limit - we'll work on this shortly.

FYI if you don't see entries in the "Connections" tab make sure the "Disable Conns Grid" option under the main Settings-Configuration tab is not checked.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 07 May 2014 at 5:24pm
Would be good to have some limits but doesn't help a lot if it starts at 2am and you wake up to find yahoo.com and gmail.com will no longer accept emails from you because of the thousands of spams that been pushed.

Been working on a more automated solution since a lot of these attacks occur in the wee hours of the morning.

Have a vb script that runs continuously in a loop with a 60 second delay. It reads in the SFI log file and uses Dictionary component to keep track of IP's and Senders addresses and number of emails sent.

If a user sends more then 150 emails in 1 minute we read in the password file and remove the authenticated email address.

We then use MS firewall CLI commands in our script to block that IP immediately. These 2 steps nip the spamming in the bud. if a valid customer happens to send more then 150 emails in 1 minute then we just apologize and unblock his IP and add his email address back in the password file.

The speed in which spammers can pump out spam is incredible and manual methods are too slow.

if you could find a way to automatically limit an IP when it get detected sending thousands of emails in a short period then the world would beat a path to your door :-)
http://www.webguyz.net
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.313 seconds.