Testing SMTP

Sometimes it's important to be able to manually reproduce what your computer/server is doing manually so you can see exactly where it's failing. Troubleshooting "utilities" are great in that they can often diagnose hundreds of issues in less time than it would take you to troubleshoot one, but they lack the nuance sometimes necessary to accurately diagnose issues.

A simple pass/fail is nice, but if you don't know WHY it's failing, what good is it?

When troubleshooting email, there are websites, utilities, diagnostic messages and more designed to help figure out what's going wrong. I have never found a single one that has been more informative than a simple SMTP session.

If you have telnet on your computer, that'll work fine. Netcat or any other command line tool that you can specify the ports to connect on will work too.

On your command/shell prompt type: telnet <the server's IP> 25

Then, when prompted, type the following commands:

---------------------------------------------------------------------------
HELO your hostname
<wait for response>
MAIL FROM: <your email address>
<wait for response>
RCPT TO: <recipient'semail address>
<wait for response>
DATA
<wait for response>
FROM: <your email address>
TO: <recipient'semail address>
SUBJECT: test

this is a test.
.

----------------------------------------------------------------

Note the additional CR/LF after the subject. That separates the headers from the body (visible part) of the message. It's very important.

Also important is the line with a "." by itself. This tells the recipient's server that the message is complete and to process it.

On to troubleshooting....

In each part above where it says "wait for a response" you're expecing a response from the server, either something like "ok, go ahead", an error, or sometimes nothing. Here are some examples:

HELO

If there's no response, this can indicate that the server is extremely busy or that sometimes there's packet filtration between you and the server that's intercepting and silently dropping the packets. Either way, if there's no response, there's no transmission, and therefore no email.

Sometimes you'll get an error here referencing that it does not like your IP (if you're blacklisted), or that it's imposing some type of lookup on your HELO argument. It's best to be sure thet you give your FQDN here to prevent issues.

MAIL FROM:

You might get SPF errors here if your SPF record is wrong, or if your email address is blacklisted you might get a response here.

RCPT TO:

"Relaying denied", "mailbox is full", or any number of other recipient level issues may result as a response to this line. Any server that has different rules per user or domain (like a shared spam filter with different settings per domain/user) might also process here.

DATA

You're now giving the actual data that will be shown to the user. Generally there won't be an issue here until you close the transmission channel (with the "." on a line by itself), but anywhere you wait for a response is a potential place for errors.

"." (or closing the transmission channel)

This is where the email in it's entirety can be scanned and processed. Sometimes nothing will be scanned until this point, so even though SPF *could* be looked up after the "mail from:" command, it *might* not be until the entire message has been transmitted. Any of the above errors can happen here as well as content based scanning of the body, etc.

Some mail servers will also accept the email here, and then scan it later before delivery. That's a horrible idea, but that doesn't stop anyone usually, so be sure to copy down any information the acceptance line gives you like the message ID, so you can reference it later if the test does not complete.

Generally, you'll only be able to run this test on a border MTA unless it's your own mail server or the recipient's firewall allows you through. This can definitely complicate troubleshooting, but if you can prove that the issue is with the border MTA, or that the border MTA accepts the email but it is not relayed it at least absolves you from any finger pointing.

Once you have these commands memorized (it doesn't take long), you'll often be able to troubleshoot and diagnose issues faster than submitting tickets to your vendors. There's nothing more frustrating than either waiting for a response only to find out you could have fixed it hours ago, or conversely that you know exactly where the error occurred, but the other admin requires "proof" that they should bother fixing the issue when the DSN or other logs obviously state the issue.

Send a couple SMTP sessions their way and copy the session to them. It's hard to argue with facts :-)


-TEA

subdomain MX records

We've all used subdomains. The popular naming convention "www.domain.com" is in itself a subdomain (the subdomain "www" in the "domain.com" domain), but did you know each subdomain can also have it's own MX records?

The obvious questions are "why would anyone do that" and "how would anyone do that?". Here's some food for thought.

Way back in the day, things were simpler. Email specifically didn't require tons of MX and A record lookups, there weren't even DNS servers to query let alone DNS record types. Users had accounts on their local servers, and the servers had names that although understandable would be different from organization to organization. Email addresses weren't "bob@company.com", they were "bob@accounting.company.com" for Bob in accounting and "bob@engineering.company.com" for Bob the engineer. Each organization would have their own server names, and although often they'd have meaning within the organization they might be goofy to outsiders. Also, desktop PCs were uncommon then, so you would connect to the server for almost all of your daily tasks. Therefore, rocky@bullwinkle.company.com was literally addressing the email to the user account "rocky" on the server named bullwinkle.company.com. Some places still conform to these odd naming conventions. I knew someone who went to the University of Michigan's Engineering school who had an email address @engin.umich.edu. It took me months to figure out that "engin" was for " school of ENGINeering".

Nowadays it's slightly different. "company.com" generally will have all of it's users email addresses directly @company.com, but not always. For instance, let's say there's a domain BigCompany.com. BigCompany.com is all over the world, and they want their customers to know that they're dealing with their "local" office. So, they may have John@USA.BigCompany.com and John@Canada.BigCompany.com, but they're totally different Johns, in totally different regions.

Let's say they not only want to have email addresses at the country level, but also at the state level. They could also have John@California.USA.BigCompany.com and John@NewYork.USA.BigCompany.com.

Now, most companies wouldn't do this because they don't want to confuse their customers by having them talk to the wrong John all the time, but you commonly see this with schools for instance. They might be something like "Principal@SchoolName.NY.k12.us".

Ok, now we know why someone would have subdomain MX records. Either to conform to a legacy naming convention or to help routing between different regions/offices. Now the question is "how do we do that"? The answer shouldn't be too surprising, we make MX records for each subdomain.

Just like each domain name can have MX records (and should have them if email is addressed to it), subdomains can get in on the fun too with their own set of records. Let's say you have an email address webmaster@www.domain.com, but you don't want to run a mail server on the same box as your web server (which is a good idea btw!). You can simply have MX records for www.domain.com go to the same server as your regular domain like this:

Host Priority Record
domain.com 10 mail.domain.com
www.domain.com 10 mail.domain.com

Now, you may have to create a separate zone for this subdomain if you're running your own DNS server, or if you're using some provider's interface they may require additional steps to configure this but most of the time it's possible. Some services don't allow subdomain records to be made if you're using their DNS, but DNS hosting is cheap. If they don't allow this, and they have ridiculously bad support who won't discuss the finer side of RFCs with you (cough, yahoo.com, cough), just change your DNS host. Outside of this issue, you're likely to find other reasons to switch providers later if you're not already disgusted with their lack of support.

It's important to keep in mind, that this is not something you can do after the fact. Once your users have you in their address book as user@domain.com, they will ALWAYS query domain.com's MX records regardless of whether you add an alias for user@sub.domain.com unless of course you choose to change your email address and deal with all of the headaches associated with that.

It also should not be used after the border MTA using address rewriting without serious contemplation. Although there are some excellent reasons to have your border MTA rewrite the delivery addresses to "user@sub.domain.com" while the rest of the world still emails the user@domain.com" address, there are serious considerations to take into account at the very least the reply-to address, or sender rewriting back to user@domain.com on the way out.

This is not likely something that many admins have dealt with before, and could be conceptually confusing at first. Don't worry, it's nothing different than what you do on the main domain, it's just a second place to deal with. User@domain.com queries domain.com's MX and sends them there. User@Sub.Domain.com queries sub.domain.com's MX and sends them there. It's pretty simple actually once you get your head around it. :-)


-TEA

DNS doesn't end at MX you know...

Ok, so you understand MX records, you're ready to take on the world, right?

Not quite yet. Patience young grasshopper.

So, you're sending email to user@domain.com, you queried dns1.domain.com for the MX records and now you've got these MX records:

Host Priority Record
domain.com 10 mail1.domain.com
domain.com 20 mail2.domain.com
domain.com 30 mail3.domain.com

Ummm, so, computers don't speak English, they don't know what a "mail1.domain.com" is. They want digits that can turn into nice 1s and 0s that they can compute . What's a poor mail server to do with this stuff?
While we're at it, how did we even know to talk to dns1.domain.com to get these records?
Also, once we knew to talk to dns1.domain.com, we're back to the first question of how does the mail server know WTH a "dns1.domain.com" is, and where are the 1s and 0s it needs?

I is so confus :-(

Don't worry, there's more to an email's life than just MX records. There's lots of types of records including type A, type SOA, and type CNAME.

I. Type A records

Type A records ("A" for "Address") turn "mail1.domain.com" into a nice pretty number (the "IP Address" or "Internet Protocol address") for our mail server (and all other computers) to compute. The A record might look like this:

Mail1.domain.com IN A 10.0.100.28

Now we can see that the first MX record for domain.com has a name of "mail1.domain.com", but that name actually resides at 10.0.100.28. This number means nothing to most of us silly humans, but to a computer it's a nice string of digits that it can compute and then connect to.

TCP/IP and all of it's routing mechanisms are beyond the scope of this article, so you'll just have to make do with "once you have the IP you can connect to the server". :-P pbbt!


II. SOA records

OK, so this is a little backwards since you need the SOA records before you can get the MX records or A records, but you should already understand MX records, and you need to understand A records before you can get SOA records.

SOA records are the "Start Of Authority" records. There are literally millions of DNS servers in the world. How do you know which ones are responsible for the records you're querying? Well, that's where SOA records come into play. SOA records are given by the "root" name servers to offload queries to other DNS servers, and to provide a distributed architecture that is easier to maintain and provides more fault tolerance.

Root name servers are the biggest and baddest name servers in the world. There are two sets of them for your .com, .net, and the like, one at root-servers.net and one at gtld-servers.net. If queried, they will either reference the other one (like a father who says "go ask your mother", or vice-versa) or they will give the DNS server registered with them as authoritative.

You can play with this yourself using nslookup on Windows or dig on *nix.

Once the root servers give the name of the DNS server, that server's A record must be resolved then it can be queried for the MX, A, CNAME, or any other type of records you wish.

III. CNAME records

CNAME records are basically aliases to A records. For instance, "www.domain.com" might be the same server as "ftp.domain.com", so you could make ftp.domain.com a CNAME for www.domain.com, then an additional query would happen for the A record "www.domain.com" which when resolved would then allow transmission.

Typically, CNAMEs should not be in MX records. There's a ton of debate over whether they're allowed in the MX at all according to RFCs. I'm not going to enter that debate, but I will say that there are enough mail servers on the planet that don't respect CNAMEs in the MX that you should do everything in your power not to put them there. At the very least it adds an additional query to your DNS server (to get the IP of the A record in the CNAME), so for simplicity's sake I'd recommend against it as well.

One time a CNAME might be used is if you have a server that accepts email that does not have MX records. Originally, user@domain.com was actually literally user@some.server.com. You could even address email to an IP address like "user@10.0.100.28" for example and it would work. Most mail servers will fall back to this convention if MX records cannot be resolved, so you could for instance have a CNAME errors.domain.com pointing to "mail1.domain.com", then you could email "www-server@errors.domain.com", "email-server@errors.domain.com", etc which would then actually send the email to mail1.domain.com.

NOTE: this does not alias other record types, only A records. For instance, setting domain2.com as a CNAME for domain1.com DOES NOT make domain2.com's MX records the same as domain1.com's. CNAME records are not copies of the zone, they are address aliases and will only route traffic to the A record they point to. Additionally, it's technically against RFCs to have both MX and CNAMEs for the same zone. I.e. if you have www.domain.com as a CNAME, you can't also have MX for www.domain.com.



So, there you have it. MX record resolution requires SOA lookups, which require their own A record resolution. MX records include A records typically, which require their own resolution by the SOA. CNAMEs shouldn't be used in the MX, and can't have their own MX, but they can still be used to route email in a pinch.

Try using nslookup or dig to do your own DNS recursion. Once you realize how many DNS queries each email requires, be sure to give your DNS server a big pat on the back for so many jobs well done.


-TEA

Email Feedback Report for IP

Really AOL, really?

Over the years I have seen remarkable feedback reports from AOL. Everything from obvious user error like newsletters, etc. to bills that the user did not want to receive (but were completely legit), to those "chain letters" telling you that you and your entire family will die painfully unless you forward the message to 200 of your closest friends.

Recently, I saw a feedback report for a read receipt. A READ RECEIPT. Someone sent an email, got the read receipt back and then reported it as spam. That part doesn't surprise me (nothing much does any more), but a provider that has been in business as long as AOL (remember when they "were" the WWW for many Americans?) not having processes in place to deal with a user error this remarkably dumb amazes me.

First, their entire concept of blacklisting IPs is so 1999. It's 2010, use a content scanner!

Second, relying on their users to create an automated IP blacklist for them misplaces the burden onto their customer base, rather than trained techs (who can tell the difference between spam and a read receipt) and simultaneously creates a worse experience for all of their other users when "spam button" happy users wreck the IPBL for everyone else.

Third, if you're going to have an IP blacklist, and it's going to automatically block based on user reports, AT LEAST try to filter out obvious user error. A READ RECEIPT! Their logs show the message leaving, and the message clearly states it's a read receipt, the headers even had the appropriate "notification" message type. AOL, do you really have to make it this easy to poke fun at you?

AOL, I would like to offer my services to you. I'll even do it for half my hourly rate. Just please, please, please revisit this horribly planned out mechanism and get the correct processes in place to handle your spam problems.

While you're at it, could you please fix your outbound spam problems too? How many of your customers are really in Nigeria that know dead rich people with my last name?


-TEA

What's an "MX Record"?

Unfortunately, MX records are one of the most misunderstood points of email transmission. Padawan mail admins have many misconceptions. Here are a few:

1. That the MX records have a specific naming convention like mail.domain.com or smtp.domain.com

2. That every server that handles email must be in the MX record

3. That the destination MTA must be in the MX record

4. That an "MX server" is another name for a mail server

5. That a "reverse DNS" lookup queries the sending domain's MX

Now that we know a few of the things an MX record is NOT, let's take a look at what it IS.

An MX record is simply an entry in a domains DNS zone. Typically they consist of one or more "A" type records (which then require resolution) with priorities assigned to them. Also, as with all DNS record a TTL value, or Time To Live, is assigned to them as a whole (and to each type A record as well).

They are simply references to the border MTA(s) responsible for relaying incoming email for a domain. When an email server is asked to email user@domain.com, it queries the DNS servers responsible for the domain (often by recursing to the root name servers looking for SOA records) and asks for the MX records. Typically, the MX records consist of type A records so then another query is given to find the IP addresses associated to them.

Let's take this story as an example. Let's say you're at a friends house discussing a long lost friend.

Friend: Hey I bumped into John last week at the mall
You: Really, I haven't seen him in years! Did you get his phone number?
Friend: Yes, but I don't remember it. You'll have to look in my phone.
You: Where's your phone? (recursive DNS query to the root NS)
Friend: Over on the table
You: What's his entry? (MX record query)
Friend: It's "John Doe"
You open the phone and look at "John Doe's" entry (type A record lookup) and call him on your phone. (transmission)

This simple story, that has probably never actually happened to any of us but should be able to be understood by most of us, is as simple as MX gets. Of course there's more to it (TTL values, "border MTAs", lots more DNS lookups, and any number of issues routing), but those are really more relegated to the realm of troubleshooting, and not something to worry our pretty little heads with right now :-)

Once it's been determined where to send the email, the transmission ensues and sooner or later after being relayed, archived, scanned for spam/viruses, and many more operations the email will eventually find it's home at the destination server and be picked up by the client to be read by the user.

Just think of all of the servers across the globe that played a part in getting your email to it's destination sometime when you're sending email. It really is amazing how it all comes together so seamlessly.

The next time you get the call from your user wondering why it took five seconds to get an email try explaining this to them. Boggle their mind a little bit ;-)


-TEA

Temporary and permanent errors

Typically, you'll only see two types of errors "Temporary" and "Permanent" errors. Standard codes can be found in RFC 2821 section 4.2.3, but there are many different servers that reply with alternate codes as well.

Temporary errors are expressed through 4.x.x error codes. These might be due to the recipient server being busy, the recipient server greylisting the sender, or closing the connection prematurely. The sending server can also generate 4.x.x errors if they can't resolve the recipient's DNS, the recipient's server is offline, the recipient server is connected to and responds but does not fully accept the email or any other issue that does not result in the recipient server accepting the email (with a 2.x.x status) or outright rejecting the message (with a 5.x.x message). Sometimes firewalls intercept the packets and mangle them or drop them, this can also cause a 4.x.x error since either the recipient's server does not receive the correct packets to accept the message and times out the connection eventually, or the sending server does not know that the recipient's server accepted the message and also times out the connection eventually.

While temporary errors occur due to transmission issues or issues that are expected to resolve themselves typically, Permanent errors mean that the server was connected to, delivered a response, and that that response was that it refused to accept the message... EVER.

Typical reasons for this are that the mailbox is full or nonexistent, the message was blocked as spam/virus, the mail server has vital errors or a policy violation such as non-local delivery for the domain, too many hops per message, or others. This error means that the server is online, listening for connections, accepted your connection but then actively refused to accept the message. It's the difference between sending someone you don't want to talk to to voicemail versus picking up the phone and telling them to stop calling. There is no confusion.

A sending server can also generate a permanent error as a result of timing out the message after repeated temporary errors, but this will typically be hours or days later, whereas a recipient server's rejection is almost instant usually. Obviously there will also be repeated 4.x.x errors in the logs to reference.

Understanding the difference between these errors is also important to determine where to troubleshoot first. A permanent error is almost always the recipient server, whereas a temporary error could be one of many things.

Senders side:

DNS issues (queries)
ISP/routing issues
Firewall issues (packet filtering)
Addressing issues

Recipient's side:

DNS issues (serving)
ISP/routing issues
Firewall issues
Greylisting
Rate Limiting
Hardware issues
NAT issues
and many more....


As always, the bounce itself will have a lot of good information, the "How to read a bounceback" post should be the first place to check if a NDR/DSN has been generated, but otherwise use of your troubleshooting tools in conjunction with the log data should provide plenty of information to determine the source of the issue.


TEA

Reading message headers

The routing of the email section is reasonably straightforward if you keep in mind that they are backwards.

1. Look for lines in the headers that say "Received: from".
2. Look for the last "Received: from" line and read upwards.

For instance, let’s assume we’re tracing an email with the following headers:

Received: from unknown (HELO BorderMTA.MyDomain.com) (9.10.11.12) by mail.MyDomain.com with SMTP; 13 Feb 2009 23:23:29 +0200
Received: from outbound.sendingdomain.com (outbound.sendingdomain.com [5.6.7.8]) by BorderMTA.MyDomain.com (8.14.1/8.14.1) with ESMTP id n1DLNSS7000915 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Fri, 13 Feb 2009 16:23:28 -0500
Received: from mail.sendingdomain.com (mail.sendingdomain.com [1.2.3.4]) by outbound.sendingdomain.com (8.14.1/8.14.1) with ESMTP id n1DLNQgX009122 for ; Fri, 13 Feb 2009 16:23:26 -0500
Received: from 10.0.0.100 ([10.0.0.100]) by mail.sendingdomain.com ([10.0.0.1]) with Microsoft Exchange Server HTTP-DAV ; Fri, 13 Feb 2009 21:23:21 +0000

When read from bottom to top says it went from 10.0.0.100 to mail.sendingdomain.com in the first line: “Received: from 10.0.0.100 ([10.0.0.100]) by mail.sendingdomain.com ([10.0.0.1])”

1. From mail.sendingdomain.com to outbound.sendingdomain.com in the second line: “Received: from mail.sendingdomain.com (mail.sendingdomain.com [1.2.3.4]) by outbound.sendingdomain.com “
2. From outbound.sendingdomain.com to BorderMTA.MyDomain.com in the third line: “Received: from outbound.sendingdomain.com (outbound.sendingdomain.com [5.6.7.8]) by BorderMTA.MyDomain.com”
3. And finally it's delivered from BorderMTA.MyDomain.com to mail.MyDomain.com in the last (topmost) line: “Received: from unknown (HELO BorderMTA.MyDomain.com) (9.10.11.12) by mail.MyDomain.com”

You can tell that the sender has an outbound email scanner (outbound.sendingdomain.com) after their mail server and that you have a border MTA before your main mail server.

Now that we know what we're reading, it's trivial to troubleshoot routing issues or delays that may not have been permanent errors or delayed enough to cause a DSN, but delayed nonetheless. Let's take the above headers and play with te timestamps a bit

Received: from unknown (HELO BorderMTA.MyDomain.com) (9.10.11.12) by mail.MyDomain.com with SMTP; 14 Feb 2009 00:53:47 +0200
Received: from outbound.sendingdomain.com (outbound.sendingdomain.com [5.6.7.8]) by BorderMTA.MyDomain.com (8.14.1/8.14.1) with ESMTP id n1DLNSS7000915 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Fri, 13 Feb 2009 18:53:13 -0500
Received: from mail.sendingdomain.com (mail.sendingdomain.com [1.2.3.4]) by outbound.sendingdomain.com (8.14.1/8.14.1) with ESMTP id n1DLNQgX009122 for ; Fri, 13 Feb 2009 16:23:26 -0500
Received: from 10.0.0.100 ([10.0.0.100]) by mail.sendingdomain.com ([10.0.0.1]) with Microsoft Exchange Server HTTP-DAV ; Fri, 13 Feb 2009 21:23:21 +0000

Can you find the delay? I included several time zone changes, so don't get duped by them ;-)

If you said it's from outbound.sendingdomain.com to BorderMTA.MyDomain.com you're correct!

It took about 2.5 hours for outbound.sendingdomain.com to successfully deliver to BorderMTA.MyDomain.com. Now that we've got the location of the delay we know where to start looking for the issue. Although the headers simply show where the delay occurred, not the actual cause of the delay, you can save a lot of time just quickly looking at them and isolating the possible trouble areas.

You might have noticed that each intermediary server is listed twice in the headers. When the server receives the message it says “by some.server.com” and then when it sends it on it says “Received: from some.server.com” in the next line. Often this is enough to tell where the delay occurred. If a server accepted the message and didn’t deliver it for many minutes, hours, days, etc. it frequently indicates that the recipient’s mail server was unavailable, deferring connections or “greylisting” the sending server. In that case, the logs you’ll want to read are on the recipient’s mail server (or possibly their firewall especially if it’s a connection issue or if it has a MTA built in). If this server does not have pertinent log files, then contact the sending server’s administrator to see if their logs show any additional information.

Sometimes the sending server may have issues sending an email that are outside of the recipient mail server’s control. A few examples would be retrieving the DNS information of the domain or recipient server, issues with connectivity or specific routing due to ISP issues, or any number of other issues. In these situations you’ll have to contact the sending server to determine the issue. Because the message never left the sending server, there will not be any logs on the recipient mail server. This is often the best place to start when troubleshooting home built or custom email applications such as bulk mailing software/vendors, website forms etc.

In general, if there's a delay you can start looking at the recipient server logs first. These will be the most verbose if in fact the issue is there. If the recipient server is a "border MTA" or another server that is responsible for all of your incoming email, and you're not experiencing issues across the board, you'll probably want to check the sending server first. If there was an issue on the border MTA it's likely (unless it's a rate limit/spam filter/etc issue) that the issue is on the sender's side. Either way, the issue must be on one of these two servers, so there's no reason to troubleshoot elsewhere unless you see additional delays in the headers (which is unlikely).

In summary, The headers go in order from bottom to top. They should all have timestamps, although they may be in different time zones, or the server clocks could be off. If you have a delay reported to you, isolate the servers that were responible for the transmission at that moment, determine what services are running on the servers (anti spam, rate limits, etc) and which is more likely to relate to the issue at hand (if the issue is all of your email, it might be your border MTA, if it's only this sender then it's likely on their end) then read the logs on this server. Once you have the error in hand from the logs, Google it. You're not the first admin to run into this issue, and most often someone has posted it to a forum, etc and gotten an answer.



TEA

Forwarding to users home addresses

In today's busy information age more people are connected to their office email at all times than ever before. Smart phones, webmail or even VPN connections to their office network are more and more common.

One unfortunately common idea to help facilitate connectivity to work email during vacations or for off site users is to set up forwarding rules. So, if I'm on vacation (like that'll ever happen ;-) ) you could email me at my email address NotA.RealAddress1@theemailauthority.com (that's not a real address, so spammers can't harvest it) and I'd just set that address to forward to MyHome.Address2@PersonalAddress.com. Then when I check my home email, I'll be able to see the work I have to respond to as well.

Seems like a very convenient way to give people access to their email while on vacation or for offsite users in general, huh? Think again.

If I had a penny for all the reasons I'm about to list I'd have, well a few more cents, but you know what I mean.

I. Habituation

Social Engineering is often used in hacker terminology to mean gaining access to restricted material by direct human contact. Although almost the exact opposite of this usage, it's still the admin's responsibility to "engineer" their end users into acceptable behavior. It's not only our job to facilitate technological requests, it's our job to explain why some things shouldn't be accomplished a certain way, if at all. If a user said "this whole logging in thing, it's a pain to deal with. Let's get rid of it" you'd look at them as if they had two heads. Similarly, there are times when we must take an almost mentor type role and explain the repercussions of user's decisions and guide them to become used to a more sound solution or adapt the solution in a way that's secure and doesn't create even more issues down the road.

Here are some examples of how users can create issues with email forwarding.

1. Reply/Reply All

OK, so a client's email gets sent to HomeUser@gmail.com or whatever. Now, if your user has to respond, where does the email come from? Their home address, that's where. This causes confusion to the sender since now they have a new address, and it can take weeks for the senders to stop just clicking "reply" or finding the old thread and "replying all".

2. Address books

In addition to the threads going back and forth to the user's home address, how many times do your users bother to look at the actual email address they're sending to? Often they rely on their address book entry, which can now be compromised with the new entry for "John Doe", but "John Doe's" home address. This can indefinitely prolong the above "replying all" issue as each new thread will be sent to whatever address "John Doe" has in the remote address book.

3. Multiple mailboxes

Skip forward a few days. Now the user is back in the office and looking at their normal email client. Where are the messages? In their home email, that's where. Now they have two places to check, again sometimes for weeks until the thread dies, offering the possibility of lost email, delayed responses, or any number of other issues that make your company look bad.

4. Misplaced blame

Ok, so your user receives a spam message (or a message they think is spam). They click the "this is spam" button in their gmail/hotmail/etc. If the message is being forwarded from your server, their home email provider often simply sees the connecting IP which if you're forwarding the email from your mail server is now you. Now the user's home provider views you as a spammer, and the legitimate emails you're trying to get to your user get caught in the spam trap as well as the spam. I've seen AOL for instance do this several times causing admins no end to headache (try getting a decent AOL support tech on the phone to fix a problem, I dare you).


II. Technology issues

In addition to the user based issues above, there are obvious technical issues as well.

1. Routing of complaints

Although mentioned above, I cannot stress this enough. If you do this with the average user, sooner or later you take a strong risk of getting blacklisted. Depending on a few factors this can have tremendous impact on your infrastructure. If your user is sending spam reports to their provider, and you have many other customers on the same provider, your email will end up not only in your user's spam box, but any other customer who uses the same provider. For instance, if you forward to AOL, and your user gets you blacklisted by AOL then you try to send a legitimate piece of email to another AOL user, you will not get it through. I simply cannot stress this enough.

2. Bad practices

Anyone remember the post about the "border MTA's" role? If not, click that link and read it right now. Now you've got your email passing through your border MTA with all of it's antispam, antivirus, DDOS protection, etc, then you're passing it to ANOTHER border MTA to handle the same things again. You can see above for one instance of how this can cause headaches, but there are so many more. Rate limiting, spam complaints, automated processes, greylisting, availability, the list goes on and on.

3. Backscatter

I'll have a whole post at some point on this topic if you're not familiar with the term. In the meantime you can click here for more info. Including multiple border MTAs will almost always result in some type of backscatter. There are RBLs that focus exclusively on the source of backscatter, and many include backscatter sources at least incidentally in their lists. If you've ever tried to get off of some of these RBLs you'll probably agree you don't want to do it again anytime soon. Additionally, I guarantee if you've ever been the victim of a spoofing attack you'll understand how frustrating backscatter can be, and you'll never want to inflict it on anyone.

4. Archiving and other requirements

Many companies are required by law to archive email sent on the company's behalf. How are you going to archive messages that were sent from the user's gmail/hotmail/yahoo account? How are you going to archive messages that were replied back to those addresses? Is it more or less likely that someone on vacation would respond in an unfavorable (and therefore liable) manner when they're sucking down Pina Coladas on a beach somewhere than when they're in the office? Don't you think these messages above all others should be archived?

5. "too many hops"

Many mail servers have rules in place for how many other servers can be listed in the headers. If there are too many, you can generate bounces that claim there were "too many hops". Since this is entirely out of your control once it leaves your network it's something to take very seriously unless you enjoy talking to other support reps all day while troubleshooting this error. Although a couple additional hops should be fine, you have no idea how badly mangled your user's home email routes are. Maybe they've had the same email address for years, but they tried Hotmail first, then used AOL, then gmail. If they set forwarding rules (which are often the home user's only option outside of changing their email address) you'll have several hops per provider, not including yours. You can see how this would quickly add up. Also, there are outbound routes too, so it could be that some of your senders place several hops before it even reaches your network. Therefore any testing you do would be moot, and those senders will still get "too many hops" errors.

III. Easy solutions

So, your user needs remote access. This is an understandable need. There are literally too many options to mention, but here are a few

1. Give them access.

Look, we have to be security conscious. There are simply too many threats out there to leave your firewall wide open for all email traffic. However, there are plenty of ways to mitigate the risk without making your users' lives harder. If you don't want to open POP/IMAP/SMTP ports, dont. But use other ports and NAT them to the correct ports on your server. Of course this doesn't stop port scans, but it will lessen the number of available threats to the point you can sleep most nights ;-)

2. Require encryption

If you feel safe logging in to your bank account via the web, then you trust SSL. Require SSL/TLS connections at least from your outside users. This will both change the default port (which you should again change), and also prevent snoops from intercepting potentially sensitive information. Sure, they could have a keylogger, etc at home, but they might in your organization too.

3. Control the hardware

Provide them with a laptop if possible that contains all of the security software you use at the office and require them to return it when they're back home. This way you know at least as of the day they left they have up-to-date virus definitions, patches, and all of the other things users forget to do at home, but it's your job to do at work.

4. Use webmail, or get a vendor

Quite simply, webmail is a must. If you don't want to offer the webmail service directly on your server, offer it through a vendor and grant them access to your server. Zimbra for instance allows you to sync multiple mailboxes via IMAP in their webmail interface. as long as you grant access from the vendor to your IMAP port they can interface with their email just fine. Zimbra also offers the option of smart hosting to your mail server for outbound communication to help with your archiving/discovery software.

5. Give them a smart phone

If it's a power user who is always on the go, give them a smart phone. Personally, I prefer typing emails on many smart phones. Maybe it's all the years of gaming, but I actually find it more convenient to type on some of the keyboards with my thumbs than on my desktop with my whole hands/wrists/etc. Then, depending on the method of delivery to/from your mail server you can enforce policies to only allow connections from the specific devices, or their carrier will often offer (in the case of Blackberries) an option for them to deliver messages for you. Just be sure to allow the carrier to send from your domain in your SPF record.

To summarize, there are just too many simple solutions that offer more security and audit trails, as well as much less risk of headache. If you're an end user who has to deal with whatever your provider gives you, feel free to use email forwarders. If you're an admin you have the power to handle these issues the "right way", or really in any of the "right ways" you see fit. Remember the K.I.S.S. principle applies to your email's path, not your setup. Think of the slight inconvenience of doing it right as an investment on all the time you won't have to spend troubleshooting later.


TEA