There's a new spam in the wild claiming to be bounce messages to messages that you've sent, but that actually send you to a meds spammer's site. They've been mutating for several days, so the body and attachment change fairly rapidly, but here's what one looks like as of the date of this post:
--- begin spam ---
Delivery Status Notification (Failure) Note: Forwarded message is attached.
This is an automatically generated Delivery Status Notification
THIS IS A WARNING MESSAGE ONLY.
Delivery to the following recipient has been delayed:
user@domain.com
Message will be retried for 2 more day(s)
---end spam---
Then there is an HTML attachment which redirects to the spammer's web site.
It's really quite ingenious really. Who wouldn't open a bounce, even if they didn't remember sending the original email, or maybe especially if they didn't remember sending it?
Another one, this time stating that it's from "mailfilter@yourdomain.com":
---another spam---
Note: Forwarded message is attached.
An email you sent could not be delivered.
Subject: Hello
Delivery to the following address has failed...
user@RecipientDomain.com
Technical information about this permanent failure:
-- Transcript of session follows --
While talking to [some.server.com] :
>>> rcpt to: user@RecipientDomain.com
<<<>: Recipient address rejected: User unknown
---end spam---
Users are so accustomed to worrying thet they're "infected" every time the get a bounce (when in fact, often it's just "spoofing") that these fake spam bounces must have a huge click through rate.
Although it's difficult to block "backscatter", I have to imagine that these are relatively easy to block. All the HTML attachment includes is a refresh to their site, any content based scanner should pick them up pretty easily.
As for the user's perspective, the old rules (slightly modified) still pertain here.
If you do not know the sender, do not open the attachment.
With these new "bounce message" spams, you can modify the above rule to:
If you did not send the email, do not open any attachments.
If you DO know the sender, and it's not a file you requested, call them to see if they actually sent it. Especially if it's in the following formats: .exe, .pif, .zip, .rar, .bat, .pdf, .htm, .html.
Of course, these spam messages have .htm file attachments, so the above pertains to them as well. Although these are a simple spam, websites can be infected and spread malware through your browser. Always exercise extreme caution when opening .htm files.
-TEA
Thursday, July 8, 2010
Spoofing
"Spoofing" someone's email address is really hard, right? Only the most elite hackers can do it, right? It's not something that I can try at home, right?
no, no and no.
In the post "Testing SMTP" we discussed all of the commands necessary to fully deliver an email. Nothing special, no encryption, just a plain text email (without MIME-types, etc), but an email nonetheless. Below is the list of commands again in case you missed that post:
---------------------------------------------------------------------------
HELO your hostname
MAIL FROM:
RCPT TO:
DATA
FROM:
TO:
SUBJECT: test
this is a test.
.
----------------------------------------------------------------
You'll notice something is missing in there. Something important for security. Something you do to many websites, to view your email, and often the first thing when you turn on your computer.
Don't scroll down, take a guess :-)
There's no login/password!
You could send yourself an email from GeorgeWashington@whitehouse.gov, or Bill@Microsoft.com saying whatever you like. Of course, you're still incredibly traceable, so don't get any bright ideas about sending an email "from" your boss giving yourself a raise or anything.
This is one way though that spammers and viruses hide their tracks or create a false sense of trust from their victim. SPF was implemented as a direct result of this flaw in SMTP's security.
Many times, spammers will "spoof" other users in their database of email addresses. Spam/virus filtering after the "Border-MTA" results then in backscatter, or bounces from email that was not sent by you. This is of course different than the article earlier about fake bounce message spam. Backscatter is legitimate bounces, but sent in response to an illegitimate use of the victim's email address.
Sometimes backscatter can inundate an inbox so thoroughly that it's hard to get any work done. In cases like this, ALL bounce messages should be blocked or moved to a folder keeping them out of the inbox. A common way to do this is to make a rule for these senders:
<>
postmaster@
mailer-daemon@
And that includes the following phrases:
"hi, this is the qmail send program"
"returned mail: see transcript for details"
"undeliverable mail returned to sender"
"delivery status notification (failure)"
"delivery status notification (delay)"
"out of office autoreply"
and any others that come in.
One of the reasons to not use a challenge-response anti-spam service is that they accept all email for your domain, then deliver "challenges" to which the sender must "respond". Since they accept the email, many deliver challenges to innocent users who are simply being abused by a spammer. Therefore you'll also want to include challenge response vendors in your filter.
Unfortunately, the lack of security in SMTP has been abused to no end. That's why it's imperative for your reputation, and a good "good neighbor" policy to implement SPF records. Although they're a little hard to grasp at first, They're the best way to be sure that nobody is using your email address without your permission. Unless you've been hacked, but that's a whole other post....
-TEA
no, no and no.
In the post "Testing SMTP" we discussed all of the commands necessary to fully deliver an email. Nothing special, no encryption, just a plain text email (without MIME-types, etc), but an email nonetheless. Below is the list of commands again in case you missed that post:
---------------------------------------------------------------------------
HELO your hostname
SUBJECT: test
this is a test.
.
----------------------------------------------------------------
You'll notice something is missing in there. Something important for security. Something you do to many websites, to view your email, and often the first thing when you turn on your computer.
Don't scroll down, take a guess :-)
There's no login/password!
You could send yourself an email from GeorgeWashington@whitehouse.gov, or Bill@Microsoft.com saying whatever you like. Of course, you're still incredibly traceable, so don't get any bright ideas about sending an email "from" your boss giving yourself a raise or anything.
This is one way though that spammers and viruses hide their tracks or create a false sense of trust from their victim. SPF was implemented as a direct result of this flaw in SMTP's security.
Many times, spammers will "spoof" other users in their database of email addresses. Spam/virus filtering after the "Border-MTA" results then in backscatter, or bounces from email that was not sent by you. This is of course different than the article earlier about fake bounce message spam. Backscatter is legitimate bounces, but sent in response to an illegitimate use of the victim's email address.
Sometimes backscatter can inundate an inbox so thoroughly that it's hard to get any work done. In cases like this, ALL bounce messages should be blocked or moved to a folder keeping them out of the inbox. A common way to do this is to make a rule for these senders:
<>
postmaster@
mailer-daemon@
And that includes the following phrases:
"hi, this is the qmail send program"
"returned mail: see transcript for details"
"undeliverable mail returned to sender"
"delivery status notification (failure)"
"delivery status notification (delay)"
"out of office autoreply"
and any others that come in.
One of the reasons to not use a challenge-response anti-spam service is that they accept all email for your domain, then deliver "challenges" to which the sender must "respond". Since they accept the email, many deliver challenges to innocent users who are simply being abused by a spammer. Therefore you'll also want to include challenge response vendors in your filter.
Unfortunately, the lack of security in SMTP has been abused to no end. That's why it's imperative for your reputation, and a good "good neighbor" policy to implement SPF records. Although they're a little hard to grasp at first, They're the best way to be sure that nobody is using your email address without your permission. Unless you've been hacked, but that's a whole other post....
-TEA
Sunday, July 4, 2010
Email and the OSI model.
All People Seem To Need Data Processing, and when it comes to networking, understanding the OSI model is imperative. However you remember this seven layer schema, it's vital when troubleshooting difficult issues to remember them. Use these handy dandy mnemonics to help:
Please Do Not Throw Sausage Pizza Away (layer 1-7)
All People Seem To Need Data Processing (backwards, from layer 7-1)
Please Do Not Trust Sales People Always (1-7 again)
More information can be found in A+ and Network+ cert books, and other study guides I'm sure. Wikipedia has more information here too.
It's important to understand that although rare, your server logs can lie. Your mail server runs on the Application layer, so it only "sees" what that layer sees. Although of course it relies on all layers on down to the Physical layer, one single server may have several daemons running on different layers intercepting the data at different points along the way.
For instance, on a *nix box, you may have Sendmail logging that it's connecting to mail.domain.com [1.2.3.4].
Sendmail made a call to the DNS resolver and pulled the MX records. Again it called the resolver to resolve the A records listed in the MX. Then it tries to connect, but IPtables has a static route for 1.2.3.4 to send the data to 5.6.7.8, which is now refusing the connection.
Since Sendmail is running on the application layer, it doesn't log (or even see) when IP it's ACTUALLY connecting to, only where it *should* be connecting to. In order to see the packets you're going to have to do some traffic dumps using tcpdump, tcpick or Wireshark.
Of course, this is a very unique situation, either intentionally done by a sophisticated admin as a workaround or as a temporary "fix" for DNS issues, etc. Either way it's a kludge, and this is why each server should keep a changelog for later reference including the exact lines entered for later grep-ability (yes, I just made that word up :-) )
Also, there obviously could be many other issues with routing or other types of packet interference from router/firewalls locally to ISP related routing issues, but this was one instance that boggled me for several hours, so it stands out as an excellent reference.
-TEA
Please Do Not Throw Sausage Pizza Away (layer 1-7)
All People Seem To Need Data Processing (backwards, from layer 7-1)
Please Do Not Trust Sales People Always (1-7 again)
More information can be found in A+ and Network+ cert books, and other study guides I'm sure. Wikipedia has more information here too.
It's important to understand that although rare, your server logs can lie. Your mail server runs on the Application layer, so it only "sees" what that layer sees. Although of course it relies on all layers on down to the Physical layer, one single server may have several daemons running on different layers intercepting the data at different points along the way.
For instance, on a *nix box, you may have Sendmail logging that it's connecting to mail.domain.com [1.2.3.4].
Sendmail made a call to the DNS resolver and pulled the MX records. Again it called the resolver to resolve the A records listed in the MX. Then it tries to connect, but IPtables has a static route for 1.2.3.4 to send the data to 5.6.7.8, which is now refusing the connection.
Since Sendmail is running on the application layer, it doesn't log (or even see) when IP it's ACTUALLY connecting to, only where it *should* be connecting to. In order to see the packets you're going to have to do some traffic dumps using tcpdump, tcpick or Wireshark.
Of course, this is a very unique situation, either intentionally done by a sophisticated admin as a workaround or as a temporary "fix" for DNS issues, etc. Either way it's a kludge, and this is why each server should keep a changelog for later reference including the exact lines entered for later grep-ability (yes, I just made that word up :-) )
Also, there obviously could be many other issues with routing or other types of packet interference from router/firewalls locally to ISP related routing issues, but this was one instance that boggled me for several hours, so it stands out as an excellent reference.
-TEA
Monday, June 21, 2010
Testing SMTP
Sometimes it's important to be able to manually reproduce what your computer/server is doing manually so you can see exactly where it's failing. Troubleshooting "utilities" are great in that they can often diagnose hundreds of issues in less time than it would take you to troubleshoot one, but they lack the nuance sometimes necessary to accurately diagnose issues.
A simple pass/fail is nice, but if you don't know WHY it's failing, what good is it?
When troubleshooting email, there are websites, utilities, diagnostic messages and more designed to help figure out what's going wrong. I have never found a single one that has been more informative than a simple SMTP session.
If you have telnet on your computer, that'll work fine. Netcat or any other command line tool that you can specify the ports to connect on will work too.
On your command/shell prompt type: telnet <the server's IP> 25
Then, when prompted, type the following commands:
---------------------------------------------------------------------------
HELO your hostname
<wait for response>
MAIL FROM: <your email address>
RCPT TO:
<wait for response>
DATA
FROM:
TO:
SUBJECT: test
this is a test.
.
----------------------------------------------------------------
Note the additional CR/LF after the subject. That separates the headers from the body (visible part) of the message. It's very important.
Also important is the line with a "." by itself. This tells the recipient's server that the message is complete and to process it.
On to troubleshooting....
In each part above where it says "wait for a response" you're expecing a response from the server, either something like "ok, go ahead", an error, or sometimes nothing. Here are some examples:
HELO
If there's no response, this can indicate that the server is extremely busy or that sometimes there's packet filtration between you and the server that's intercepting and silently dropping the packets. Either way, if there's no response, there's no transmission, and therefore no email.
Sometimes you'll get an error here referencing that it does not like your IP (if you're blacklisted), or that it's imposing some type of lookup on your HELO argument. It's best to be sure thet you give your FQDN here to prevent issues.
MAIL FROM:
You might get SPF errors here if your SPF record is wrong, or if your email address is blacklisted you might get a response here.
RCPT TO:
"Relaying denied", "mailbox is full", or any number of other recipient level issues may result as a response to this line. Any server that has different rules per user or domain (like a shared spam filter with different settings per domain/user) might also process here.
DATA
You're now giving the actual data that will be shown to the user. Generally there won't be an issue here until you close the transmission channel (with the "." on a line by itself), but anywhere you wait for a response is a potential place for errors.
"." (or closing the transmission channel)
This is where the email in it's entirety can be scanned and processed. Sometimes nothing will be scanned until this point, so even though SPF *could* be looked up after the "mail from:" command, it *might* not be until the entire message has been transmitted. Any of the above errors can happen here as well as content based scanning of the body, etc.
Some mail servers will also accept the email here, and then scan it later before delivery. That's a horrible idea, but that doesn't stop anyone usually, so be sure to copy down any information the acceptance line gives you like the message ID, so you can reference it later if the test does not complete.
Generally, you'll only be able to run this test on a border MTA unless it's your own mail server or the recipient's firewall allows you through. This can definitely complicate troubleshooting, but if you can prove that the issue is with the border MTA, or that the border MTA accepts the email but it is not relayed it at least absolves you from any finger pointing.
Once you have these commands memorized (it doesn't take long), you'll often be able to troubleshoot and diagnose issues faster than submitting tickets to your vendors. There's nothing more frustrating than either waiting for a response only to find out you could have fixed it hours ago, or conversely that you know exactly where the error occurred, but the other admin requires "proof" that they should bother fixing the issue when the DSN or other logs obviously state the issue.
Send a couple SMTP sessions their way and copy the session to them. It's hard to argue with facts :-)
-TEA
A simple pass/fail is nice, but if you don't know WHY it's failing, what good is it?
When troubleshooting email, there are websites, utilities, diagnostic messages and more designed to help figure out what's going wrong. I have never found a single one that has been more informative than a simple SMTP session.
If you have telnet on your computer, that'll work fine. Netcat or any other command line tool that you can specify the ports to connect on will work too.
On your command/shell prompt type: telnet <the server's IP>
Then, when prompted, type the following commands:
---------------------------------------------------------------------------
HELO your hostname
<wait for response>
this is a test.
.
----------------------------------------------------------------
Note the additional CR/LF after the subject. That separates the headers from the body (visible part) of the message. It's very important.
Also important is the line with a "." by itself. This tells the recipient's server that the message is complete and to process it.
On to troubleshooting....
In each part above where it says "wait for a response" you're expecing a response from the server, either something like "ok, go ahead", an error, or sometimes nothing. Here are some examples:
HELO
If there's no response, this can indicate that the server is extremely busy or that sometimes there's packet filtration between you and the server that's intercepting and silently dropping the packets. Either way, if there's no response, there's no transmission, and therefore no email.
Sometimes you'll get an error here referencing that it does not like your IP (if you're blacklisted), or that it's imposing some type of lookup on your HELO argument. It's best to be sure thet you give your FQDN here to prevent issues.
MAIL FROM:
You might get SPF errors here if your SPF record is wrong, or if your email address is blacklisted you might get a response here.
RCPT TO:
"Relaying denied", "mailbox is full", or any number of other recipient level issues may result as a response to this line. Any server that has different rules per user or domain (like a shared spam filter with different settings per domain/user) might also process here.
DATA
You're now giving the actual data that will be shown to the user. Generally there won't be an issue here until you close the transmission channel (with the "." on a line by itself), but anywhere you wait for a response is a potential place for errors.
"." (or closing the transmission channel)
This is where the email in it's entirety can be scanned and processed. Sometimes nothing will be scanned until this point, so even though SPF *could* be looked up after the "mail from:" command, it *might* not be until the entire message has been transmitted. Any of the above errors can happen here as well as content based scanning of the body, etc.
Some mail servers will also accept the email here, and then scan it later before delivery. That's a horrible idea, but that doesn't stop anyone usually, so be sure to copy down any information the acceptance line gives you like the message ID, so you can reference it later if the test does not complete.
Generally, you'll only be able to run this test on a border MTA unless it's your own mail server or the recipient's firewall allows you through. This can definitely complicate troubleshooting, but if you can prove that the issue is with the border MTA, or that the border MTA accepts the email but it is not relayed it at least absolves you from any finger pointing.
Once you have these commands memorized (it doesn't take long), you'll often be able to troubleshoot and diagnose issues faster than submitting tickets to your vendors. There's nothing more frustrating than either waiting for a response only to find out you could have fixed it hours ago, or conversely that you know exactly where the error occurred, but the other admin requires "proof" that they should bother fixing the issue when the DSN or other logs obviously state the issue.
Send a couple SMTP sessions their way and copy the session to them. It's hard to argue with facts :-)
-TEA
Sunday, June 20, 2010
subdomain MX records
We've all used subdomains. The popular naming convention "www.domain.com" is in itself a subdomain (the subdomain "www" in the "domain.com" domain), but did you know each subdomain can also have it's own MX records?
The obvious questions are "why would anyone do that" and "how would anyone do that?". Here's some food for thought.
Way back in the day, things were simpler. Email specifically didn't require tons of MX and A record lookups, there weren't even DNS servers to query let alone DNS record types. Users had accounts on their local servers, and the servers had names that although understandable would be different from organization to organization. Email addresses weren't "bob@company.com", they were "bob@accounting.company.com" for Bob in accounting and "bob@engineering.company.com" for Bob the engineer. Each organization would have their own server names, and although often they'd have meaning within the organization they might be goofy to outsiders. Also, desktop PCs were uncommon then, so you would connect to the server for almost all of your daily tasks. Therefore, rocky@bullwinkle.company.com was literally addressing the email to the user account "rocky" on the server named bullwinkle.company.com. Some places still conform to these odd naming conventions. I knew someone who went to the University of Michigan's Engineering school who had an email address @engin.umich.edu. It took me months to figure out that "engin" was for " school of ENGINeering".
Nowadays it's slightly different. "company.com" generally will have all of it's users email addresses directly @company.com, but not always. For instance, let's say there's a domain BigCompany.com. BigCompany.com is all over the world, and they want their customers to know that they're dealing with their "local" office. So, they may have John@USA.BigCompany.com and John@Canada.BigCompany.com, but they're totally different Johns, in totally different regions.
Let's say they not only want to have email addresses at the country level, but also at the state level. They could also have John@California.USA.BigCompany.com and John@NewYork.USA.BigCompany.com.
Now, most companies wouldn't do this because they don't want to confuse their customers by having them talk to the wrong John all the time, but you commonly see this with schools for instance. They might be something like "Principal@SchoolName.NY.k12.us".
Ok, now we know why someone would have subdomain MX records. Either to conform to a legacy naming convention or to help routing between different regions/offices. Now the question is "how do we do that"? The answer shouldn't be too surprising, we make MX records for each subdomain.
Just like each domain name can have MX records (and should have them if email is addressed to it), subdomains can get in on the fun too with their own set of records. Let's say you have an email address webmaster@www.domain.com, but you don't want to run a mail server on the same box as your web server (which is a good idea btw!). You can simply have MX records for www.domain.com go to the same server as your regular domain like this:
Host Priority Record
domain.com 10 mail.domain.com
www.domain.com 10 mail.domain.com
Now, you may have to create a separate zone for this subdomain if you're running your own DNS server, or if you're using some provider's interface they may require additional steps to configure this but most of the time it's possible. Some services don't allow subdomain records to be made if you're using their DNS, but DNS hosting is cheap. If they don't allow this, and they have ridiculously bad support who won't discuss the finer side of RFCs with you (cough, yahoo.com, cough), just change your DNS host. Outside of this issue, you're likely to find other reasons to switch providers later if you're not already disgusted with their lack of support.
It's important to keep in mind, that this is not something you can do after the fact. Once your users have you in their address book as user@domain.com, they will ALWAYS query domain.com's MX records regardless of whether you add an alias for user@sub.domain.com unless of course you choose to change your email address and deal with all of the headaches associated with that.
It also should not be used after the border MTA using address rewriting without serious contemplation. Although there are some excellent reasons to have your border MTA rewrite the delivery addresses to "user@sub.domain.com" while the rest of the world still emails the user@domain.com" address, there are serious considerations to take into account at the very least the reply-to address, or sender rewriting back to user@domain.com on the way out.
This is not likely something that many admins have dealt with before, and could be conceptually confusing at first. Don't worry, it's nothing different than what you do on the main domain, it's just a second place to deal with. User@domain.com queries domain.com's MX and sends them there. User@Sub.Domain.com queries sub.domain.com's MX and sends them there. It's pretty simple actually once you get your head around it. :-)
-TEA
The obvious questions are "why would anyone do that" and "how would anyone do that?". Here's some food for thought.
Way back in the day, things were simpler. Email specifically didn't require tons of MX and A record lookups, there weren't even DNS servers to query let alone DNS record types. Users had accounts on their local servers, and the servers had names that although understandable would be different from organization to organization. Email addresses weren't "bob@company.com", they were "bob@accounting.company.com" for Bob in accounting and "bob@engineering.company.com" for Bob the engineer. Each organization would have their own server names, and although often they'd have meaning within the organization they might be goofy to outsiders. Also, desktop PCs were uncommon then, so you would connect to the server for almost all of your daily tasks. Therefore, rocky@bullwinkle.company.com was literally addressing the email to the user account "rocky" on the server named bullwinkle.company.com. Some places still conform to these odd naming conventions. I knew someone who went to the University of Michigan's Engineering school who had an email address @engin.umich.edu. It took me months to figure out that "engin" was for " school of ENGINeering".
Nowadays it's slightly different. "company.com" generally will have all of it's users email addresses directly @company.com, but not always. For instance, let's say there's a domain BigCompany.com. BigCompany.com is all over the world, and they want their customers to know that they're dealing with their "local" office. So, they may have John@USA.BigCompany.com and John@Canada.BigCompany.com, but they're totally different Johns, in totally different regions.
Let's say they not only want to have email addresses at the country level, but also at the state level. They could also have John@California.USA.BigCompany.com and John@NewYork.USA.BigCompany.com.
Now, most companies wouldn't do this because they don't want to confuse their customers by having them talk to the wrong John all the time, but you commonly see this with schools for instance. They might be something like "Principal@SchoolName.NY.k12.us".
Ok, now we know why someone would have subdomain MX records. Either to conform to a legacy naming convention or to help routing between different regions/offices. Now the question is "how do we do that"? The answer shouldn't be too surprising, we make MX records for each subdomain.
Just like each domain name can have MX records (and should have them if email is addressed to it), subdomains can get in on the fun too with their own set of records. Let's say you have an email address webmaster@www.domain.com, but you don't want to run a mail server on the same box as your web server (which is a good idea btw!). You can simply have MX records for www.domain.com go to the same server as your regular domain like this:
Host Priority Record
domain.com 10 mail.domain.com
www.domain.com 10 mail.domain.com
Now, you may have to create a separate zone for this subdomain if you're running your own DNS server, or if you're using some provider's interface they may require additional steps to configure this but most of the time it's possible. Some services don't allow subdomain records to be made if you're using their DNS, but DNS hosting is cheap. If they don't allow this, and they have ridiculously bad support who won't discuss the finer side of RFCs with you (cough, yahoo.com, cough), just change your DNS host. Outside of this issue, you're likely to find other reasons to switch providers later if you're not already disgusted with their lack of support.
It's important to keep in mind, that this is not something you can do after the fact. Once your users have you in their address book as user@domain.com, they will ALWAYS query domain.com's MX records regardless of whether you add an alias for user@sub.domain.com unless of course you choose to change your email address and deal with all of the headaches associated with that.
It also should not be used after the border MTA using address rewriting without serious contemplation. Although there are some excellent reasons to have your border MTA rewrite the delivery addresses to "user@sub.domain.com" while the rest of the world still emails the user@domain.com" address, there are serious considerations to take into account at the very least the reply-to address, or sender rewriting back to user@domain.com on the way out.
This is not likely something that many admins have dealt with before, and could be conceptually confusing at first. Don't worry, it's nothing different than what you do on the main domain, it's just a second place to deal with. User@domain.com queries domain.com's MX and sends them there. User@Sub.Domain.com queries sub.domain.com's MX and sends them there. It's pretty simple actually once you get your head around it. :-)
-TEA
DNS doesn't end at MX you know...
Ok, so you understand MX records, you're ready to take on the world, right?
Not quite yet. Patience young grasshopper.
So, you're sending email to user@domain.com, you queried dns1.domain.com for the MX records and now you've got these MX records:
Host Priority Record
domain.com 10 mail1.domain.com
domain.com 20 mail2.domain.com
domain.com 30 mail3.domain.com
Ummm, so, computers don't speak English, they don't know what a "mail1.domain.com" is. They want digits that can turn into nice 1s and 0s that they can compute . What's a poor mail server to do with this stuff?
While we're at it, how did we even know to talk to dns1.domain.com to get these records?
Also, once we knew to talk to dns1.domain.com, we're back to the first question of how does the mail server know WTH a "dns1.domain.com" is, and where are the 1s and 0s it needs?
I is so confus :-(
Don't worry, there's more to an email's life than just MX records. There's lots of types of records including type A, type SOA, and type CNAME.
I. Type A records
Type A records ("A" for "Address") turn "mail1.domain.com" into a nice pretty number (the "IP Address" or "Internet Protocol address") for our mail server (and all other computers) to compute. The A record might look like this:
Mail1.domain.com IN A 10.0.100.28
Now we can see that the first MX record for domain.com has a name of "mail1.domain.com", but that name actually resides at 10.0.100.28. This number means nothing to most of us silly humans, but to a computer it's a nice string of digits that it can compute and then connect to.
TCP/IP and all of it's routing mechanisms are beyond the scope of this article, so you'll just have to make do with "once you have the IP you can connect to the server". :-P pbbt!
II. SOA records
OK, so this is a little backwards since you need the SOA records before you can get the MX records or A records, but you should already understand MX records, and you need to understand A records before you can get SOA records.
SOA records are the "Start Of Authority" records. There are literally millions of DNS servers in the world. How do you know which ones are responsible for the records you're querying? Well, that's where SOA records come into play. SOA records are given by the "root" name servers to offload queries to other DNS servers, and to provide a distributed architecture that is easier to maintain and provides more fault tolerance.
Root name servers are the biggest and baddest name servers in the world. There are two sets of them for your .com, .net, and the like, one at root-servers.net and one at gtld-servers.net. If queried, they will either reference the other one (like a father who says "go ask your mother", or vice-versa) or they will give the DNS server registered with them as authoritative.
You can play with this yourself using nslookup on Windows or dig on *nix.
Once the root servers give the name of the DNS server, that server's A record must be resolved then it can be queried for the MX, A, CNAME, or any other type of records you wish.
III. CNAME records
CNAME records are basically aliases to A records. For instance, "www.domain.com" might be the same server as "ftp.domain.com", so you could make ftp.domain.com a CNAME for www.domain.com, then an additional query would happen for the A record "www.domain.com" which when resolved would then allow transmission.
Typically, CNAMEs should not be in MX records. There's a ton of debate over whether they're allowed in the MX at all according to RFCs. I'm not going to enter that debate, but I will say that there are enough mail servers on the planet that don't respect CNAMEs in the MX that you should do everything in your power not to put them there. At the very least it adds an additional query to your DNS server (to get the IP of the A record in the CNAME), so for simplicity's sake I'd recommend against it as well.
One time a CNAME might be used is if you have a server that accepts email that does not have MX records. Originally, user@domain.com was actually literally user@some.server.com. You could even address email to an IP address like "user@10.0.100.28" for example and it would work. Most mail servers will fall back to this convention if MX records cannot be resolved, so you could for instance have a CNAME errors.domain.com pointing to "mail1.domain.com", then you could email "www-server@errors.domain.com", "email-server@errors.domain.com", etc which would then actually send the email to mail1.domain.com.
NOTE: this does not alias other record types, only A records. For instance, setting domain2.com as a CNAME for domain1.com DOES NOT make domain2.com's MX records the same as domain1.com's. CNAME records are not copies of the zone, they are address aliases and will only route traffic to the A record they point to. Additionally, it's technically against RFCs to have both MX and CNAMEs for the same zone. I.e. if you have www.domain.com as a CNAME, you can't also have MX for www.domain.com.
So, there you have it. MX record resolution requires SOA lookups, which require their own A record resolution. MX records include A records typically, which require their own resolution by the SOA. CNAMEs shouldn't be used in the MX, and can't have their own MX, but they can still be used to route email in a pinch.
Try using nslookup or dig to do your own DNS recursion. Once you realize how many DNS queries each email requires, be sure to give your DNS server a big pat on the back for so many jobs well done.
-TEA
Not quite yet. Patience young grasshopper.
So, you're sending email to user@domain.com, you queried dns1.domain.com for the MX records and now you've got these MX records:
Host Priority Record
domain.com 10 mail1.domain.com
domain.com 20 mail2.domain.com
domain.com 30 mail3.domain.com
Ummm, so, computers don't speak English, they don't know what a "mail1.domain.com" is. They want digits that can turn into nice 1s and 0s that they can compute . What's a poor mail server to do with this stuff?
While we're at it, how did we even know to talk to dns1.domain.com to get these records?
Also, once we knew to talk to dns1.domain.com, we're back to the first question of how does the mail server know WTH a "dns1.domain.com" is, and where are the 1s and 0s it needs?
I is so confus :-(
Don't worry, there's more to an email's life than just MX records. There's lots of types of records including type A, type SOA, and type CNAME.
I. Type A records
Type A records ("A" for "Address") turn "mail1.domain.com" into a nice pretty number (the "IP Address" or "Internet Protocol address") for our mail server (and all other computers) to compute. The A record might look like this:
Mail1.domain.com IN A 10.0.100.28
Now we can see that the first MX record for domain.com has a name of "mail1.domain.com", but that name actually resides at 10.0.100.28. This number means nothing to most of us silly humans, but to a computer it's a nice string of digits that it can compute and then connect to.
TCP/IP and all of it's routing mechanisms are beyond the scope of this article, so you'll just have to make do with "once you have the IP you can connect to the server". :-P pbbt!
II. SOA records
OK, so this is a little backwards since you need the SOA records before you can get the MX records or A records, but you should already understand MX records, and you need to understand A records before you can get SOA records.
SOA records are the "Start Of Authority" records. There are literally millions of DNS servers in the world. How do you know which ones are responsible for the records you're querying? Well, that's where SOA records come into play. SOA records are given by the "root" name servers to offload queries to other DNS servers, and to provide a distributed architecture that is easier to maintain and provides more fault tolerance.
Root name servers are the biggest and baddest name servers in the world. There are two sets of them for your .com, .net, and the like, one at root-servers.net and one at gtld-servers.net. If queried, they will either reference the other one (like a father who says "go ask your mother", or vice-versa) or they will give the DNS server registered with them as authoritative.
You can play with this yourself using nslookup on Windows or dig on *nix.
Once the root servers give the name of the DNS server, that server's A record must be resolved then it can be queried for the MX, A, CNAME, or any other type of records you wish.
III. CNAME records
CNAME records are basically aliases to A records. For instance, "www.domain.com" might be the same server as "ftp.domain.com", so you could make ftp.domain.com a CNAME for www.domain.com, then an additional query would happen for the A record "www.domain.com" which when resolved would then allow transmission.
Typically, CNAMEs should not be in MX records. There's a ton of debate over whether they're allowed in the MX at all according to RFCs. I'm not going to enter that debate, but I will say that there are enough mail servers on the planet that don't respect CNAMEs in the MX that you should do everything in your power not to put them there. At the very least it adds an additional query to your DNS server (to get the IP of the A record in the CNAME), so for simplicity's sake I'd recommend against it as well.
One time a CNAME might be used is if you have a server that accepts email that does not have MX records. Originally, user@domain.com was actually literally user@some.server.com. You could even address email to an IP address like "user@10.0.100.28" for example and it would work. Most mail servers will fall back to this convention if MX records cannot be resolved, so you could for instance have a CNAME errors.domain.com pointing to "mail1.domain.com", then you could email "www-server@errors.domain.com", "email-server@errors.domain.com", etc which would then actually send the email to mail1.domain.com.
NOTE: this does not alias other record types, only A records. For instance, setting domain2.com as a CNAME for domain1.com DOES NOT make domain2.com's MX records the same as domain1.com's. CNAME records are not copies of the zone, they are address aliases and will only route traffic to the A record they point to. Additionally, it's technically against RFCs to have both MX and CNAMEs for the same zone. I.e. if you have www.domain.com as a CNAME, you can't also have MX for www.domain.com.
So, there you have it. MX record resolution requires SOA lookups, which require their own A record resolution. MX records include A records typically, which require their own resolution by the SOA. CNAMEs shouldn't be used in the MX, and can't have their own MX, but they can still be used to route email in a pinch.
Try using nslookup or dig to do your own DNS recursion. Once you realize how many DNS queries each email requires, be sure to give your DNS server a big pat on the back for so many jobs well done.
-TEA
Thursday, June 10, 2010
Email Feedback Report for IP
Really AOL, really?
Over the years I have seen remarkable feedback reports from AOL. Everything from obvious user error like newsletters, etc. to bills that the user did not want to receive (but were completely legit), to those "chain letters" telling you that you and your entire family will die painfully unless you forward the message to 200 of your closest friends.
Recently, I saw a feedback report for a read receipt. A READ RECEIPT. Someone sent an email, got the read receipt back and then reported it as spam. That part doesn't surprise me (nothing much does any more), but a provider that has been in business as long as AOL (remember when they "were" the WWW for many Americans?) not having processes in place to deal with a user error this remarkably dumb amazes me.
First, their entire concept of blacklisting IPs is so 1999. It's 2010, use a content scanner!
Second, relying on their users to create an automated IP blacklist for them misplaces the burden onto their customer base, rather than trained techs (who can tell the difference between spam and a read receipt) and simultaneously creates a worse experience for all of their other users when "spam button" happy users wreck the IPBL for everyone else.
Third, if you're going to have an IP blacklist, and it's going to automatically block based on user reports, AT LEAST try to filter out obvious user error. A READ RECEIPT! Their logs show the message leaving, and the message clearly states it's a read receipt, the headers even had the appropriate "notification" message type. AOL, do you really have to make it this easy to poke fun at you?
AOL, I would like to offer my services to you. I'll even do it for half my hourly rate. Just please, please, please revisit this horribly planned out mechanism and get the correct processes in place to handle your spam problems.
While you're at it, could you please fix your outbound spam problems too? How many of your customers are really in Nigeria that know dead rich people with my last name?
-TEA
Over the years I have seen remarkable feedback reports from AOL. Everything from obvious user error like newsletters, etc. to bills that the user did not want to receive (but were completely legit), to those "chain letters" telling you that you and your entire family will die painfully unless you forward the message to 200 of your closest friends.
Recently, I saw a feedback report for a read receipt. A READ RECEIPT. Someone sent an email, got the read receipt back and then reported it as spam. That part doesn't surprise me (nothing much does any more), but a provider that has been in business as long as AOL (remember when they "were" the WWW for many Americans?) not having processes in place to deal with a user error this remarkably dumb amazes me.
First, their entire concept of blacklisting IPs is so 1999. It's 2010, use a content scanner!
Second, relying on their users to create an automated IP blacklist for them misplaces the burden onto their customer base, rather than trained techs (who can tell the difference between spam and a read receipt) and simultaneously creates a worse experience for all of their other users when "spam button" happy users wreck the IPBL for everyone else.
Third, if you're going to have an IP blacklist, and it's going to automatically block based on user reports, AT LEAST try to filter out obvious user error. A READ RECEIPT! Their logs show the message leaving, and the message clearly states it's a read receipt, the headers even had the appropriate "notification" message type. AOL, do you really have to make it this easy to poke fun at you?
AOL, I would like to offer my services to you. I'll even do it for half my hourly rate. Just please, please, please revisit this horribly planned out mechanism and get the correct processes in place to handle your spam problems.
While you're at it, could you please fix your outbound spam problems too? How many of your customers are really in Nigeria that know dead rich people with my last name?
-TEA
Subscribe to:
Posts
(
Atom
)