In part 1 of this series, we discussed what TTL is, and provided a few exercises to help strengthen your understanding of TTL. We'll pick up where we left off with our same DNS zone for domain1.com:
domain1.com. 86400 IN NS ns1.domain1s-ISP.com.
domain1.com. 86400 IN NS ns2.domain1s-ISP.com.
domain1.com. 14400 IN MX mail.domain1.com.
mail.domain1.com. 3600 IN A 10.11.12.13.
As we discussed, the NS records will expire 1 day after they were last cached, the MX records will expire 4 hours after they were last cached, and the A record will expire 60 minutes after it was last cached. What does this mean for you and your planned migration?
A common misconception about TTL is that it's the amount of time the Name Server requires to update the record. This is absolutely not the case. Do you remember what RFC 1034 said a TTL is? A TTL is:
...primarily used by resolvers when they cache RRs. The TTL describes how long a RR can be cached before it should be* discarded.
Where in there does it say that a TTL is how long a name server takes to update? What does it say? It says that a DNS resolver (which is anything that resolves DNS, your computer has a DNS resolver in it) caches the records, and the TTL describes how long the DNS resolver should cache the record before discarding it (and then re-requesting it).
What this means is that (for the sake of simplicity) every computer connected to the internet has it's own local cache of websites it's visited, mail servers it's talked to, etc. and will not re-request those records until it's own, local, cache expires. TTL is how long it should* keep the records until it discards them and re-requests them.
Object lesson time...
UserPC1.ValuedReader.com visits www.TheEmailAuthority.com at 2:30 PM. www.TheEmailAuthority.com has a 1 hour TTL for it's A record. If I move the website at 3 PM, and the old IP stops serving the site, how long would it be until UserPC1.ValuedReader.com can read my site again?
UserPC-2.ValuedReader.com visits www.TheEmailAuthority.com at 3:10 PM How long will it be until they can read the site?
The answers and explanations will be in part three.
Friday, December 7, 2012
TTL in DNS part 1
TTL, or Time To Live, is a confusing subject for many people. Due to DNS's distributed nature, there are many different scenarios to consider. In this post we will explain what TTL is, and what it is not, using a real world example.
First, I suppose we'll answer why TTL is important. TTL can impact your email infrastructure any time you make changes to a Name Server that hosts your DNS, the A record that your mail server resolves to (for instance, when you change to a new hosting provider), or when you change your MX records (for a new hosting provider, or a new mail relay). By no means is this a comprehensive list, but it will suffice for this example. Since DNS records can be cached for days sometimes, it's important to understand how they work when planning a migration so you can account for several day's worth of traffic.
How it works:
According to RFC 1034, "DOMAIN NAMES - CONCEPTS AND FACILITIES", each DNS record (or "Resource Record", "RR" for short) has a TTL or Time To Live. It describes TTL as:
TTL which is the time to live of the RR. This field is a 32 bit integer in units of seconds, an is primarily used by resolvers when they cache RRs. The TTL describes how long a RR can be cached before it should be* discarded.
A "Resolver" is anything that resolves a DNS record. This could be your ISP's name servers, or your local computer. Anything that queries and resolves DNS records is a "DNS Resolver".
Let's take the below zone into consideration:
domain1.com. 86400 IN NS ns1.domain1s-ISP.com.
domain1.com. 86400 IN NS ns2.domain1s-ISP.com.
domain1.com. 14400 IN MX mail.domain1.com.
mail.domain1.com. 3600 IN A 10.11.12.13.
So, let's take our three examples above and do some "real world" exercises. Remember, TTL is expressed in seconds. You might want to open your calculator app now ;-)
If domain1.com wanted to switch their Name Server records to a different ISP, how many days would it be until the NS TTL expired?
If domain1.com wanted to add a hosted anti-spam service (which would require them to change the MX records), how many hours would it take for the MX TTL to expire?
If domain1.com wanted to upgrade their mail server and point the A record to a new IP, how many minutes would it take for the A record's TTL to expire?
Ok, so we've gotten through the math portion of this post. Pencils on the table please. Now we know that records can take DAYS to update depending on their TTL. How does this impact our migration? I'll answer in part two...
First, I suppose we'll answer why TTL is important. TTL can impact your email infrastructure any time you make changes to a Name Server that hosts your DNS, the A record that your mail server resolves to (for instance, when you change to a new hosting provider), or when you change your MX records (for a new hosting provider, or a new mail relay). By no means is this a comprehensive list, but it will suffice for this example. Since DNS records can be cached for days sometimes, it's important to understand how they work when planning a migration so you can account for several day's worth of traffic.
How it works:
According to RFC 1034, "DOMAIN NAMES - CONCEPTS AND FACILITIES", each DNS record (or "Resource Record", "RR" for short) has a TTL or Time To Live. It describes TTL as:
TTL which is the time to live of the RR. This field is a 32 bit integer in units of seconds, an is primarily used by resolvers when they cache RRs. The TTL describes how long a RR can be cached before it should be* discarded.
A "Resolver" is anything that resolves a DNS record. This could be your ISP's name servers, or your local computer. Anything that queries and resolves DNS records is a "DNS Resolver".
Let's take the below zone into consideration:
domain1.com. 86400 IN NS ns1.domain1s-ISP.com.
domain1.com. 86400 IN NS ns2.domain1s-ISP.com.
domain1.com. 14400 IN MX mail.domain1.com.
mail.domain1.com. 3600 IN A 10.11.12.13.
So, let's take our three examples above and do some "real world" exercises. Remember, TTL is expressed in seconds. You might want to open your calculator app now ;-)
If domain1.com wanted to switch their Name Server records to a different ISP, how many days would it be until the NS TTL expired?
If domain1.com wanted to add a hosted anti-spam service (which would require them to change the MX records), how many hours would it take for the MX TTL to expire?
If domain1.com wanted to upgrade their mail server and point the A record to a new IP, how many minutes would it take for the A record's TTL to expire?
Ok, so we've gotten through the math portion of this post. Pencils on the table please. Now we know that records can take DAYS to update depending on their TTL. How does this impact our migration? I'll answer in part two...
Saturday, July 10, 2010
Stopping spam
Albert Einstein said “We can't solve problems by using the same kind of thinking we used when we created them". This is certainly true of spam.
In the early days of email being widely adopted people were excited to use this new tool - and unaware of the threats that this new tool came with - so they handed out their email address willy-nilly. Every site that asked them for it, they gave it. If they wanted a personal response on a forum, they posted their email address for someone to respond to. In order to communicate with customers, they'd put their email address proudly on their "Contact us" page to show how "tech savvy" they were.
Then the unthinkable happened. People started emailing them.
Not about the thing they downloaded, not in response to their forum post, or even any customers from their "contact us" page. These emails were offers to buy prescription drugs or pornography. The senders had the unfortunate duty of informing them of the death of a rich loved one in Africa somewhere, or had good news of their winning a lottery Microsoft was running right now.
By the time they realized what was happening, their email address had already been abused, traded to other spammers, or sold in batches of "opt in" email addresses to the highest bidder. Their new email "toy" rapidly became much less fun to play with.
This is the "kind of thinking" that "we used when we created (the problem)". Email is a phenomenal tool, but it is a tool not a toy. You wouldn't let someone who had never used a chainsaw play with yours, and you wouldn't use a bandsaw to hammer a nail. Tools have purposes, and misuse can be dangerous.
So, what do we do? What's the "right" way of thinking we should start using?
1. Protect the email addresses you have.
Don't give it out to just anybody, and block unauthorized access to it. Use a spam blocker with not just a high block rate, but that also won't block your real email. This will save untold amounts of time (and therefore money) troubleshooting issues or even just searching through your junk looking for legit messages.
2. Use a unique email address.
This doesn't always mean creating new addresses every time you want to sign up for something. If your email address is YourUsername@YourDomain.com, try sending email to YourUsername+Whatever@YourDomain.com. If you receive it, you can do this when you sign up for things online and use something unique for each.
For instance, YourUsername+TheEmailAuthority@YourDomain.com if you sign up for a newsletter from me or YourUsername+TigerDirect@YourDomain.com if you sign up with Tiger Direct.
This will allow you to see WHO has compromised your email address, and also set up filtering rules if one of your "new addresses" gets compromised.
3. Use a "disposable" email address.
Ugh. We live in such a disposable culture. Disposable razors, disposable diapers, disposable this, that and something else. You mean to tell me there are disposable email addresses now too?!?! Yes. And they have a GREAT purpose.
When you sign up for a service, or a newsletter, coupons, etc you WANT to continue to receive emails from them. If you didn't, you wouldn't have signed up, right? Well, what about all of the myriad of websites that you need to give them your email address just to sign up and then you never want to receive another email from them, EVER? Use a disposable address, that's what.
Keep in mind, password resets, changes in TOSes and other vital email will go to this address as well, so don't use it for everything, but it's a great addition to your privacy protecting arsenal. Someone could guess to strip off everything after the "+" sign if you use the method above, but they'll never guess the real address associated to a completely fake address.
There are several services out there. I've used several, and I like Mailinator, but you can find a ton with this Google search.
So, here's the breakdown:
Give your real email address to friends, family and business partners.
Give your "+" address to sites you need to have ongoing communication with, but aren't in your immediate "circle of friends". Examples are stores, newsletters, blogs/websites, etc.
Use a disposable address for anyone you only need to communicate with once.
Using these simple steps will help keep your inbox clean and your privacy protected. To help protect against everything else, you need a good antispam service.
-TEA
In the early days of email being widely adopted people were excited to use this new tool - and unaware of the threats that this new tool came with - so they handed out their email address willy-nilly. Every site that asked them for it, they gave it. If they wanted a personal response on a forum, they posted their email address for someone to respond to. In order to communicate with customers, they'd put their email address proudly on their "Contact us" page to show how "tech savvy" they were.
Then the unthinkable happened. People started emailing them.
Not about the thing they downloaded, not in response to their forum post, or even any customers from their "contact us" page. These emails were offers to buy prescription drugs or pornography. The senders had the unfortunate duty of informing them of the death of a rich loved one in Africa somewhere, or had good news of their winning a lottery Microsoft was running right now.
By the time they realized what was happening, their email address had already been abused, traded to other spammers, or sold in batches of "opt in" email addresses to the highest bidder. Their new email "toy" rapidly became much less fun to play with.
This is the "kind of thinking" that "we used when we created (the problem)". Email is a phenomenal tool, but it is a tool not a toy. You wouldn't let someone who had never used a chainsaw play with yours, and you wouldn't use a bandsaw to hammer a nail. Tools have purposes, and misuse can be dangerous.
So, what do we do? What's the "right" way of thinking we should start using?
1. Protect the email addresses you have.
Don't give it out to just anybody, and block unauthorized access to it. Use a spam blocker with not just a high block rate, but that also won't block your real email. This will save untold amounts of time (and therefore money) troubleshooting issues or even just searching through your junk looking for legit messages.
2. Use a unique email address.
This doesn't always mean creating new addresses every time you want to sign up for something. If your email address is YourUsername@YourDomain.com, try sending email to YourUsername+Whatever@YourDomain.com. If you receive it, you can do this when you sign up for things online and use something unique for each.
For instance, YourUsername+TheEmailAuthority@YourDomain.com if you sign up for a newsletter from me or YourUsername+TigerDirect@YourDomain.com if you sign up with Tiger Direct.
This will allow you to see WHO has compromised your email address, and also set up filtering rules if one of your "new addresses" gets compromised.
3. Use a "disposable" email address.
Ugh. We live in such a disposable culture. Disposable razors, disposable diapers, disposable this, that and something else. You mean to tell me there are disposable email addresses now too?!?! Yes. And they have a GREAT purpose.
When you sign up for a service, or a newsletter, coupons, etc you WANT to continue to receive emails from them. If you didn't, you wouldn't have signed up, right? Well, what about all of the myriad of websites that you need to give them your email address just to sign up and then you never want to receive another email from them, EVER? Use a disposable address, that's what.
Keep in mind, password resets, changes in TOSes and other vital email will go to this address as well, so don't use it for everything, but it's a great addition to your privacy protecting arsenal. Someone could guess to strip off everything after the "+" sign if you use the method above, but they'll never guess the real address associated to a completely fake address.
There are several services out there. I've used several, and I like Mailinator, but you can find a ton with this Google search.
So, here's the breakdown:
Give your real email address to friends, family and business partners.
Give your "+" address to sites you need to have ongoing communication with, but aren't in your immediate "circle of friends". Examples are stores, newsletters, blogs/websites, etc.
Use a disposable address for anyone you only need to communicate with once.
Using these simple steps will help keep your inbox clean and your privacy protected. To help protect against everything else, you need a good antispam service.
-TEA
Thursday, July 8, 2010
Bounce message spam
There's a new spam in the wild claiming to be bounce messages to messages that you've sent, but that actually send you to a meds spammer's site. They've been mutating for several days, so the body and attachment change fairly rapidly, but here's what one looks like as of the date of this post:
--- begin spam ---
Delivery Status Notification (Failure) Note: Forwarded message is attached.
This is an automatically generated Delivery Status Notification
THIS IS A WARNING MESSAGE ONLY.
Delivery to the following recipient has been delayed:
user@domain.com
Message will be retried for 2 more day(s)
---end spam---
Then there is an HTML attachment which redirects to the spammer's web site.
It's really quite ingenious really. Who wouldn't open a bounce, even if they didn't remember sending the original email, or maybe especially if they didn't remember sending it?
Another one, this time stating that it's from "mailfilter@yourdomain.com":
---another spam---
Note: Forwarded message is attached.
An email you sent could not be delivered.
Subject: Hello
Delivery to the following address has failed...
user@RecipientDomain.com
Technical information about this permanent failure:
-- Transcript of session follows --
While talking to [some.server.com] :
>>> rcpt to: user@RecipientDomain.com
<<<>: Recipient address rejected: User unknown
---end spam---
Users are so accustomed to worrying thet they're "infected" every time the get a bounce (when in fact, often it's just "spoofing") that these fake spam bounces must have a huge click through rate.
Although it's difficult to block "backscatter", I have to imagine that these are relatively easy to block. All the HTML attachment includes is a refresh to their site, any content based scanner should pick them up pretty easily.
As for the user's perspective, the old rules (slightly modified) still pertain here.
If you do not know the sender, do not open the attachment.
With these new "bounce message" spams, you can modify the above rule to:
If you did not send the email, do not open any attachments.
If you DO know the sender, and it's not a file you requested, call them to see if they actually sent it. Especially if it's in the following formats: .exe, .pif, .zip, .rar, .bat, .pdf, .htm, .html.
Of course, these spam messages have .htm file attachments, so the above pertains to them as well. Although these are a simple spam, websites can be infected and spread malware through your browser. Always exercise extreme caution when opening .htm files.
-TEA
--- begin spam ---
Delivery Status Notification (Failure) Note: Forwarded message is attached.
This is an automatically generated Delivery Status Notification
THIS IS A WARNING MESSAGE ONLY.
Delivery to the following recipient has been delayed:
user@domain.com
Message will be retried for 2 more day(s)
---end spam---
Then there is an HTML attachment which redirects to the spammer's web site.
It's really quite ingenious really. Who wouldn't open a bounce, even if they didn't remember sending the original email, or maybe especially if they didn't remember sending it?
Another one, this time stating that it's from "mailfilter@yourdomain.com":
---another spam---
Note: Forwarded message is attached.
An email you sent could not be delivered.
Subject: Hello
Delivery to the following address has failed...
user@RecipientDomain.com
Technical information about this permanent failure:
-- Transcript of session follows --
While talking to [some.server.com] :
>>> rcpt to: user@RecipientDomain.com
<<<>: Recipient address rejected: User unknown
---end spam---
Users are so accustomed to worrying thet they're "infected" every time the get a bounce (when in fact, often it's just "spoofing") that these fake spam bounces must have a huge click through rate.
Although it's difficult to block "backscatter", I have to imagine that these are relatively easy to block. All the HTML attachment includes is a refresh to their site, any content based scanner should pick them up pretty easily.
As for the user's perspective, the old rules (slightly modified) still pertain here.
If you do not know the sender, do not open the attachment.
With these new "bounce message" spams, you can modify the above rule to:
If you did not send the email, do not open any attachments.
If you DO know the sender, and it's not a file you requested, call them to see if they actually sent it. Especially if it's in the following formats: .exe, .pif, .zip, .rar, .bat, .pdf, .htm, .html.
Of course, these spam messages have .htm file attachments, so the above pertains to them as well. Although these are a simple spam, websites can be infected and spread malware through your browser. Always exercise extreme caution when opening .htm files.
-TEA
Spoofing
"Spoofing" someone's email address is really hard, right? Only the most elite hackers can do it, right? It's not something that I can try at home, right?
no, no and no.
In the post "Testing SMTP" we discussed all of the commands necessary to fully deliver an email. Nothing special, no encryption, just a plain text email (without MIME-types, etc), but an email nonetheless. Below is the list of commands again in case you missed that post:
---------------------------------------------------------------------------
HELO your hostname
MAIL FROM:
RCPT TO:
DATA
FROM:
TO:
SUBJECT: test
this is a test.
.
----------------------------------------------------------------
You'll notice something is missing in there. Something important for security. Something you do to many websites, to view your email, and often the first thing when you turn on your computer.
Don't scroll down, take a guess :-)
There's no login/password!
You could send yourself an email from GeorgeWashington@whitehouse.gov, or Bill@Microsoft.com saying whatever you like. Of course, you're still incredibly traceable, so don't get any bright ideas about sending an email "from" your boss giving yourself a raise or anything.
This is one way though that spammers and viruses hide their tracks or create a false sense of trust from their victim. SPF was implemented as a direct result of this flaw in SMTP's security.
Many times, spammers will "spoof" other users in their database of email addresses. Spam/virus filtering after the "Border-MTA" results then in backscatter, or bounces from email that was not sent by you. This is of course different than the article earlier about fake bounce message spam. Backscatter is legitimate bounces, but sent in response to an illegitimate use of the victim's email address.
Sometimes backscatter can inundate an inbox so thoroughly that it's hard to get any work done. In cases like this, ALL bounce messages should be blocked or moved to a folder keeping them out of the inbox. A common way to do this is to make a rule for these senders:
<>
postmaster@
mailer-daemon@
And that includes the following phrases:
"hi, this is the qmail send program"
"returned mail: see transcript for details"
"undeliverable mail returned to sender"
"delivery status notification (failure)"
"delivery status notification (delay)"
"out of office autoreply"
and any others that come in.
One of the reasons to not use a challenge-response anti-spam service is that they accept all email for your domain, then deliver "challenges" to which the sender must "respond". Since they accept the email, many deliver challenges to innocent users who are simply being abused by a spammer. Therefore you'll also want to include challenge response vendors in your filter.
Unfortunately, the lack of security in SMTP has been abused to no end. That's why it's imperative for your reputation, and a good "good neighbor" policy to implement SPF records. Although they're a little hard to grasp at first, They're the best way to be sure that nobody is using your email address without your permission. Unless you've been hacked, but that's a whole other post....
-TEA
no, no and no.
In the post "Testing SMTP" we discussed all of the commands necessary to fully deliver an email. Nothing special, no encryption, just a plain text email (without MIME-types, etc), but an email nonetheless. Below is the list of commands again in case you missed that post:
---------------------------------------------------------------------------
HELO your hostname
SUBJECT: test
this is a test.
.
----------------------------------------------------------------
You'll notice something is missing in there. Something important for security. Something you do to many websites, to view your email, and often the first thing when you turn on your computer.
Don't scroll down, take a guess :-)
There's no login/password!
You could send yourself an email from GeorgeWashington@whitehouse.gov, or Bill@Microsoft.com saying whatever you like. Of course, you're still incredibly traceable, so don't get any bright ideas about sending an email "from" your boss giving yourself a raise or anything.
This is one way though that spammers and viruses hide their tracks or create a false sense of trust from their victim. SPF was implemented as a direct result of this flaw in SMTP's security.
Many times, spammers will "spoof" other users in their database of email addresses. Spam/virus filtering after the "Border-MTA" results then in backscatter, or bounces from email that was not sent by you. This is of course different than the article earlier about fake bounce message spam. Backscatter is legitimate bounces, but sent in response to an illegitimate use of the victim's email address.
Sometimes backscatter can inundate an inbox so thoroughly that it's hard to get any work done. In cases like this, ALL bounce messages should be blocked or moved to a folder keeping them out of the inbox. A common way to do this is to make a rule for these senders:
<>
postmaster@
mailer-daemon@
And that includes the following phrases:
"hi, this is the qmail send program"
"returned mail: see transcript for details"
"undeliverable mail returned to sender"
"delivery status notification (failure)"
"delivery status notification (delay)"
"out of office autoreply"
and any others that come in.
One of the reasons to not use a challenge-response anti-spam service is that they accept all email for your domain, then deliver "challenges" to which the sender must "respond". Since they accept the email, many deliver challenges to innocent users who are simply being abused by a spammer. Therefore you'll also want to include challenge response vendors in your filter.
Unfortunately, the lack of security in SMTP has been abused to no end. That's why it's imperative for your reputation, and a good "good neighbor" policy to implement SPF records. Although they're a little hard to grasp at first, They're the best way to be sure that nobody is using your email address without your permission. Unless you've been hacked, but that's a whole other post....
-TEA
Sunday, July 4, 2010
Email and the OSI model.
All People Seem To Need Data Processing, and when it comes to networking, understanding the OSI model is imperative. However you remember this seven layer schema, it's vital when troubleshooting difficult issues to remember them. Use these handy dandy mnemonics to help:
Please Do Not Throw Sausage Pizza Away (layer 1-7)
All People Seem To Need Data Processing (backwards, from layer 7-1)
Please Do Not Trust Sales People Always (1-7 again)
More information can be found in A+ and Network+ cert books, and other study guides I'm sure. Wikipedia has more information here too.
It's important to understand that although rare, your server logs can lie. Your mail server runs on the Application layer, so it only "sees" what that layer sees. Although of course it relies on all layers on down to the Physical layer, one single server may have several daemons running on different layers intercepting the data at different points along the way.
For instance, on a *nix box, you may have Sendmail logging that it's connecting to mail.domain.com [1.2.3.4].
Sendmail made a call to the DNS resolver and pulled the MX records. Again it called the resolver to resolve the A records listed in the MX. Then it tries to connect, but IPtables has a static route for 1.2.3.4 to send the data to 5.6.7.8, which is now refusing the connection.
Since Sendmail is running on the application layer, it doesn't log (or even see) when IP it's ACTUALLY connecting to, only where it *should* be connecting to. In order to see the packets you're going to have to do some traffic dumps using tcpdump, tcpick or Wireshark.
Of course, this is a very unique situation, either intentionally done by a sophisticated admin as a workaround or as a temporary "fix" for DNS issues, etc. Either way it's a kludge, and this is why each server should keep a changelog for later reference including the exact lines entered for later grep-ability (yes, I just made that word up :-) )
Also, there obviously could be many other issues with routing or other types of packet interference from router/firewalls locally to ISP related routing issues, but this was one instance that boggled me for several hours, so it stands out as an excellent reference.
-TEA
Please Do Not Throw Sausage Pizza Away (layer 1-7)
All People Seem To Need Data Processing (backwards, from layer 7-1)
Please Do Not Trust Sales People Always (1-7 again)
More information can be found in A+ and Network+ cert books, and other study guides I'm sure. Wikipedia has more information here too.
It's important to understand that although rare, your server logs can lie. Your mail server runs on the Application layer, so it only "sees" what that layer sees. Although of course it relies on all layers on down to the Physical layer, one single server may have several daemons running on different layers intercepting the data at different points along the way.
For instance, on a *nix box, you may have Sendmail logging that it's connecting to mail.domain.com [1.2.3.4].
Sendmail made a call to the DNS resolver and pulled the MX records. Again it called the resolver to resolve the A records listed in the MX. Then it tries to connect, but IPtables has a static route for 1.2.3.4 to send the data to 5.6.7.8, which is now refusing the connection.
Since Sendmail is running on the application layer, it doesn't log (or even see) when IP it's ACTUALLY connecting to, only where it *should* be connecting to. In order to see the packets you're going to have to do some traffic dumps using tcpdump, tcpick or Wireshark.
Of course, this is a very unique situation, either intentionally done by a sophisticated admin as a workaround or as a temporary "fix" for DNS issues, etc. Either way it's a kludge, and this is why each server should keep a changelog for later reference including the exact lines entered for later grep-ability (yes, I just made that word up :-) )
Also, there obviously could be many other issues with routing or other types of packet interference from router/firewalls locally to ISP related routing issues, but this was one instance that boggled me for several hours, so it stands out as an excellent reference.
-TEA
Monday, June 21, 2010
Testing SMTP
Sometimes it's important to be able to manually reproduce what your computer/server is doing manually so you can see exactly where it's failing. Troubleshooting "utilities" are great in that they can often diagnose hundreds of issues in less time than it would take you to troubleshoot one, but they lack the nuance sometimes necessary to accurately diagnose issues.
A simple pass/fail is nice, but if you don't know WHY it's failing, what good is it?
When troubleshooting email, there are websites, utilities, diagnostic messages and more designed to help figure out what's going wrong. I have never found a single one that has been more informative than a simple SMTP session.
If you have telnet on your computer, that'll work fine. Netcat or any other command line tool that you can specify the ports to connect on will work too.
On your command/shell prompt type: telnet <the server's IP> 25
Then, when prompted, type the following commands:
---------------------------------------------------------------------------
HELO your hostname
<wait for response>
MAIL FROM: <your email address>
RCPT TO:
<wait for response>
DATA
FROM:
TO:
SUBJECT: test
this is a test.
.
----------------------------------------------------------------
Note the additional CR/LF after the subject. That separates the headers from the body (visible part) of the message. It's very important.
Also important is the line with a "." by itself. This tells the recipient's server that the message is complete and to process it.
On to troubleshooting....
In each part above where it says "wait for a response" you're expecing a response from the server, either something like "ok, go ahead", an error, or sometimes nothing. Here are some examples:
HELO
If there's no response, this can indicate that the server is extremely busy or that sometimes there's packet filtration between you and the server that's intercepting and silently dropping the packets. Either way, if there's no response, there's no transmission, and therefore no email.
Sometimes you'll get an error here referencing that it does not like your IP (if you're blacklisted), or that it's imposing some type of lookup on your HELO argument. It's best to be sure thet you give your FQDN here to prevent issues.
MAIL FROM:
You might get SPF errors here if your SPF record is wrong, or if your email address is blacklisted you might get a response here.
RCPT TO:
"Relaying denied", "mailbox is full", or any number of other recipient level issues may result as a response to this line. Any server that has different rules per user or domain (like a shared spam filter with different settings per domain/user) might also process here.
DATA
You're now giving the actual data that will be shown to the user. Generally there won't be an issue here until you close the transmission channel (with the "." on a line by itself), but anywhere you wait for a response is a potential place for errors.
"." (or closing the transmission channel)
This is where the email in it's entirety can be scanned and processed. Sometimes nothing will be scanned until this point, so even though SPF *could* be looked up after the "mail from:" command, it *might* not be until the entire message has been transmitted. Any of the above errors can happen here as well as content based scanning of the body, etc.
Some mail servers will also accept the email here, and then scan it later before delivery. That's a horrible idea, but that doesn't stop anyone usually, so be sure to copy down any information the acceptance line gives you like the message ID, so you can reference it later if the test does not complete.
Generally, you'll only be able to run this test on a border MTA unless it's your own mail server or the recipient's firewall allows you through. This can definitely complicate troubleshooting, but if you can prove that the issue is with the border MTA, or that the border MTA accepts the email but it is not relayed it at least absolves you from any finger pointing.
Once you have these commands memorized (it doesn't take long), you'll often be able to troubleshoot and diagnose issues faster than submitting tickets to your vendors. There's nothing more frustrating than either waiting for a response only to find out you could have fixed it hours ago, or conversely that you know exactly where the error occurred, but the other admin requires "proof" that they should bother fixing the issue when the DSN or other logs obviously state the issue.
Send a couple SMTP sessions their way and copy the session to them. It's hard to argue with facts :-)
-TEA
A simple pass/fail is nice, but if you don't know WHY it's failing, what good is it?
When troubleshooting email, there are websites, utilities, diagnostic messages and more designed to help figure out what's going wrong. I have never found a single one that has been more informative than a simple SMTP session.
If you have telnet on your computer, that'll work fine. Netcat or any other command line tool that you can specify the ports to connect on will work too.
On your command/shell prompt type: telnet <the server's IP>
Then, when prompted, type the following commands:
---------------------------------------------------------------------------
HELO your hostname
<wait for response>
this is a test.
.
----------------------------------------------------------------
Note the additional CR/LF after the subject. That separates the headers from the body (visible part) of the message. It's very important.
Also important is the line with a "." by itself. This tells the recipient's server that the message is complete and to process it.
On to troubleshooting....
In each part above where it says "wait for a response" you're expecing a response from the server, either something like "ok, go ahead", an error, or sometimes nothing. Here are some examples:
HELO
If there's no response, this can indicate that the server is extremely busy or that sometimes there's packet filtration between you and the server that's intercepting and silently dropping the packets. Either way, if there's no response, there's no transmission, and therefore no email.
Sometimes you'll get an error here referencing that it does not like your IP (if you're blacklisted), or that it's imposing some type of lookup on your HELO argument. It's best to be sure thet you give your FQDN here to prevent issues.
MAIL FROM:
You might get SPF errors here if your SPF record is wrong, or if your email address is blacklisted you might get a response here.
RCPT TO:
"Relaying denied", "mailbox is full", or any number of other recipient level issues may result as a response to this line. Any server that has different rules per user or domain (like a shared spam filter with different settings per domain/user) might also process here.
DATA
You're now giving the actual data that will be shown to the user. Generally there won't be an issue here until you close the transmission channel (with the "." on a line by itself), but anywhere you wait for a response is a potential place for errors.
"." (or closing the transmission channel)
This is where the email in it's entirety can be scanned and processed. Sometimes nothing will be scanned until this point, so even though SPF *could* be looked up after the "mail from:" command, it *might* not be until the entire message has been transmitted. Any of the above errors can happen here as well as content based scanning of the body, etc.
Some mail servers will also accept the email here, and then scan it later before delivery. That's a horrible idea, but that doesn't stop anyone usually, so be sure to copy down any information the acceptance line gives you like the message ID, so you can reference it later if the test does not complete.
Generally, you'll only be able to run this test on a border MTA unless it's your own mail server or the recipient's firewall allows you through. This can definitely complicate troubleshooting, but if you can prove that the issue is with the border MTA, or that the border MTA accepts the email but it is not relayed it at least absolves you from any finger pointing.
Once you have these commands memorized (it doesn't take long), you'll often be able to troubleshoot and diagnose issues faster than submitting tickets to your vendors. There's nothing more frustrating than either waiting for a response only to find out you could have fixed it hours ago, or conversely that you know exactly where the error occurred, but the other admin requires "proof" that they should bother fixing the issue when the DSN or other logs obviously state the issue.
Send a couple SMTP sessions their way and copy the session to them. It's hard to argue with facts :-)
-TEA
Subscribe to:
Posts
(
Atom
)