Jul 132019
 

This question popped on Twitter just now; I’m not sure the question was a serious one, but as it happens I can answer the question as if it were serious. I’ve been running relatively small mail servers for nearly 30 years, so do know a litle bit about it.

This of course does not cover lots of “bolt ons” to email such as SPF, DKIM, DMARC, and that’s just the simple stuff.

To keep things relatively sane, I’m going to gloss over an awful lot of details – for example, there are all sorts of ways of composing and sending emails, but I will assume you’re using some sort of web interface such as Google Mail.

So you compose an email with an interesting picture or two, and click “Send”.

Hang on a bit! You also did some technical stuff when you entered an email address into the “To” field (if you just enter someone’s name, something somewhere is changing that to an email address) and possibly something summarising the email (or trying to get attention) into the “Subject” field.

Headers

Those two headers you have filled in – ‘To’ and ‘Subject’ – are just two of a whole collection of headers that are usually invisible. They are invisible because your mail client is hiding them from you; for good reasons as they’re almost always an unnecessary distraction.

But when you have to diagnose an email problem, seeing the full set of raw headers is kind of essential even if it is dead simple (for the relatively technically sophisticated) to forge headers.

For example, if you set up a dedicated mail client rather than use your mail provider’s web-based mail client, you will have to fill out your own email address (to go into the “From” field). You can fill anything you like into that field although you are unlikely to get a reply

Email Addresses

If you fiddle with your mail client enough, you will realise that the “To” field can be changed to a “Cc” field or a “BCc” field. All three take email addresses, and can be used to send an email to someone but they operate in very slightly different ways.

The “To” header is for the main audience of a message.

The “Cc” header is for an audience who should also see the message, but perhaps don’t need to take any action based on the email.

The “Bcc” header is a bit special because it isn’t added to the raw mail message as a header, but is instead used to compose the envelope listing who should receive the email. If you need (or should) keep the list of people receiving an email secret, or just want to avoid an inconveniently long header for people to read then use the “Bcc” header.

Although your mail client may hide the details, email addresses look like :-

  1. some-name@example.com (a “pure” email address)
  2. “Some Name <some-name@example.com” (as normally formatted)

The Message Submission Server

So you click on “send”. What happens then?

The first thing that happens is that your mail client converts your email (attachments, rich formatting, etc.) into plain text which is why a file attachment just under your ISP’s limit on mail sizes can exceed that maximum.

To the top of that plain text version of your message, your mail client attaches some headers (again in plain text). In addition your mail client will prepare an “invisible” envelope independent of the headers – it contains just the address(es) you want to send to, and your own address.

Finally the mail client is ready to talk to the “mail submission server”. It will login (usually) with your credentials, and sends the message to the server using the “Simple Mail Transport Protocol” (SMTP).

The mail server adds your message to a queue and before it says to your mail client “Okay, I have it”, it will make sure that the message is safe on disk. Mail servers go to a lot of effort to make sure your messages don’t get “dropped on the floor” even when the operating system they are running on crashes.

The Journey

After you have submitted a mail message to the message submission server, the message is held in a queue. This is significant because email is a store and forward messaging solution, so every server your message journeys through will make sure it is stored on disk before saying to the server sending the message “Yep. Got it.”.

In theory this should mean that messages never get lost “in the system” (although practice is often different to theory). The store and forward style of architecture suits the era that email is from – back in the 1970s when email could (and did) traverse many different kinds of network rather than “the Internet” that we have today.

What does that mean today? Amongst other things, it means that whilst you may think that your message only visits two mail servers – the one belonging to your ISP and the one belonging to the recipient’s ISP – and that is certainly a valid architecture for very small organisations.

But it does not work so well for larger organisations that may handle millions, or hundreds of millions of message a day. Summing up a large architecture without actually seeing an architecture diagram for every single ISP and large organisation on the Internet is going to be a bit hit and miss, but I’ll give it a go.

Your message submission server will probably simply hand your message off to a server responsible for sending email messages across the Internet. If it cannot send to the recipient’s receiving mail server on the first run through the queue, it may well just hand the message off to a “slow message server” (processing large queues is a bit of a drag).

In either case, the recipient’s server will accept the message and then hand it over to a “mailbox server” that the recipient will connect to.

And the journey is over. At every stage, reliability is favoured over speed.

How do servers send your message to each other? Using that SMTP standard I mentioned earlier.

Speed

The speed of email is a good topic to cover here. Speed is not what email is about in the same way that the postal service for physical letters is not about speed. It’s about reliability

Your messages may be readable by the recipient within 5 minutes 99 times out of 100, but that 1 time may take a day or two. That’s just the way that email works – if you want instant messaging, use an instant messaging client.

Security

At no point have I mentioned encryption, and that is intentional. At no point is encryption required; email was designed in an era when computers were far slower and encryption was expensive (even though it was far weaker).

At every point, servers are trusting that the senders are not lying and the original sender is trusting that nobody is snooping on their emails.

To correct that there are a whole bunch of optional add-ons to the email standard to deal with that. But that is a story for another day.

End of the Pier
Feb 272017
 

Strictly speaking how some cloud services do mail wrong, but whilst it is not all, there are still quite a few that do which is why there are no names contained within this rant.

When you have some cloud-based service send email, it makes sense for the “From” header (i.e. what sensible normal people think of as the sender address) to contain the email address of the person using the cloud-based service.

Fair enough.

But if the real sender address or envelope sender address (which is contained within the SMTP transaction) comprises the email address of the person using the cloud-based service you may well run into problems. Many organisations publish an SPF record in their DNS to indicate what network addresses are approved for, and many mail servers check the envelope sender against the published authorised network addresses.

If the network address used by the cloud service provider does not match what is in the organisation’s SPF record then the recipient’s mail server is free to reject the mail. And they often do.

Now the most obvious “fix” for this is to add the cloud service provider’s network address to the organisation’s SPF record.

The only trouble with that is that it isn’t always possible. There are various limits to how long an SPF record can be so adding addresses to the SPF record recklessly is unwise, and a sensible DNS administrator will only add to the SPF record for important services. So if the cloud service is being evaluated or being used by something less important, or is being used for non-work related purposes, then it likely won’t meet the “important enough to get added to the SPF record” criteria.

So why not fix the source of the problem?

All that has to be done is to use a different address for the envelope sender, and you can even arrange things to send bounces back to the right place.

Set the envelope sender to something like “customer+${original email}@${cloud service address}” (obviously when replacing the ${original email} you will have to change the “@” sign to something reversible). All of a sudden you are no longer “forging” the envelope sender, and not tripping over anyone’s spam defences.

Process the bounces to “customer@${cloud service address}” and you can send the bounces to the right place.

Nov 132015
 

There was a story this morning about how normal people find a constant flood of emails very stressful because it is constantly interrupting whatever they are doing. As someone who has been emailing since 1987 (with a six month break), I find dealing with it less stressful than phone calls and desk visits because you can leave it until later.

It is definitely true that being interupted whilst trying to concentrate on some tricky piece of work can be somewhat stressful (and it is cumulative). One aspect that was not mentioned is that you have to discard everything you were thinking of and bring some other subject to mind; not always the easiest thing to do especially if you are deep in concentration.

Of course it is also unavoidable, and not always unwelcome. But back to emails,

Don't read emails when they arrive, and if you have a ping that announces each email that arrives, turn it off. Check your inbox regularly – on the hour, every half hour, or even every 15 minutes (although that is perhaps too frequent for when you are concentrating on something). You can change the frequency based on what you are doing – if you're concentrating on a project or for a deadline, check less often.

The point is you choose when to deal with emails.

Another source of stress is the amount of emails in your inbox; your inbox is not a "todo" list. Create a "todo" folder and move emails from your inbox into it, and concentrate on keeping that folder reasonably well processed. Your inbox becomes a cesspool of spam (there's always some that gets through), near-spam, and general information. Of course you can get much more organised than this!

And make your own decision on the priority of tasks given through emails. You may agree with the priority a sender sets on a task, or you may disagree, but you decide. 

Ultimately the more control you take over your email, the less angst it causes you.

damascus-unix-prompt

Mar 282013
 

This article is short on references because I haven’t gotten around to filling them in … they will come

The fuss in the mainstream media about the distributed denial of service (DDoS for short) attack against Spamhaus goes to show that journalists need to buy more drinks for geeks, and the right geeks. It is nowhere near as bad as described, although the DDoS attack was real enough and definitely caused “damage” :-

  1. New York Times: http://www.nytimes.com/2013/03/27/technology/internet/online-dispute-becomes-internet-snarling-attack.html?pagewanted=all&_r=0
  2. Daily Mail:  http://www.dailymail.co.uk/news/article-2300810/CyberBunker-revealed-Secretive-fanatic-worst-cyber-attack.html

This article is not intended to be totally technically accurate in every detail; it is intended to describe the incident in enough detail and with enough accuracy that it can be understood without übergeek status.

So What Happened?

Spamhaus are experiencing on ongoing distributed denial of service attack that started on the 20th March, and is ongoing. The initial attack very quickly overwhelmed their 10Gbps (that’s about 1,000 times faster than your Internet connection) link to the Internet. Whilst this disrupted the Spamhaus web site, and various back office services, the main service that Spamhaus provides kept running (as it is distributed).

The very clued up geeks at Spamhaus who have had plenty of experience of being under attack, very quickly contacted CloudFlare which started hosting their web sites and other back office services at numerous data centres around the globe. Their services rapidly started returning to life – it isn’t the sort of thing that can be done instantly, and probably took a lot of late nights.

However the attacks escalated and reached levels of up to at least 300Gbps (that’s about 30,000 times faster than your Internet connection) or about 13Gbps of traffic for each of CloudFlare’s 23 data centres. That’s a lot and could be responsible for Internet slowdowns …

The Internet Is Slow. Is It The DDoS?

Well perhaps. We all have a very understandable tendency to blame known events for problems we’re having. Is the Internet slow? It must be that DDoS . But it is not necessarily so.

And if all the Internet was slow for you, it is quite possible that you were unknowingly taking part in the attack! Because the attack relied on infected PCs together with other stuff described below.

It is also possible that some parts of the Internet were overwhelmed by the DDoS. Reports have indicated that Internet services plugged in alongside the CloudFlare data centres (or in them) were suffering somewhat because of the extraordinary levels of traffic. However, this is the Internet and there is always lots of stuff going on that may cause slower performance than normal in various corners of the ‘net.

Was This The Biggest DDoS Attack?

Possibly. The figure of 300Gbps (and it was probably larger than that – the 300Gbps figure was through one Tier-1 ISP) probably qualifies as the largest known public DDoS.

However DDoS attacks are not always made public; there could well have been larger attacks that were not made public.

Various responses have indicated that the attack was not as serious as described by others :-

  1. http://cluepon.net/ras/gizmodo
  2. http://gizmodo.com/5992652/that-internet-war-apocalypse-is-a-lie

It may be that these commentators are mistaken to the extent that they didn’t see a problem; it may be that European and Asian networks were more prone to a slow-down than elsewhere.

What Is A Distributed Denial Of Service Attack?

If you were an attacker, you could try sending network traffic as fast as your PC could handle to the target of your attack. However the amount of traffic you could send would be very limited – you can’t send more than the speed of your Internet connection. Say 10Mbps … a lot less than most large services use for their own Internet connections.

To make an attack more effective, you will want to have lots of people send traffic as quick as they can. And the easy way to do that is to infect PCs with some sort of malware, and use your control of those infected PCs to send out that denial of service traffic. At which point it becomes a distributed denial of service attack because the attack traffic is distributed around the Internet.

And if you can find some way of amplifying your attack traffic so that say 10Mbps of traffic becomes 1Gbps of traffic, you make your attack much more effective.

So How Was This Done?

The details of what went on become pretty hairy very quickly, but very simply :-

  1. The attacker takes control of a large number of infected PCs to make his or her “robot army” to send out network traffic under their control.
  2. The attacker instructs their robot army to send out DNS requests as quickly as possible with the source address forged as the victim’s address.
  3. The negligent ISP allows those packets out by not applying source filtering.
  4. The network traffic reaches any number of misconfigured DNS servers that answer with a larger reply sent to the victim’s address.

DNS?

This is short for the domain name system and is a service that turns names into numbers (amongst other things). You type in a name such as www.google.com and the DNS server your PC is configured to talk to turns that name into an Internet address such as 203.0.113.63 or possibly 2001:db8:0:1234:0:5678:9:12. Your PC then makes a network connection to that numeric address in the background, and fetches a web page, a music stream or some other content you want.

Without the DNS we would all have to rely on numeric addresses to make connections – a lot tougher!

There’s another factor here as that DNS is an amplifying service – you ask for a name such as www.google.com, and the answer is a whole lot longer than just the numeric address you “need” as it can (and often does) contain a number of network addresses together with associated information :-

% dig www.google.com  

; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> www.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;www.google.com.			IN	A

;; ANSWER SECTION:
www.google.com.		61	IN	A	74.125.138.104
www.google.com.		61	IN	A	74.125.138.106
www.google.com.		61	IN	A	74.125.138.99
www.google.com.		61	IN	A	74.125.138.147
www.google.com.		61	IN	A	74.125.138.105
www.google.com.		61	IN	A	74.125.138.103

;; AUTHORITY SECTION:
google.com.		126160	IN	NS	ns3.google.com.
google.com.		126160	IN	NS	ns2.google.com.
google.com.		126160	IN	NS	ns4.google.com.
google.com.		126160	IN	NS	ns1.google.com.

;; ADDITIONAL SECTION:
ns1.google.com.		126160	IN	A	216.239.32.10
ns2.google.com.		126160	IN	A	216.239.34.10
ns3.google.com.		126160	IN	A	216.239.36.10
ns4.google.com.		126160	IN	A	216.239.38.10

;; Query time: 1 msec
;; SERVER: 10.0.0.26#53(10.0.0.26)
;; WHEN: Sat Mar 30 09:52:59 2013
;; MSG SIZE  rcvd: 264

If you are talking to a misconfigured DNS server, it could answer even when it should not. Normally DNS servers are configured to answer just for those they are intended to provide answers to – your ISP’s DNS servers will answer your questions, and not mine. However if they are misconfigured, they will answer any question and will function as a DDoS amplifier.

This does not include public DNS servers such as OpenDNS, or Google’s public DNS servers – they are specially configured to avoid acting as a DDoS amplifier – probably by imposing a rate limit to stop answering if you ask too many questions.

Source Filtering?

When you click on a link in your web browser, your browser sends out a network packet containing the request (“GET /webpage”), and that network packet contains the destination of the web server – so your request reaches it, and your own address – so the web server knows where to send the answer! Your own address (in these circumstances) is known as the source address.

With appropriate software, you can forge your source address so that replies to your request go back to a different place. Without that only the very simplest DDoS attacks would work.

Of course, it has been best practice to block forged source addresses since well, not long after the beginning of the Internet. This is known as source filtering. An Internet router is capable of deciding that packets coming in from wire A should not have the address assigned to wire B, so should be dropped on the floor.

An Internet router that doesn’t do that is poorly configured.

So How Can This Be Stopped?

The answer is that we have known how to stop this sort of attack for at least a decade. And indeed the best Internet citizens have done so for years.

The trouble lies with those on the Internet who are not necessarily the best Internet citizens. Of the big three remedies, two are probably being neglected because managers of ISPs do not see the business benefits of applying those remedies. And there isn’t a business benefit, but a social responsibility.

The three remedies are :-

  1. The average Internet user needs to take action to prevent their PC from getting infected. Get anti-virus protection, and an Internet firewall. If the PC acts weird, get it looked at. And if the Mac acts weird, get it looked at too (yes they do get infected).
  2. ISPs should apply BCP38 (which dates back to 2000) which specifies source filtering.
  3. ISPs running DNS servers should ensure that their DNS servers are properly configured to only answer queries for legitimate clients.

And if you happen to know a senior manager at an ISP, ask them about BCP38 and if they’re doing it – source filtering is probably the most important action here.

But Who Is Responsible?

It is easy to get distracted by the problems caused by those leaving poorly configured router, and insecure PC lying around on the Internet. Whilst their owners are responsible for effectively leaving tools around that attackers can use (and all too often do use), they are not directly responsible for the attack.

The attacker is.

But who were they?

The fairly credible rumours are that the attackers were either Cyberbunker or Stophaus.com, as part of a campaign against the actions of Spamhaus. Various criminals behind the flood of spam targeting your mailbox with all sorts of rubbish have long complained about the actions of Spamhaus, as they try and prevent spam arriving. And Cyberbunker is an ISP dedicated to providing hosting to services that may get shut down elsewhere – they deal with anyone except paedophiles and terrorists, which leaves a whole world of swamp dwellers that you would really rather not know about. And spammers.

Who Are Spamhaus?

Spamhaus are subject to a great deal of black propoganda – including accusations of blackmail, extortion, censorship, and probably kicking cats too. The reason? They help identify spammers, so that ISPs can choose to block spam.

Spammers are somewhat irritated by this – their business model relies on polluting your mailbox so that the 1% (or so) of people who do respond to spam is a large enough number that they can carry on making money. And they get irritated very quickly if someone tries to interfere with their “right” to use your inbox to make money.

Mail server operators have long been blocking spammers using a whole variety of methods, and some of the best collaborated on producing lists of addresses of spammers that others could use. These evolved into DNS based RBLs, and one of the most respected groups of volunteers became known as Spamhaus.

You may be thinking that you still get plenty of spam, so they cannot be doing too great a job. But :-

  1. You may be with an ISP that chooses not to use Spamhaus.
  2. You don’t see the spam that gets blocked. Even if you see dozens of spam messages a day, you may be seeing only 5% of the spam that was sent your way.

It is telling that amongst those in the know, those who deal with spam and Internet abuse in general, there is practically nobody who thinks of Spamhaus as anything other than the good guys.

 

Sep 092011
 

I was alerted to this by an article on The Register which points to the Godai Group‘s investigation into what happens when you register domains “close” to a reputable company and grab all the emails that happen to pop by. It is hardly a surprise to anyone who has run an email system, but you will get tons of email delivered caused by email address typos. Specifically Godai Group looked at a specific type of typo – accidentally leaving out a “.”. For example, one of the domains that the Godai Group picked up on was some-person@ca.ibm.com where “someone” has registered caibm.com (no dot) … whether or not that person is sniffing those emails cannot be known, but they could.

Again, to those who have run email systems it is no surprise to learn that some of the emails contain “interesting” information not limited to :-

  • Trade secrets
  • Business invoices
  • Personal information about employees
  • Usernames and passwords!
  • Network diagrams.

What is not mentioned is that those Fortune 500 companies almost certainly have policies in place prohibiting acts such as sending passwords and other sensitive information by email. But of course there is a description for someone who reads all of the corporate policies – someone who isn’t doing their job!

There is an interesting list of mitigations in the Godai Group report, but it could be a lot more extensive :-

  • When sending out an email to an address where the left hand side would be a valid internal address, flag the destination in your logs. Use that information to build up a list of domains for which you should check for valid internal addresses and freeze (hold in the queue) any messages that match. As an example, if mike.meredith@ca.ibm.com were a valid address you might want to freeze any emails addressed to mike.meredith@caibm.com.
  • Use your email logs to build up a database of domains that you send email to. This will allow you to identify similar domains that may be practicing so-called “doppleganger domains” that you may want to take some action against. You may think you can guess what the domains would be, but there is a lot to be said for hard evidence.
  • Perform content filtering on outgoing email, and build up a set of rules to catch emails containing patterns that match certain kinds of emails you do not want leaving your organisation – to begin with a pattern matching “password [is] XXXXXX”. This could take considerable effort to build, and there will always be the chance of a false positive so you will want a sensible warning message when emails matching the relevant content filter get caught – “Please check that this email does not contain confidential information; please check the recipient address, and if necessary re-phrase the email”.
  • Encourage the use of end-to-end encryption such as PGP. Plain encryption is not sufficient – “walled garden” email systems such as GroupWise support encryption for internal emails, but this is about external (even if it isn’t intentionally so) email which is not encrypted with such corporate email systems. In fact systems such as GroupWise may be considered dangerous in this context – it comes with the word encryption on the tin, and even allows you to “take back” emails that you have sent that you regret. These facilities encourage dangerous practices.
  • Education, education, education. But this will not accomplish much – not only are the people who really need to be educated not listening, but these problems are mistakes – both in terms of accidentally sending emails to the wrong address, and in terms of emailing information that should probably not be sent via email.
  • Lastly, and perhaps for amusement value, you could try persuading senior managers that the danger of them sending inappropriate information accidentally out to third parties via email is so great that it justifies setting up a process by which all their email sent to external address is manually reviewed to ensure that it is not an accidental release of internal information. Good luck on that one!
WP2FB Auto Publish Powered By : XYZScripts.com

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close