The Network Time Protocol “Glitch”

 Computing, Linux, Security  No Responses »
Nov 242012
 

NTP is one of those strange services that are so vital to the operation of an organisation’s network; if the servers around the network get their time in a muddle, all sorts of strange things can start happening. Besides which most people expect their computers to be able to tell the right time.

But often it is one of the unloved services. After all no user is going to ask about the health of the NTP service. And if you are a senior manager involved in IT, do you know who manages your NTP infrastructure ? If so, have you ever asked them to explain the design of the NTP infrastructure ? If not, you may find a nasty surprise – your network’s NTP infrastructure may rely on whatever servers can be scavenged and with the minimum investment of time.

Of course, NTP is pretty reliable and in most circumstances extremely resilient. NTP has built in safeguards against against confused time servers sending wildly inappropriate time adjustments, and even in the event of a total NTP failure, servers should be able to keep reasonable time for at least a while. Even with a minimal of investment, an NTP infrastructure can often run merrily in the background for years without an issue.

Not that it is a good idea to ignore NTP for years. It is better by far to spend a little time and money on a yearly basis to keep things fresh – perhaps a little server, and a day’s time each year.

That was quite a long rambling introduction to the NTP “glitch” that I learned about this week, but perhaps goes some way to explaining why such a glitch occurred.

A number of organisations reported that their network had started reporting a time way back in the year 2000. It turns out that :-

  • The USN(aval)O(observatory) had a server that for 51 minutes reported the year as 2000 rather than 2012.
  • A number of organisations with an insufficient number of clock sources (i.e. just the erroneous USNO one) attempted to synchronise to the year 2000 causing the NTP daemon to stop.
  • Some “clever” servers noticed that NTP had stopped, and restarted it. Because most default NTP startup scripts set the clock on startup, these servers were suddenly sent back in time to the year 2000.

And a cascade of relative minor issues, becomes a major issue.

Reading around, the recommendations to prevent this sort of thing happening :-

  1. Use an appropriate number of time sources for your main NTP servers; various suggestions have been made ranging from 5 (probably too few) to 8 (perhaps about right) to 20 (possibly overkill).
  2. Have an appropriate number of main NTP servers for your servers (and other equipment) to synchronise their time with. Anything less than 3 is inadequate; more than 4 is recommended.
  3. Prevent your main NTP servers from setting their time when NTP is restarted and monitor the time on each server regularly.
  4. And a personal recommendation: Restart all your NTP daemons regularly – perhaps daily – to get them to check with the DNS for any updated NTP server names.
  5. And as suggested above, regularly review your NTP infrastructure.

Debian: Adding Entropy to /dev/random

 Computing, Linux, Security  1 Response »
Oct 172012
 

I have recently become interested in the amount of entropy available in Linux and decided to spend some time poking around on my Debian workstation. Specifically looking to increase the amount of entropy available to improve the speed of random number generation. There are a variety of different ways of accomplishing this including hardware devices (some of which cost rather too much for a simple experiment).

Eh?

Linux has a device (/dev/random) which makes available random numbers to software packages that really need access to a high quality source of random numbers. Any decently written cryptographic software will use /dev/random (and not /dev/urandom which does not generate “proper” random numbers of quality) to implement encryption.

Using poor quality random numbers can potentially result in encryption not being secure. Or perhaps more realistically, because Linux waits until there is sufficient entropy available before releasing numbers through /dev/random, software reading from that device may be subject to random stalling. Not necessarily long enough to cause a major problem, but perhaps enough to have an effect on performance.

Especially for a server in a virtualised environment!

Adding Entropy The Software Way (haveged)

HAVEGED is a way of using processor flutter to add entropy to the Linux /dev/random device. It can be installed relatively easily with :-

apt-get install haveged
/etc/init.d/haveged start

As soon as this was running the amount of entropy available (cat /proc/sys/kernel/random/entropy_avail) jumped from several hundred to close to 4,000.

Now does this increased entropy have an effect on performance? Copying a CD-sized ISO image file using ssh :-

Default entropy 29.496
With HAVEGED 28.636

A 2% improvement in performance is hardly a dramatic improvement, but every little bit helps and it may well have a more dramatic effect on a server which regularly exhausts entropy.

Checking The Randomness

But hang on … more important than performance is the randomness of the numbers generated. And you cannot mess with the generation of random numbers without checking the results. The first part of checking the randomness is making sure you have the right tools installed :-

apt-get install rng-tools

Once installed you can test the current set of random numbers :-

dd if=/dev/random bs=1k count=32768 iflag=fullblock| rngtest

This produces a whole bunch of output, but the key bits of output are the “FIPS 140-2 failures” and “FIPS 140-2 successes”; if you have too many failures something is wrong. For the record my failure rate is 0.05% with haveged running (without: tests ongoing).

Links

… to more information.

Are Julian Assange’s “Hacktivists” Becoming Mindless Thugs ?

 Computing, Media, Security  No Responses »
Aug 312012
 

There comes a moment in some violent anti-capitalist protests where genuine if illegal protest becomes mindless thuggery; for example turning from daubing slogans on the windows of the nearest bank, to throwing objects through the windows of the small independent shop next door. And you do have to wonder if those “hacktivists” who are supporting Julian Assange’s wish to be given safe passage to Ecuador have reached beyond that point.

First of all, I should point out that whilst I’m a supporter of WikiLeaks – or at least the idea of a website where whistleblowers can responsibly publish leaked material in raw form – I’m no supporter of Julian Assange in his attempt at escaping justice. A mentioned previously, I believe he should go back to Sweden to face the charges that will be made once he arrives.

But neither do I think that Julian Assange’s supporters should be silenced however mistaken they are about the situation. They have a right to protest, and I’m not even opposed to a bit of responsible “hacktivism” – in my private life I’m quite willing to go along with the ideal that sometimes it is ethical to break the law. But I also believe that the current flood of ‘hacktivism” is going just a little bit too far.

Those who have been reading just the mainstream media (and here) may be under the impression that the hacktivists have been attacking just a few places; more relevant media makes it plain that there is something more widespread. The first story mentions Cambridge University; none of the stories mentions that the hacktivists have claimed to have broken into up to 5 universities. The list of victims of this week’s surge seems to include :-

  • Up to 5 UK Universities.
  • One or possibly two UK police forces.
  • A UK recruitment agency (which just so happens to mention a couple of UK government bodies).
  • A Pakistani agency specialising in assisting students to come to the UK, or other English-speaking countries.
  • Plus a few UK government agencies.

And this list looks a little random to me.

It’s not that difficult to break into a website – even I could do it, but the question to ask is just how many websites did they rattle the doorknobs of before they found these low-hanging fruits? And it’s always worth remembering the old classic cartoon by xkcd.com :-

Of course they didn’t just widdle a picture of Julian Assange over the front page of a web site; they also broke into some databases and stole some personal information! That’s a bit more serious. And in the case of the information grabbed from the police, it’s a lot more serious.

But if you look closely at the data stolen from the UK universities involved, it becomes a little less dramatic. It would appear that the hackers have managed to break into a few databases used by various departmental web applications. Web applications often use databases as a convenient place to “stash” stuff including account details, which is what appears to have been leaked here. These account details are normally separate from any other account details (unless of course the owner of the account uses the same password), and give access only to the web application itself.

It does not appear that any core business function data has been exposed by this – i.e. the personal details of all the students for example. If it were not for Julian Assange’s name being attached to the incident, it is very likely that the media would not be interested in the story itself which would make it far less serious for the institutions concerned.

When you come down to it, Julian Assange’s real supporters should probably be a bit dismayed by this mindless thuggery – it doesn’t reflect well on their protests if it appears the best hacktivists that they can get to support them are rather on the low end of the scale. Of course a conspiracy theorist might take this as evidence that the hacktivists here are actually deliberate making the supporters of Julian Assange look bad.

LinkedIn Password Compromise: The Unanswered Questions…

 Computing, Security  No Responses »
Jun 062012
 

If you have not already heard about it, and you have a LinkedIn account, you should be aware that a large number of password hashes has been found in the wild. This means it is possible that hackers have the ability to crack your password and break into your account.

Change any LinkedIn account passwords now.

But there are still just a few unanswered questions :-

Why were the password hashes unsalted ?

Storing passwords in the clear is just about the most irresponsible thing a website operator can do, but storing passwords in hashed form without a so-called salt is also a clear indication that someone needs a slap and told to go the extra 10m. It has long been known (i.e. for decades) that using a simple password hash allows for someone to find out what the original password was.

This is why the Unix system from the 1970s used a salt to make revealing passwords harder.

Technically a salt is a few extra bits of randomness added to the hash (and included in the output) to make pre-computing the password hashes more expensive. It also obfuscates identical passwords.

So why weren’t LinkedIn salting their passwords? Couldn’t be bothered? Assumed that their systems were so secure that nobody could break in? Whatever the reason, it was not a good enough reason – allowing their site to be hacked is bad enough, but caring so little about the security of our data shows pure incompetence and arrogance.

Are We Sure These Password Hashes Belong To LinkedIn?

In a word: No. We assume it is, and there’s some evidence to support that assumption. Several bloggers (one), have posted indicating that they have checked and found that their own LinkedIn password hash can be found in the file.

So we can assume that these password hashes are from LinkedIn, and to change our password if we have an account. Perhaps this is wrong and this huge list of password hashes is just some prankster’s idea of a fun day, but this is one of those cases where you assume it is real to be safe.

But There Are No Usernames. Aren’t We Safe?

I’ve come across at least one comment indicating that because the usernames aren’t associated, there isn’t anything to worry about.

It is true that the information as released is not especially helpful – if you cracked all the password hashes you still wouldn’t know if my password was #32768, #65536, or any of the others. But you could still use that information with the help of a botnet army and enough time to let the tools like Hydra do their work.

And we do not know that the person or group who obtained this information in the first place does not have access to further information. Even if all they had access to was a database table containing just the password hashes, they will almost certainly know the frequencies of every password.

So no, we’re not safe.

Only 6.5million? I Thought LinkedIn Had 150million Accounts?

Indeed! It does seem strange that there are only 6.5million password hashes in the released file.

But those who have had a chance to poke around in the released file (including myself) have found that there are no duplicate hashes. Which would be normal in a salted password hash file, but given how woeful most people are at picking good passwords you would expect a very large number of duplicates in 150 million password hashes. Whether you would get as few as 6.5 million unique password hashes seems a touch unlikely, but possible.

Of course it may be that the person or group who grabbed this password dump in the first place only managed a partial dump for some reason.

But If The Original Leak Isn’t Fixed, Isn’t Changing Our Password A Waste Of Time?

It is certainly true that if LinkedIn hasn’t fixed their original problem, or has not implemented some form of remedial action, then it is possible that an attacker could break in with exactly the same method as they did before, and steal the passwords again. Which means we will probably have to change our passwords again – once LinkedIn finally gets around to announcing this has all been fixed.

But not changing your password now is foolish in the extreme – you should assume that the attacker(s) have your account details now.

Offline Web Application Scanners?

 Computing, Security  2 Responses »
Mar 172012
 

This is at least partially an appeal for information – if anyone knows of a web application scanner that does what I describe here, please let me know!

All the web application scanners I have come across so far seem to only try “online” scanning where the work is done by connecting to a web server using the same method as someone with a web browser would use. Or in other words the scanning tools replicate what an attacker might do. Hardly the wrong thing to do – it is probably the best method given that so much can only be determined by going through the web server.

In addition, there are also tools to scan the source code of web applications that you have written yourself. These pick out bits of the application that could do with looking at. Fair enough for a web developer, but I’m after something a bit different.

What I want is a tool that will when given the directory containing the website, will go through it looking for weaknesses like the following :-

  1. Look for problems with the permissions – such as directories and files writeable by the web server owner.
  2. Look for common applications and components – such as WordPress – and identify them, and indicate whether they’re out of date or not.
  3. Look for signs of exploits – PHP ‘shells’ and the like.
  4. Look for content that isn’t linked to as an indication that it shouldn’t be present.

Of course most people could think of a few more things to add to that list! It would be a handy additional source of information when it comes to securing a website.

 Older Entries  Newer Entries