Debian: Adding Entropy to /dev/random

 Computing, Linux, Security  1 Response »
Oct 172012
 

I have recently become interested in the amount of entropy available in Linux and decided to spend some time poking around on my Debian workstation. Specifically looking to increase the amount of entropy available to improve the speed of random number generation. There are a variety of different ways of accomplishing this including hardware devices (some of which cost rather too much for a simple experiment).

Eh?

Linux has a device (/dev/random) which makes available random numbers to software packages that really need access to a high quality source of random numbers. Any decently written cryptographic software will use /dev/random (and not /dev/urandom which does not generate “proper” random numbers of quality) to implement encryption.

Using poor quality random numbers can potentially result in encryption not being secure. Or perhaps more realistically, because Linux waits until there is sufficient entropy available before releasing numbers through /dev/random, software reading from that device may be subject to random stalling. Not necessarily long enough to cause a major problem, but perhaps enough to have an effect on performance.

Especially for a server in a virtualised environment!

Adding Entropy The Software Way (haveged)

HAVEGED is a way of using processor flutter to add entropy to the Linux /dev/random device. It can be installed relatively easily with :-

apt-get install haveged
/etc/init.d/haveged start

As soon as this was running the amount of entropy available (cat /proc/sys/kernel/random/entropy_avail) jumped from several hundred to close to 4,000.

Now does this increased entropy have an effect on performance? Copying a CD-sized ISO image file using ssh :-

Default entropy 29.496
With HAVEGED 28.636

A 2% improvement in performance is hardly a dramatic improvement, but every little bit helps and it may well have a more dramatic effect on a server which regularly exhausts entropy.

Checking The Randomness

But hang on … more important than performance is the randomness of the numbers generated. And you cannot mess with the generation of random numbers without checking the results. The first part of checking the randomness is making sure you have the right tools installed :-

apt-get install rng-tools

Once installed you can test the current set of random numbers :-

dd if=/dev/random bs=1k count=32768 iflag=fullblock| rngtest

This produces a whole bunch of output, but the key bits of output are the “FIPS 140-2 failures” and “FIPS 140-2 successes”; if you have too many failures something is wrong. For the record my failure rate is 0.05% with haveged running (without: tests ongoing).

Links

… to more information.

Understanding URLs Or How To Avoid Annoying Your Scary DNS Administrator

 Computing  No Responses »
Oct 162012
 

Sometimes I get surprised by how many people do not fully understand how URLs work … or more specifically how they are decomposed and what each part means. And not just people who have no real reason to understand them, but people in IT. As a DNS administrator (amongst other things) I get some surprising requests – surprising to me at least – which involve explaining how I would like to help, but accomplishing the impossible is a task somewhat above my pay grade.

With any luck (so probably not then), this little post may go some way towards explaining URLs and what can and cannot be accomplished with the dark arts of the domain name system.

To start with, URLs can be thought of as web addresses. Not the kind you find painted on the sides of vans (www.plumbers-are-us.com) but what they turn into in the location bar with an honest web browser when you visit a site. Such as http://www.plumbers-are-us.com/. Although I note that my own browser is less than honest!

But just to make things a little more interesting, I will make that example URL a little more interesting: http://www.plumbers-are-us.com:8080/directory/portsmouth.html.

And now to the dissection. The first part of that URL above is the http bit … to be precise that which appears before the two slashes (apologies if you have been deceived by Microsoft but a ‘/’ is a forwards slash and a ‘\’ is a backwards slash, although those formal graphemologists who write the standards prefer to call a slash a solidus). This part of the URL is the scheme.

The scheme defines what protocol should be used to fetch a page with. You should be familiar with http and https as these are conventionally used to fetch web pages … with the later involving SSL encryption of course. There are of course other schemes less well known :-

ftp File Transfer Protocol – a pre-web method for transferring files.
gopher Gopher – an earlier competitor to the Web.
mailto Used to compose a mail message to an address.

In fact that is just a tiny sneak peak at the full list which contains a number of things even I have never heard of. But the usual scheme is either http or https (at least for now), so we can skip over the scheme part.

The next part (between the ‘//’ and the next ‘/’) contains two items of information :-

  1. The “hostname” where the web server can be found.
  2. The “port” to attach to on that web server.

The “port” is relatively uninteresting. If the server where the URL is served from is configured properly, there is no need to specify a port number, as any browser is capable of realising that the default port number for http is 80 (computers are good with numbers after all) and 443 for https. Unfortunately, whilst there is (arguably) no real excuse for running web servers on non-standard ports these days, some people insist on doing the Wrong Thing; quite often through archaic knowledge picked up during the 1990s which would be best recycled.

The “hostname” part is where it starts to get interesting. This is turned into an IP address by your browser, so it can go off across the Internet and have a polite conversation with a web server at the other end to ask nicely for a copy of the web page you have asked for. You can just put an IP address in there, but the expectation is that sometimes URLs may be typed in, and isn’t really.zonky.org slightly more memorable than 2001:8b0:640c:dead::d00d ?

But wait! It gets more interesting: The DNS allows you to point more than one name at a server, so mine can be reached with several different URLs such as http://zonky.org and http://really.zonky.org plus a few others. Which in fact show different web pages, by using so called virtual servers (which has nothing to do with virtual machines).

So the DNS can be used to change a boring server name such as server0032.facilities.north.some.organisation into a more meaningful name such as internet.some.organisation, but it can only pull tricks with the “hostname” part. Any messing with any other part of the URL including the bit after the slash is the job of something else; usually the web server itself, although that can sometimes require additional support.

The last part of the URL comes after the first single slash – in our example the “/directory/portsmouth.html” part – which can be best called the pathname as it provides a path to the page within the web server to fetch. In a very simplistic way, web servers can be thought of as file servers which require you to tell it which file to request; just like working with the command-line on a Linux machine or even a Windows machine.

BTW: I’m not really that scary – I haven’t bitten anyone’s head off for ages … at least a couple of weeks at least!

Those Pesky Backups …

 Computing  No Responses »
Oct 152012
 

Backups are funny things … everyone says that they’re important, but actually it is restores that are important. Backups merely make restores possible. Because restores are so infrequent (and frankly backups so boring) there are far too many of us who do not spend enough time making sure everything is backed up properly.

This is not a blog entry saying how backups should be performed in the technical sense … I’m not going to suggest you use rsync to an offsite cloud storage provider, or how frequently the backups should be made. But rather a random wandering around the problems of backups.

Before I forget, I will be using a shortcut: When I say “to tape”, it merely means copying the backup to whatever medium you use. I’ve merely been doing backups long enough that “to tape” just flows naturally. In fact my current backup media is an external hard disk attached to an off-site server.

What Do You Want To Restore?

Before you can decide what you need to backup, you need to decide what you might want to restore. And what types of data you might want to restore. You can split up restores into three broad categories – the system data, the user data, and any databases there might be. Each has slightly different requirements – you might well back them all up in the same way, but it may also be better to back them up in three different ways.

Restoring Systems Data

If you backup the operating system of your computer using the same mechanism intended for backing up user data, then in a disaster situation you will be faced with the interesting situation that the data you need to make your computer functional again will be sat on a tape, an external disk, or (even worse) sitting on a patch of this cloud stuff that is currently so trendy.

Whilst this is perhaps not the ideal situation to be in, it is also not the end of the world. At least if you are not pushed for time to get the computer back up and running again.

There are basically two options here – either a dedicated computer imaging solution that clones your operating system disk in some way, or to use the original installation DVDs as a restoration method. The later may be the cheap option, but it does work to the extent of at least getting you going again. And of course lets you into those other backups you have made.

The decision on which to go for comes down to time – how long would it be acceptable for you to be without your data? Bearing in mind that the restore can only start after you have the hardware to restore it to.

Restoring Databases

When it comes to restoring from backups, databases can be a touch fragile. However it is worth pointing out that by this I mean real databases such as Oracle, PostgreSQL, or MySQL rather than database applications such as Microsoft Access. If you just copy the files making up a database to a backup tape, then the result can probably be restored but you may well end up with a corrupt database, and it may be missing some data.

The classic method of backing up a database is to shut it down, and then copy the files to tape. That is a pretty safe way of doing things, but if you are trying to run a 24×7 service, then it is rather inconvenient. That is not to say it is not still a good method – simply accept the need to shutdown the database once a week and concentrate on methods to minimise that downtime (filesystem snapshots work brilliantly here).

There are also database specific methods of generating a backup whilst the database is running. The lowest common denominator here is the equivalent of an export – the database generates a whole bunch of SQL commands which when run re-create the database. These methods can be used in combination with the old shutdown and copy to tape mechanisms to double your chances of getting a good backup.

And indeed allow you to minimise the disruption by only performing a shut down and copy to tape less frequently than every night.

Of course you probably do not have a database at home that you need to keep running 24×7, but some people will. But even if you don’t care how often the database gets taken down to perform a backup, you still should spend some time making sure that your database is backed up properly. It is too late to check when you are trying to perform a restore.

Oh! And if you do shut down your database to back it up, please remember to start it again afterwards!

Restoring Ordinary Files

Backing up ordinary files is definitely on the most boring side of backups. But for most of us, they are the most important backups we perform; as more and more of our important data becomes digital rather than physical, we need to be sure that our digital data is safe. And safe does not mean just safe from the odd hard disk failure, but from disasters such as house fires!

Or from foolish decisions to delete the wrong files, etc. We tend to assume that restores are performed after disasters such as the aforementioned hard disk crash, but in practice – at least in an organisation with a team responsible for performing restores – files are restored almost continuously and far more frequently than hard disk crashes.

You can choose to backup everything – which means you can be sure that you have everything you need to restore in an emergency, although it can be a lot slower as you will be backing up a lot of “junk”, or you can be very selective in what you backup which makes things a lot quicker, but there is always the danger that something important will be lost.

Or you could do both! It is perfectly sensible to backup only the most important files every day – perhaps to DropBox – and then do a full backup once a week.

One thing to look for is something along the lines of Apple’s Time Machine; there are approximate equivalents for Windows and Linux, and the key advantage that all of them has, is the ability to perform differential backups which means that only the changed files are copied. My own backup script ran last night and ‘refreshed’ a backup of nearly 500Gbytes in about 7 minutes (and that was to a very remote server).

And use those backups! Checking whether the backups have worked or not is another tedious job, but using yesterday´s files is far less tedious.

A Few Misconceptions

  1. RAID isn’t a backup method. You can mirror your hard disks (I do), but that merely reduces the probability of a hard disk crash causing you to reach for the backup tapes. That is not to say that it is worthless, but you still need to perform backups.
  2. Tape isn’t dead. It may well be too expensive for home use, but tape is still a perfectly sensible way of keeping backups. It can be “enhanced” in various ways such as snapshots to give the impression of backups being performed very quickly, or a disk buffer to keep the most recent backup online.
  3. Cloud backup solutions are cool, but not without issues. For a start you have to worry about the legal aspects (especially if you are a business), such as whether the backups are stored within an acceptable jurisdiction. In addition, what happens if the cloud storage provider goes out of business for some reason ? There are quite a few people who could tell you the problems of using certain cloud storage providers who have for one reason or another ceased operation.
  4. A backup on the same disk as the source files may well be a poorly considered option. After all, it will not help you if the hard disk goes “bang”. But it could be quite a good supplementary option to another method. Similarly an external disk is all very well, but will not help you much in the case of a house fire.
  5. Whatever backup method you choose is subject to failure. The external hard disk that fails just when you need it, the encrypted cloud backup where you’ve forgotten the passphrase because it’s held in a password store on the disk you’re trying to recover, etc. Having multiple backup destinations is worth considering especially when so many cloud storage providers are giving so much space away for free.

Debian: Booting Floppy Images

 Working Notes  No Responses »
Oct 152012
 

It is very convenient to be able to boot floppy disk images or CD images directly from Grub without having to write the images to their associated media … not least because floppy disk drives are getting quite long in the tooth. And in fact I haven’t had a floppy disk drive installed for a while now. As it happens it is very easy to do :-

sudo zsh
apt-get install grub-imageboot
mkdir /boot/images
# Move any images (CD or floppy) to /boot/images
# With either the extension .img (for floppies) or .iso (for CDs)
update-grub2

… and if you use this to update your BIOS, it is worth writing down all the key settings before upgrading!

The Sensible iPhone 5

 iPhone  No Responses »
Oct 062012
 

Now that all the fuss over the new iPhone5 has calmed down a bit, it is time to take a look at the iPhone5. Okay, so what fuss was there amongst anybody other than the die-hard Apple fans who would buy any iPhone5 without considering whether or not it was carved out of a pile of petrified rhino bile? Well of course there was the media crowd jumping up and down in excitement at the prospect of an expenses paid trip to somewhere exotic.

But not many others.

It isn’t as if the iPhone5 were a bad phone; when you come down to it, it is a sensible upgrade from the iPhone4S. The key new features are :-

  • Most “exciting” of all (which is itself a bad sign) is the new dock connector which effectively makes all previous iPhone add-ons redundant. Understandably this has annoyed many people, but it had to be done sooner or later. After all the old connector was nearly 10 years old which is positively geriatric in the technology world.
  • The processor was a little bit faster, and there is a little bit more memory.
  • The screen is “bigger” … or rather taller. Nice enough I suppose, although it is hardly a Galaxy SIII or a Galaxy Note (1 or 2).
  • Apple have realised that there are a few people outside North America who might want to use 4G network speeds, and so their LTE support covers more frequency bands. Although despite having three variants of the iPhone5 which cover different frequency bands, they still cannot offer LTE in all markets. And of course having three different phones not only makes manufacturing more costly, but prevents customers roaming so easily as they could do with the iPhone4S.
  • The new phone is thinner and lighter than the old iPhones, although those who have encountered problems with the aluminium case getting scratched may prefer the old weight of the stainless steel case which was more robust in this sense.
  • And of course the iPhone5 now uses the even fiddlier nano-SIM.

Ignoring the software side of things, this all looks a little depressing. All very sensible, but one person’s sensible is another person’s resting upon your laurels. And if you keep that up, sooner or later someone comes along and tips you into a muddy ditch.

Just ask Nokia.

 Older Entries  Newer Entries