IT: Why ZFS In OSX Is Good For “Ordinary” Users

 Computing  No Responses »
Nov 072007
 

I have been spending some time looking up information on ZFS for OSX because I’ve used ZFS under Solaris and would quite like it on my new Macbook. In many of the places I looked, there were tons of comments wondering why ZFS would be of any use for ordinary users. Oddly the responders indicating features that are more useful for servers than workstations. The doubters were responding with “So?”.

This is perhaps understandable because most of the information out there is for Solaris ZFS and tends to concentrate on the advantages for the server (and the server administrator). This is perhaps unfortunate because I can see plenty of advantages for ordinary users.

I will go through some of the advantages of ZFS that may work for ordinary users. In some cases I will give examples using a command-line. Apple will undoubtedly come up with a GUI for doing much of this, but I don’t have access to that version of OSX and the command-line still works.

ZFS Checks Writes

Unlike most conventional filesystems, ZFS does not assume that hard disks are perfect and uses checks on the data it writes to ensure that what gets read back is what was written. As each “block” is written to disk, ZFS will also write a checksum; when reading a “block” ZFS will verify that the block read matches the checksum.

This has already been commented on by people using ZFS under Solaris as showing up problematic disks that were thought to be fine. Who wants to lose data ?

This checksum checking that zfs does will not protect from the most common forms of data loss … hard disk failures or accidentally removing files. But it does protect against silent data corruption. As someone who has seen this personally, I can tell you it is more than a little scary with mysterious problems becoming more and more common. Protecting against this is probably the biggest feature of ZFS although it is not something that is immediately obvious.
ZFS Filesystems Are Easy To Create

So easy in fact that it frequently makes sense to create a filesystem where in the past we would create a directory. Why? So that it is very easy and quick to see who or what is using all that disk space that got eaten up since last week.

Lets assume you currently have a directory structure like :-

/Users/mike
/Users/john
/Users/stuart
/Users/stuart/music
/Users/stuart/photos

If those directories were ZFS filesystems you could instantly see how much disk space is in use for each with the command zfs list

% zfs list
NAME                                 USED   AVAIL   REFER   MOUNTPOINT
zpool0                               3.92G  23G     3.91M   /zpool0
zpool0/Users/mike                    112M   23G     112M    /Users/mike
zpool0/Users/john                    919M   23G     919M    /Users/john
zpool0/Users/stuart                  309M   23G     309M    /Users/stuart
zpool0/Users/stuart/music            78G    23G     78G     /Users/stuart/music
zpool0/Users/stuart/photos           12G    23G     12G     /Users/stuart/photos

With one very simple (and quick) command you can see that Stuart is using the most space in his ‘music’ folder … perhaps he has discovered Bittorrent! The equivalent for a series of directories on a normal filesystem can take a long time to complete.

With any luck Apple will modify the Finder so that alongside the option to create a new folder is a new option to “create a new folder as a ZFS filesstem” (or something more user-friendly).

It may seem silly to have many filesystems when we are used to filesystems that are fixed in size (or are adjustable but in limited ways), but zfs filesystems are allocated out of a common storage pool and grow and shrink as required.

ZFS Supports Snapshots

Heard of “Time Machine” ? Nifty isn’t it ?

Well ZFS snapshots do the same thing … only better. Time Machine is pretty much limited to an external hard disk which is all very well if you happen to have one with you, but not much use when you only have a single disk. ZFS snapshots work “in place” and are instantaneous. In addition you can create a snapshot when you want to … for instance just before starting to revise a large document so that if everything goes wrong you can quickly revert.

Time Machine has one little disadvantage … if you modify a very large file, it will need to duplicate the entire file multiple times. For instance if you have a 1Gbyte video that you are editing over multiple days, Time Machine will store the entire video every time it ‘checkpoints’ the filesystem. This can add up pretty quick, and could be a problem if you work on very large files. Zfs snapshots stores only the changes to the file (although an application can accidentally ‘break’ this) making it far more space efficient.

One thing that zfs snapshots does not do that Time Machine does, is to ensure you have a backup of your data on an external hard disk. The zfs equivalent is the zfs send command which sends a zfs snapshot “somewhere”. The somewhere could be to a zfs storage pool on an external hard disk, to a zfs pool on a remote server somewhere (for instance an external hard disk attached to your Mac at work to give you offsite backups), or even to a storage server that does not understand ZFS! And yes you can send “incrementals” in much the same way too.

Currently using zfs send (and the opposite zfs receive) requires inscrutable Unix commands, but somebody will soon come up with a friendlier way of doing it. Oh! It seems they already have!

Unfortunately I’ve found out that using ZFS with Leopard is currently (10.5.0) pretty difficult … the beta code for ZFS is hard to get hold of, and may not be too reliable. Funnily enough this mirrors what happened when Solaris 10 first came out … ZFS was not ready until the first update of Solaris 10!

Unfortunately it seems that Apple have retreated back from using ZFS in OSX which is a great shame, and until they come up with something better, we are stuck with HFS+, which means not only do we lack the features of a modern filesystem, but we are also stuck with slow fsck times. Ever wonder why sometimes that blue screen of a Mac starting sometimes takes much longer ? The chances are that it is because a filesystem is being checked – something that isn’t necessary with a modern filesystem.

Wildcat Postal Strikes

 Politics  No Responses »
Oct 102007
 

Like most people in the UK, I am suffering from a lack of postal deliveries because of an official strike that ended today (with rolling strikes due to start next week) … I have several parcels stuck waiting for delivery and it is more than a little frustrating! I was more than a little surprised (and initially annoyed) when the early morning news announced at least one wildcat (unofficial) strike taking place.

Fortunately the media let slip a little detail about why the wildcat strikes started. It seems that the post office managers had changed the working hours without talking it over with the union first. Now perhaps many people reading this will think the managers had every right to change the working hours without negotiating with the work force … personally I disagree, but I am not going over that issue.

The postal workers returned to work at 5:15am (or a similar time … it has been a few hours now) probably in a bit of a militant mood (I’ve been on strike myself and it has that effect), but mostly also keen to get on with dealing with the large piles of unprocessed post. Only to be told by the managers that the hours had changed and they would not get paid for the work done before 6:00am.

Now in normal circumstances the workers would have been prepared for this change … they may not have been happy about it, but they would know and would probably casually mention it to each other on the way home the previous night. Anyone who forgot would probably just slap themselves on their forehead and think “how dumb am I?” (I’ve done something very similar myself).

But these are obviously not normal circumstances. Communications between workers and managers tend to break down during a strike, and the workers may not have been aware of the change or they could have thought that it would be quietly dropped, or simply forgotten about it. And anyone with any sense would see that quietly dropping the change in working hours for now would be diplomatic.

Again there are those who say that the managers had the “right” to change the working hours, and again that’s not the point. The point is that making workers who are keen to get stuck into that big pile of letters and parcels (and most probably were keen) either wait for nearly an hour or to work for free is not likely to encourage good worker-management relations. And in these particular circumstances is likely to provoke exactly what we have seen … wildcat strikes.

Whilst the wildcatters probably deserve a bit of condemnation for what they are doing, it would seem that gross stupidity on the part of the management also deserves some of the blame.

Glasses Wearers Can Do Things

 General  No Responses »
Oct 102007
 

I recently saw a TV ad for laser eye correction surgery which used clips of people doing all sorts of exciting things … riding bikes on rough terrain, paragliding, swimming, etc. The whole implication was that as soon as one gets the surgery, one can suddenly take part in all sorts of exciting activities that were not possible before.

Eye surgery is not something I want to condemn … undoubtedly it can correct eyes to the point where glasses are unnecessary. Personally I am too squeamish to go for it, but I can see the attraction. But I do object to the idea that people who wear glasses are somehow excluded from exciting and active activities because of their glasses. Glasses do not somehow make an active life impossible or even difficult.

Admittedly ball games can be a little tricky without a bit of preparation (such as plastic lenses), but taking off the glasses does not suddenly empower anyone.

It may be that some wearers of glasses do not agree with this, but perhaps most of those are people who have had glasses inflicted on them half-way through their lives; those who have worn glasses all our lives do not think this way.

Alcohol Fueled Violence

 General, Politics  No Responses »
Oct 062007
 

There is no such thing as alcohol-fueled violence; there is only idiot-fueled violence. Claims about how alcohol fuels violence are regularly heard on TV and radio, and in one small sense they are right. Violence can be found in areas where a large amount of alcohol is consumed. But take a closer look …

It is not everyone who goes out drinking on a Saturday night who ends up involved in violence, or even everyone who drinks far more too much who ends up in violence. If that were the case, the level of violence in certain streets would be probably two orders of magnitude worse than it is (i.e. 2 fights would be 200 fights; 10 fights would be 1000 fights). It is only a tiny minority of idiots who cause problems with violence.

These idiots may blame the alcohol, or they may even only become idiots when they have consumed large quantities of alcohol. Either way, the alcohol is not the problem, it is the idiots themselves.

Most of those who complain about the violence want to fight it by doing something about alcohol sales … make it more expensive, stop selling to people who’ve had a few, etc. But why should the majority who drink be punished because of the idiots ? That is not to say that the violence should not be tackled in some way … and the majority of drinkers would be happy to see something done about it. After all we have more interest in seeing something done than those who prefer to stay at home.

One simple idea is to ‘tag’ those who get involved in violence (both sides … one person may be obviously to blame but I’ve seen myself people who go around trying to provoke a violent response) with an electronic tag, and give them on the first occasion a month’s ban from all drinking centres. For a second offense, it becomes two months, then four, and so on, doubling each time. This would be in addition to any other punishment the law may enforce … a prosecution is (and should be) difficult to get a result from, but a simple curfew could be given without so much evidence.

To enforce the curfew, give each doorman a tag detector so they can easily scan people going into a pub or bar, and prevent them.

Don’t punish the drinkers; punish the idiots.

The BBC TV License

 Politics  No Responses »
Sep 292007
 

One of the things that periodically surfaces to my attention is the debate over the TV license that those in the UK pay. This funds the BBC including TV, radio and Internet activities (there are ‘fringe’ activities which are not funded by the license fee such as the BBC’s World Service. The notion of a tax on watching television is archaic and unfair to commercial broadcasters. However it should stay.

Why?

The first reason for keeping the license fee is that the BBC actually does quite a good job. Not all of it’s activities please everyone, but that is impossible goal to achieve and it is not something that the BBC should be trying to do anyway. Look at BBC News. This is the largest news organisation in the world with a well deserved reputation for probity and honest reporting … in any “big” news event, the UK population tends to switch to the BBC for news.

There are those who criticise the BBC News for impartiality … the one who makes the biggest noise is Israel who claim the BBC is anti-Israeli. Of course Israel views any kind of criticism of Israeli government actions as anti-semitic, so we can probably discount this criticism. In fact the BBC probably is not doing its job properly if it does not get criticised by Israel.

The BBC also produces a large range of original drama, documentary and entertainment programmes of high quality. In recent TV awards, the BBC was awarded 9 out of 20 awards. That sounds like the result a quality organisation would get.

There are those who say that having a well funded public broadcasting organisation is unfair to commercial broadcasters. They are right. So what? Life is unfair, get over it.

A far more subtle point is how the BBC could well be improving the quality of commercial broadcasting by existing. Thus those who only watch commercial TV (I have trouble believing anyone in the UK can actually stick to this!) are actually benefiting from the BBC without watching the BBC! It is hard to justify this claim especially if we are talking about the quality, but we can sort of see the effect on something that most people find irritating … the frequency of ad breaks on commercial channels.

When you watch American TV imports you can often see the pauses where advert breaks would have occurred had you seen the programme in the US; however over here we seem to have settled on having rather fewer breaks. Why? Well there is little commercial reason for doing so, except that if people get too irritated by ad breaks they can switch over to the BBC. So it seems that the existence of the BBC may have resulted in fewer ad breaks on commercial TV in the UK … which I’m sure most would agree is a good thing.

So we have decided that the BBC is good, but what about the license fee ? Well, the license fee is bad but it works … the BBC gets a fairly big chunk of money from it. Any change would risk how much money the BBC gets, and why break something that is not broken ?

 Older Entries  Newer Entries