Looking At Ubuntu 9.10 (Karmic Koala)

 Computing  No Responses »
Nov 022009
 

So I quickly install the CD in a virtual machine, login, get delayed by doing something with a passphrase, install the “guest additions”, and reboot. But why doesn’t Ubuntu include the drivers for various different virtual machine environments ? It would certainly make it easier to quickly setup a machine to test the feasibility of switching or upgrading.

One tiny little niggle with the first boot – the passphrase request to “unlock” the encrypted drive (I wanted to say “unencrypt” there but of course it doesn’t actually do that) is a little less than obvious. It should perhaps be a little more “in your face” – a popup. And of course the new Ubuntu boot straight into X might look a little prettier than the previous mechanism, but it isn’t quite “geeky” enough for me (my OSX machines are set to boot in verbose mode to scare those who think the command line is scary). This new startup is supposedly significantly quicker than previous releases; I’m afraid I didn’t notice. Perhaps it is of concern more to others than to me, but I rarely restart my machines – my somewhat less than totally reliable main machine has been up for 7 days, so a faster boot time is not of great significance to me.

I couldn’t seem to find anything to tweak the desktop effects settings. Perhaps not that important, but sort of peculiar. I dare say there’s an addon to do that.

I next dove into “Software Centre” to find a couple of applications that I use, but couldn’t. Admittedly they are somewhat towards the geeky side of things, but they are not that unpopular (zsh and enlightenment). Dropping to a terminal window and they were quickly found using apt-cache. Perhaps the Software Centre intentionally hides things to make the default list of new application choices a little less scary ? Maybe, but it needs an easily found button to say “show more”. Of course the Synaptic package manager can still be found, so this isn’t a real problem although having two ways of installing/removing software could be.

After I installed my favourite window manager (Enlightenment for now), I went on to try setting it up in my usual way with a .xsession file. No luck! It seems that Ubuntu’s version of gdm has accidentally (?) removed this functionality. This is quite a serious problem for those who like to run seriously customised environments. Perhaps not Ubuntu’s major audience, but it seems rather unfortunate to remove this functionality as it will seriously annoy those like me who prefer their own environment.

Going back to the default environment, I take a quick look at Firefox to realise that there is no Flash plugin. A dive into the Software Centre fixes that, although it would have been nice to be taken through a “wizard” when starting Firefox for the first time to suggest installing a number of proprietary extensions (and explain why they are not installed by default). Not that it does not install with a good selection already, but a browser without flash is perhaps not what people are expecting (although there are advantages in not having flash or turning it on only for those pages you want it turned on for).

Browsing through the settings, and I find the theme browser which does not really offer much choice by default – you have to install some additional themes. The choice of backgrounds is fine, although I’m not too sure why the frog was included (I chose the falling coffee). The most obvious improvement here, would be to include a hint on how to include your own photos as a background – quite possibly the first thing that many want to do! At least the Font tab defaults to using a method of rendering suitable for LCD panels (I’m not sure if this is new with the Koala).

In the keyboard preferences, the Layout options are somewhat confusing. Admittedly the number of options here is bound to make it more confusing, and those who choose “Layout Options” are likely to be self-educated to some degree. However it may be worth looking again at how the options are described. Oddly enough the Mouse preferences shows an option to “Show mouse pointer position when Control is pressed” but does not allow you to enable it!

Going through the applications, most (without extensive testing) seem fine. However Empathy (the replacement for the Pidgin instant messaging client) does seem a little on the flaky side with a few “misfeatures” – for instance the “Room” dialog box is a little immature and it is not obvious what you should do with it. Are you supposed to know some sort of “server name” ? Seems a little odd.

This may get added to when I find the time and patience to do more, but I am sure there are plenty of other far more complete looks at Ubuntu 9.10 out there!

Revisiting Unix Basics: The df Command

 Computing  No Responses »
Feb 082009
 

I was reading a comment about the df command (in relation to reserved filesystem space) and realised that the clueless newbie was right; it is odd that df does not mention reserved space. Of course it would also be wrong for df to lie about the matter too. I then realised that df is long overdue for a bit of refreshing. If you look at the typical output of the df command, you will find it inconveniently cluttered :-

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/datavg-810root
                       12G  7.8G  4.3G  65% /
tmpfs                 2.0G     0  2.0G   0% /lib/init/rw
varrun                2.0G  416K  2.0G   1% /var/run
varlock               2.0G     0  2.0G   0% /var/lock
udev                  2.0G  3.1M  2.0G   1% /dev
tmpfs                 2.0G  344K  2.0G   1% /dev/shm
lrm                   2.0G  2.4M  2.0G   1% /lib/modules/2.6.27-7-generic/volatile
/dev/sdb1             130M   36M   88M  29% /boot
/dev/mapper/datavg-opt
                      2.0G  776M  1.3G  39% /opt
/dev/mapper/datavg-810var
                      5.0G  1.4G  3.7G  28% /var
/dev/mapper/datavg-home
                      256G  116G  141G  46% /home
/dev/mapper/datavg-vmachines
                       96G   62G   35G  64% /vmachines
/dev/mapper/datavg-bragspool
                      256G  6.2G  250G   3% /var/spool/brag
/dev/mapper/datavg-herpesbackup
                       16G  4.6G   12G  29% /var/herpes
/dev/sda1             463G  147G  293G  34% /mdata
/dev/scd0             2.4G  2.4G     0 100% /media/CIVCOMPLETEEU
/dev/mapper/datavg-cdimages
                       32G  1.9G   31G   6% /cdimages
/dev/mapper/datavg-ontapsim
                       16G  498M   16G   4% /sim

Part of the problem is that df does not do quite what it claims to do … to report free space on the mounted filesystems. It also gives some (a very small amount) of additional information about the relevant filesystems … particularly the device the filesystem is mounted on. This “helps” to make the output more cluttered that it needs to be. It is possible that there are those who will argue that the device is the filesystem and not where it is mounted; they are arguably right, but when you use df you are either looking at where in the Unix file hierarchy there are places that have less space than is comfortable, or for places that have enough space to put that big file you are about to download.

Next the command itself has an obscure command to make it easier to type on a slow type-writter like terminal (those who are below a certain age will not realise that we used to comminicate with Unix machines using a terminal that was more like a printer than the screens we use today). It might be better named fsspace with an alias of diskspace for those who want to concentrate on what worries them rather than on what worries the machine.

Next why not take advantage of certain features that have crept almost silently into the command line over the last few decades ? Why not adjust the output to the width of the terminal window (look for the $COLUMNS evironment variable), spacing things out or even adding more information when you have enough space?

Finally if you were to dig around the df command a little bit you will encounter something peculiar called “inodes”. Now I know what an inode is, and I dare say quite a few people reading this will know, but if you do not, knowing how many inodes there are is not very useful information. It is relatively rare (these days) for a filesystem to run out of inodes so this information has a low priority, and why not use a term more understandable than “inodes” ?

Changing a term is something to be avoided in most circumstances which is why we still have “inode” where even the originator of the term has to guess that the “i” means “index”. I would suggest that something like “fileslots”or perhaps “fslots”

We now have the basic specification of something that should look like :-

% diskspace
Filesystem                            Size  %Used %fslots  Avail
/                                      12G    70%      3%   3.6G
/lib/init/rw                          2.0G     0%      0%   2.0G
/var/run                              2.0G     0%      0%   2.0G
/var/lock                             2.0G     0%      0%   2.0G
/dev                                  2.0G     0%      1%   2.0G
/dev/shm                              2.0G     0%      0%   2.0G
/lib/modules/2.6.27-7-generic/vola+   2.0G     0%      0%   2.0G
/boot                                 130M    29%      0%    88M
/opt                                  2.0G    40%      1%   1.2G
/var                                  5.0G    18%      0%   4.1G
/home                                 256G    52%      0%   124G
/cdimages                              32G    65%      0%    12G
/mdata                                463G    36%      1%   280G

This could be improved in some ways – for instance it would be helpful to skip over certain of the filesystems that are not strictly speaking backed by disk. However it is beginning to be useful.

Or would be if the code exists. Fortunately it does.

Weird Open Source Names

 Computing  No Responses »
Dec 052008
 

I recently encountered a dead blog entitled “Linux Haters” and instantly thought up writing about tedious fan-boys that think that the operating system they like is the best and everyone should use it. I’ve no time for people like that as they tend to annoy rather than educate. I’ve no problem with people who prefer to use Windows, Linux, Solaris or OSX; it is their choice. Of course in the case of Windows, I do have to wonder why 🙂

But one of the links on that blog led to a place that (amongst other things) ranted about how FOSS projects always have dumb names, and that these projects need a big dose of marketing intelligence. He went on to whinge about the word-games often embedded into the project name.

First of all, he misunderstands how many open source projects start – with a geek or a group of geeks deciding they want something different. Either a new package or a variation on an existing one. There are no marketing types in sight, and the geeks involved probably have no great expectation that they are coming up with the next big thing – they are just having fun and hoping to come up with something useful for themselves. So what if they have a bit of fun playing word games to come up with a name for their project ? Not only do many such projects end up disappearing without a trace, but as marketing types have fun playing with words, why can’t geeks ?

Perhaps the names they come up with are not as punchy as a name thought up by a marketing department, but weirdness does have its own value in this area. A name such as Amarok does tend to stick in the mind more than Music Player 52. And over time, formally weird names such as google and yahoo do tend to become more normal if they are attached to popular projects.

Secondly he specifically criticises names invented by geeks for being recursive acronyms … but does that matter ? He specifically names GIMP which is admittedly particularly guilty being a recursive acronym with no termination. But most users won’t care … once they learn that GIMP does images (and most distributions will tell you so in the menu), they are not going to care that the name is an infinitely recursive acronym … they will just get on and use it.

Thirdly he overlooks the fact that some of the names may in fact have “sensible” names but are in fact sensible names in non-English languages.

Finally he tails off into a moderately incoherant rant with more insults than proper facts.

Perhaps “funny” names do put people off, but perhaps not. Most people are in fact more concerned with compatibility (they use Word because everyone else does) or features.

And of course there are more than a few commercial software packages whose name is not entirely sensible … does Photoshop have anything to do with setting up a shop to sell photos? What does Trent do ? Or Cedar ?

Upgrading to Ubuntu 8.10; An Alternative Method and Some Rough Edges

 Computing, Working Notes  No Responses »
Nov 082008
 

Well I decided it was time to upgrade my main workstation to Ubuntu 8.10 from 8.04. This was a somewhat nervous upgrade because the SATA disks are connected to a controller that gives me trouble when booting … the last few times I rebooted the machine I have had to re-assemble one of the mirrors at the “miniroot” (the shell you get from the initrd environment).

Given the trouble I wanted the option to revert back to the current install if everything went wrong … something I have described in the past.

Environment

I have four SATA disks in my main workstation that are mirrored and encapsulated into a single volume group with numerous logical volumes. So /dev/md0 is constructed of a mirror of /dev/sda2 and /dev/sdb2 and /dev/md1 is constructed of a mirror of /dev/sdc1 and /dev/sdd1. /dev/sda1 is used as /boot and /dev/sdb1 is “spare”.

The volume group datavg has two “physical disks” in it – /dev/md0 and /dev/md1. In that volume group are (amongst others) two logical volumes of interest – 804root and 804var.

Preparation

First I ensured that /dev/sdb1 contained an up to date copy of /boot :-

mkfs -t ext2 /dev/sdb1
mount /dev/sdb1 /mnt
star -copy /boot /mnt
umount /mnt

The next step was to create two new logical volumes to be used (eventually) for the new release of Ubuntu :-

lvcreate --size 12G -name 810root /dev/datavg
lvcreate --size 3G --name 810var /dev/datavg
mkfs -t xfs /dev/datavg/810root
mkfs -t xfs /dev/datavg/810var

The next step was to copy the filesystems across – at this point no further non-upgrade activity took place :-

mount /dev/datavg/810var /mnt
star -copy -xdev /var /mnt
mount /dev/datavg/810root /mnt
star -copy -xdev / /mnt
(edited /mnt/etc/fstab to change references to 804 to 810)
umount /mnt

At this point the remaining task (or so I thought) was to change the references in /boot/grub/menu.lst so that Grub would boot the kernel with 810root as the root filesystem.

I then rebooted to check the functionality and discovered that for some reason /var/run/network had not been created on the new /var but that seemed to be the only issue. I re-created this directory and rebooted to be sure (and Linux without a loopback device behaves very strangely!).

I then in theory had an environment I could revert to with a bit of fiddling with grub so was happy to attempt the upgrade.

The Upgrade Itself

I then started the upgrade (sudo update-manager -d) only to be told I did not have enough disk space with the suggestion that I run sudo apt-get clean to free up enough space. I tried this and the update started on the next time through. Why doesn’t the upgrade process automatically run apt-get clean ? In fact why doesn’t the upgrade process realise that it is on a system with LVM available (with plenty of free space) and create a new LVM especially for the upgrade files ?

Anyway the second attempt to run update-manager did start downloading files so I settled down for a well earned beer.

After the files had downloaded the actual upgrade process began. At some point the process stopped whilst a certain upgtade (libpam) asked whether it should restart certain services (including gdm which is the graphical login). Unfortunately it did not explain clearly what it was doing, why, or even if restarting gdm would terminate the current X session. Perhaps it is something that would only “worry” someone with a dangerous amount of knowledge, but perhaps the prompt could be improved slightly.

As a later example of how it should be done, the dialog warning about changing the ssl certificate was ideal … basically saying “if this doesn’t mean much to you, don’t worry about it”.

The use of dialogs during the upgrade does raise a couple of issues though :-

  1. Apparently not “stopping the clock” whilst waiting for the user to respond to dialogs caused the estimate of how long the upgrade would take place to increase dramatically. Just after dealing with that libpam prompt I was slightly alarmed by the estimate of 6 further hours to complete (as it was already 21:30) only to be reassured by the rapid drop to 2 hours.
  2. From a users perspective it would be preferable to ask all questions at the beginning or at the end of a long process. Keeping an eye on the upgrade whilst it was churning through the bulk of the upgrade (which took place without user intervention) was somewhat tedious.

The unfortunate thing was that at this point, the upgrade process ran out of space. It also left my machine in an unbootable state … from what little investigation I made it would appear that the initrd file was not created. Perhaps because of the lack of space, but others have also encountered this problem.

Obviously the Ubuntu upgrade process should be a little more careful about estimating the amount of space available. It would also be nice if it would notice that it was installed on an LVMed system with free space available … it could create a new LVM of an appropriate size and use that.

Recovery

Fortunately my preparation left me with a root filesystem that I could revert to by selecting one of the older kernels in the grub menu, editing the command used to boot, and by replacing the root filesystem with “804root”. This resulted in a system almost identical to that before the upgrade process.

I re-tried the “Preparation” stage, ensuring that /var was a little bigger and restarted the upgrade process …

Upgrading (The Second Attempt)

The second upgrade went through pretty much the same process as the first … except that it did not run out of disk space. This let it complete apparently normally with a warning about removing several hundred packages (!).

Finally it suggested rebooting to finish the upgrade.

The First Boot

The first boot was a triffle problematic … firstly I still had to manually assemble /dev/md0 and the process that built /boot/grub/menu.lst hadn’t noticed that an invalid root filesystem was specified. Whilst the later was my fault, it would be nice if it had realised that it was invalid and had warned me. The former is a long standing problem and is not the result of the upgrade.

Apart from that, the standard user interface looked ok, but it had messed around with the Enlightenment window manager. I had to re-install the package, my settings and menus had disappeared. Whilst I am an “unusual” user in not using the standard user interface, this was not something I was happy with!

IT: Linux, ZFS and Open-Source Licenses

 Computing, Politics  No Responses »
Aug 252007
 

If you’re hoping to read about Linux finally getting ZFS (except as a FUSE module) then you are going to be disappointed … this is merely a rant about the foolishness shown by the open-source world. It seems that the reason we won’t see ZFS in the Linux kernel is not because of technical issues but because of licensing issues … the two open-source licenses (GPL and CDDL) are allegedly incompatible!

Now some may wonder why ZFS is so great given that most of the features are available in other storage/filesystem solutions. Well as an old Unix systems administrator, I have seen many different storage and filesystem solutions over time … Veritas, Solaris Volume Manager, the AIX logical volume manager, Linux software RAID, Linux LVM, …, and none come as close to perfection as ZFS. In particular ZFS is insanely simple to manage, and those who have never managed a server with hundreds of disks may not appreciate just how desireable this simplicity is.

Lets take a relatively common example from Linux; we have two disks and no RAID controller so it makes sense to use Linux software RAID to create a virtual disk that is a mirror of the two physical disks. Not a difficult task. Now we want to split that disk up into seperate virtual disks to put filesystems on; we don’t know how large the different filesystems will become so we need to have some facility to grow and shrink those virtual disks. So we use LVM and make that software RAID virtual disk into an LVM “physical volume”, add the “physical volume” to a volume group, and finally create “logical volumes” for each filesystem we want. Then of course we need to put a filesystem on each “logical volume”. None of these steps are particularly difficult, but there are 5 seperate steps, and the separate software components are isolated from each other … which imposes some limitations.

Now imagine doing the same thing with ZFS … we create a storage pool consisting of two mirrored physical disks with a single command. This storage pool is automatically mounted as a filesystem ready for immediate use. If we need separate filesystems, we can create each with a single command. Now we come to the advantages … filesystem ‘snapshots’ are almost instantaneous and do not consume additional disk space until changes are made to the original filesystem at which point the increase in size is directly proportional to the changes made. Each ZFS filesystem shares the storage pool with the size being totally dynamic (by default) so that you do not have a set size reserved for each filesystem … essentially the free space on every single filesystem is available to all filesystems.

So what is the reason for not having ZFS under Linux ? It is open-source so it is technically possible to add to the Linux kernel. It has already been added to the FreeBSD kernel (in “-CURRENT”) and will shortly be added to the released version of OSX. Allegedly because the license is incompatible. The ZFS code from Sun is licensed under the CDDL license and the Linux kernel is licensed under the GPL license. I’m not sure how they are incompatible because frankly I have better things to do with my time than read license small-print and try to determine the effects.

But Linux (reluctantly admittedly) allows binary kernel modules to be loaded into the kernel and the license on those certainly isn’t the GPL! So why is not possible to allow GPLed code and CDDLed code to co-exist peacefully ? After all it seems that if ZFS were compiled as a kernel module and released as a binary blob, it could then be used … which is insane!

The suspicion I have is that there is a certain amount of “not invented here” going on.

 Older Entries  Newer Entries