upgrade – More Zonkyness

Upgrading to Ubuntu 8.10; An Alternative Method and Some Rough Edges

Nov 082008

Well I decided it was time to upgrade my main workstation to Ubuntu 8.10 from 8.04. This was a somewhat nervous upgrade because the SATA disks are connected to a controller that gives me trouble when booting … the last few times I rebooted the machine I have had to re-assemble one of the mirrors at the “miniroot” (the shell you get from the initrd environment).

Given the trouble I wanted the option to revert back to the current install if everything went wrong … something I have described in the past.

Environment

I have four SATA disks in my main workstation that are mirrored and encapsulated into a single volume group with numerous logical volumes. So /dev/md0 is constructed of a mirror of /dev/sda2 and /dev/sdb2 and /dev/md1 is constructed of a mirror of /dev/sdc1 and /dev/sdd1. /dev/sda1 is used as /boot and /dev/sdb1 is “spare”.

The volume group datavg has two “physical disks” in it – /dev/md0 and /dev/md1. In that volume group are (amongst others) two logical volumes of interest – 804root and 804var.

Preparation

First I ensured that /dev/sdb1 contained an up to date copy of /boot :-

mkfs -t ext2 /dev/sdb1
mount /dev/sdb1 /mnt
star -copy /boot /mnt
umount /mnt

The next step was to create two new logical volumes to be used (eventually) for the new release of Ubuntu :-

lvcreate --size 12G -name 810root /dev/datavg
lvcreate --size 3G --name 810var /dev/datavg
mkfs -t xfs /dev/datavg/810root
mkfs -t xfs /dev/datavg/810var

The next step was to copy the filesystems across – at this point no further non-upgrade activity took place :-

mount /dev/datavg/810var /mnt
star -copy -xdev /var /mnt
mount /dev/datavg/810root /mnt
star -copy -xdev / /mnt

(edited /mnt/etc/fstab to change references to 804 to 810)
umount /mnt

At this point the remaining task (or so I thought) was to change the references in /boot/grub/menu.lst so that Grub would boot the kernel with 810root as the root filesystem.

I then rebooted to check the functionality and discovered that for some reason /var/run/network had not been created on the new /var but that seemed to be the only issue. I re-created this directory and rebooted to be sure (and Linux without a loopback device behaves very strangely!).

I then in theory had an environment I could revert to with a bit of fiddling with grub so was happy to attempt the upgrade.

The Upgrade Itself

I then started the upgrade (sudo update-manager -d) only to be told I did not have enough disk space with the suggestion that I run sudo apt-get clean to free up enough space. I tried this and the update started on the next time through. Why doesn’t the upgrade process automatically run apt-get clean ? In fact why doesn’t the upgrade process realise that it is on a system with LVM available (with plenty of free space) and create a new LVM especially for the upgrade files ?

Anyway the second attempt to run update-manager did start downloading files so I settled down for a well earned beer.

After the files had downloaded the actual upgrade process began. At some point the process stopped whilst a certain upgtade (libpam) asked whether it should restart certain services (including gdm which is the graphical login). Unfortunately it did not explain clearly what it was doing, why, or even if restarting gdm would terminate the current X session. Perhaps it is something that would only “worry” someone with a dangerous amount of knowledge, but perhaps the prompt could be improved slightly.

As a later example of how it should be done, the dialog warning about changing the ssl certificate was ideal … basically saying “if this doesn’t mean much to you, don’t worry about it”.

The use of dialogs during the upgrade does raise a couple of issues though :-

Apparently not “stopping the clock” whilst waiting for the user to respond to dialogs caused the estimate of how long the upgrade would take place to increase dramatically. Just after dealing with that libpam prompt I was slightly alarmed by the estimate of 6 further hours to complete (as it was already 21:30) only to be reassured by the rapid drop to 2 hours.
From a users perspective it would be preferable to ask all questions at the beginning or at the end of a long process. Keeping an eye on the upgrade whilst it was churning through the bulk of the upgrade (which took place without user intervention) was somewhat tedious.

The unfortunate thing was that at this point, the upgrade process ran out of space. It also left my machine in an unbootable state … from what little investigation I made it would appear that the initrd file was not created. Perhaps because of the lack of space, but others have also encountered this problem.

Obviously the Ubuntu upgrade process should be a little more careful about estimating the amount of space available. It would also be nice if it would notice that it was installed on an LVMed system with free space available … it could create a new LVM of an appropriate size and use that.

Recovery

Fortunately my preparation left me with a root filesystem that I could revert to by selecting one of the older kernels in the grub menu, editing the command used to boot, and by replacing the root filesystem with “804root”. This resulted in a system almost identical to that before the upgrade process.

I re-tried the “Preparation” stage, ensuring that /var was a little bigger and restarted the upgrade process …

Upgrading (The Second Attempt)

The second upgrade went through pretty much the same process as the first … except that it did not run out of disk space. This let it complete apparently normally with a warning about removing several hundred packages (!).

Finally it suggested rebooting to finish the upgrade.

The First Boot

The first boot was a triffle problematic … firstly I still had to manually assemble /dev/md0 and the process that built /boot/grub/menu.lst hadn’t noticed that an invalid root filesystem was specified. Whilst the later was my fault, it would be nice if it had realised that it was invalid and had warned me. The former is a long standing problem and is not the result of the upgrade.

Apart from that, the standard user interface looked ok, but it had messed around with the Enlightenment window manager. I had to re-install the package, my settings and menus had disappeared. Whilst I am an “unusual” user in not using the standard user interface, this was not something I was happy with!

IT: Upgrading To Ubuntu 8.04

Computing No Responses »

Apr 272008

This entry is about upgrading a machine running Ubuntu 7.10 to Ubuntu 8.04 which is only just out. But not in the standard way which would be quite boring.

I have at least two computers running Ubuntu, both configured in a fairly complex way and both fairly important (in the sense I really don’t need to try an upgrade and end up with a broken system). Whilst Ubuntu frequently does upgrade without a hitch, it can occasionally choke; this is seemingly more common with more complex installations.

Why not preserve an old copy of the install around to revert to ? Well with LVM it is perfectly possible. Ignoring what happens underneath, I have an LVM volume group called “internal” (actually I don’t, but I would if I were to re-install) which has :-

var – 4Gbytes to be mounted as /var
root – 8Gbytes to be mounted as /
home – “enough” to be mounted as /home

Note I do not believe in allocating all available disk space with a storage management system like LVM available; I do a great deal of storage management work and the biggest mistake anyone can make is assuming that they know the storage requirements of a system throughout it’s whole lifetime. This applies in spades to a desktop machine. Without some free space, the suggested upgrade mechanism won’t work.

Now with modern hard disks, we are likely to have more than enough storage to allocate. For instance on this machine right away I have 138Gbytes of free storage (mirrored). And that it is on a two year old machine; a newer machine would have larger disks. Easily enough storage to have two or more “copies” of different versions of Ubuntu around.

It would be nice if Ubuntu could do much of the work for us, but for now it’s pretty much a manual process. As an aside, the Ubuntu developers should probably think about using LVM in the default installer to assist in the development of this kind of feature.

The first stage is to create new logical volumes and build filesystems on them. I chose to name the logical volumes after the operating system version they would be running …

lvcreate -n 804root --size=8G /dev/internal
lvcreate -n 804var --size=4G /dev/internal
mkfs -t xfs /dev/internal/804root
mkfs -t xfs /dev/internal/804var

Now the key here is not to look at the current size of your /var filesystem and decide you need a much smaller filesystem … or the upgrade process will refuse to start. You can always reduce it later if you really want to quibble over 1-2Gbytes.

The next stage is to copy the relevant filesystems across. At this point you should avoid running as much as possible and probably do this from a text terminal after shutting down GDM …

/etc/init.d/gdm stop
apt-get install star
     (If you don't have it installed already)
mount /dev/internal/804var /mnt
star -v -xdev -acl -copy /var/* /mnt
umount /mnt
mount /dev/internal/804root /mnt
star -v -xdev -acl -copy / /mnt

This stage will take some time to complete. You will want to do a quick check of the new / and /var to ensure they look roughly like the originals (I always seem to come up with the equivalent of /var/var when I do something like this). Notice that the new root filesystem is still mounted … you need to edit /mnt/etc/fstab to alter what devices are mounted for / and /var.

The next stage is a bit tricky because I didn’t do it “right”, so I will be suggesting something that I didn’t try myself. The task is to modify /boot/grub/menu.lst in such a way as to result in two separate menu entries that will boot either the old operating environment or the new operating environment.

I would suggest that you :-

Create an entry outside of the “DEBIAN AUTOMAGIC KERNELS LIST” that essentially replicates one of the entries. It should not be modified to boot off the new root filesystem.
Modify all of the entries in the “DEBIAN AUTOMAGIC KERNELS LIST” (it makes sense when you review the menu.lst file) to alter the “root=’ kernel parameter to point to the new root filesystem. This is not the “root (hd0,0)” part, but the kernel parameter “root”. It will specify the old root filesystem logical volume (something like “root=/dev/internal/root”) and you want to change this to “root=/dev/internal/804root”.

At this point you should probably reboot to check that both environments work. Just make sure you have a recent rescue CD knocking around before you do.

After you have done the checking you can boot the new environment and use ‘update-manager’ to upgrade the new environment to Ubuntu 8.04. This will probably work (it worked fine for me).

Undoubtedly the next time I try this, I will figure out how to make it work better, but it is good enough to have a “fallback” option in case an upgrade goes badly. For instance until last week, running Vmware Server under 8.04beta was pretty tricky and if it were still the case I would have to revert back to 7.10.