No ads? Contribute with BitCoins: 16hQid2ddoCwHDWN9NdSnARAfdXc2Shnoa
Jun 262014

Came across a hint today about reporting on ECC memory errors. For those who do not know, ECC memory detects memory errors and corrects correctable errors. Normal memory (as found in almost all laptops and desktops) simply ignores the errors and lets them accumulate and cause problems either with data corruption or by causing software errors.

As I happen to have ECC memory in my desktop machine I thought I would have a look into the hint. Turns out that Linux does not report on ECC events automatically; you need to install the relevant EDAC (Error Detection and Correction) tools. Which for Debian, turns out to be pretty simple :-

# apt-get install edac-utils

As part of the installation process, a daemon process is started. But for whatever reason, it didn’t automatically detect what driver to load. So I edited /etc/default/edac and added :-


Once that is done, a simple /etc/init.d/edac restart loads the driver and starts monitoring. Messages should appear in your log files (/var/log/messages) and reports can be displayed with edac-util :-

# edac-util --report=full 

Of course memory errors are relatively rare (or at least should be) so it may take months before any error is reported. More later ūüôā

Jan 162014

This is not original work, but merely a set of notes on how to do the set up. The core information (and the code) came from this blog posting.

Essentially I’ve re-ordered the steps in which to work and excluded anything other than the bare essentials to get it all working. With the intention I can get my missile launcher working at home and at work¬† ūüėé

Step 1 is to prevent the HID driver from clamping on to the missile launcher. This was done by :-

  1. Editing /etc/default/grub and adding usbhid.quirks=0x2123:0x1010:0x04 to the existing variable GRUB_CMDLINE_LINUX_DEFAULT.
  2. Run update-grub (I always manage to forget this).
  3. Reboot the machine and check /var/log/messages for 2123 (the VendorID) to see if it has been claimed by usbhid (which will show up as a line beginning generic-usb 0003:2123:1010.0006: hiddev0 if it does claim it).

The next step is to download and compile the code given in the blog link above. If you need instructions on how to do this, then you probably need to look elsewhere – it builds easily.

Once built, an sudo insmod launcher_driver.ko will verify that the kernel module loads Рyou can double check by looking at /var/log/messages.

It’s also necessary to install both the kernel driver and the control program manually :-

  1. Copy the compiled kernel module to /lib/modules: sudo cp  launcher_driver.ko /lib/modules
  2. Edit /etc/rc.local and add the command: /sbin/insmod /lib/modules/launcher_driver.ko
  3. Copy the control program to a sensible location: sudo install launcher_control /opt/bin

There’s probably better ways of doing this, and better places to stick things but as you’re following my instructions you’re stuck with my suggestions! It’s tempting to try a reboot at this stage to verify that this works, but as there’s just one small extra step we may as well get that done too. This is to create a¬†udev rule to set up a device file in¬†/dev.

Create a file (/etc/udev/rules.d/99-usb-launcher.rules) with the following contents :-


The choice of group name is rather inappropriate except it will work well enough, and I have changed the permissions on this to something a little more restrictive. This can be tested with sudo udevadm trigger which will re-run udev. This should change the permissions on any existing /dev/launcher* file(s). If it doesn’t work, the blog pointer above is the place to head.

Lastly, there’s a couple of corrections to the¬†launcher_control.c that is convenient to make :-

% diff launcher_control.c launcher_control.c.orig
<         while ((c = getopt(argc, argv, "m:lrudfsht:")) != -1) {
>         while ((c = getopt(argc, argv, "mlrudfsht:")) != -1) {
< 		fprintf(stderr, "Couldn't open file: %s\n", dev);
<                 /*perror("Couldn't open file: %m");*/
>                 perror("Couldn't open file: %m");


Oct 232013

Crazy experiment time. What happens when you have a disk with 100 partitions? The replacement for the old MBR standard for partitions on PC hardware is slowly being replaced with GUID partitions. The later increased the maximum number of partitions to 128 which is probably far more than anyone needs, but what happens when you have a disk with 100 partitions?

As it happens, I had a spare external drive to play with, so set something up :-

for x in {1..99}     
  parted /dev/sdc mkpart FAT $(($x * 100)) $((x * 100 + 99))
  mkfs -t vfat /dev/sdc${x} 

This took a surprising amount of time to run with two interesting effects :-

  1. The mkfs tool refused to make a filesystem on /dev/sdc16 and /dev/sdc80 as it claimed it would be creating a filesystem on a full disk device. I suspect that this is a bug due to simplistic assumption of what constitutes a full disk device based on minor device numbers (/dev/sdc16 happened to be 0 and /dev/sdc80 happened to be 64). This could probably be solved by using device nodes within /dev/disk/by-${something}/${whatever}.
  2. The Unity Launcher appeared to attempt to populate itself with the new filesystems as they were being created, but very rapidly decided not to bother. This happened several times.

Once the creation process was complete, I reconnected the external drive to my Ubuntu machine, and yes the launcher does contain a ton of hard disk icons. The launcher is still full functional, but having a hundred (or so) devices below the normal icons does make using it a little clumsy.

Fortunately it did not mount all the filesystems automatically – closing that many windows would be very tedious. Mounting them all via a file manager window was pretty tedious, but it worked :-

/dev/sdc56             95M     0   95M   0% /media/mike/8663-39C5
/dev/sdc65             95M     0   95M   0% /media/mike/8673-0919
/dev/sdc71             95M     0   95M   0% /media/mike/873E-FEE7
/dev/sdc72             95M     0   95M   0% /media/mike/8741-47B3
/dev/sdc79             95M     0   95M   0% /media/mike/874D-4B53
/dev/sdc81             95M     0   95M   0% /media/mike/874E-D280
/dev/sdc82             95M     0   95M   0% /media/mike/8752-1ACE
/dev/sdc83             95M     0   95M   0% /media/mike/8754-2562
/dev/sdc84             95M     0   95M   0% /media/mike/8755-D262
/dev/sdc86             95M     0   95M   0% /media/mike/8759-0D82
/dev/sdc87             95M     0   95M   0% /media/mike/875A-E5C5
/dev/sdc89             95M     0   95M   0% /media/mike/875E-035B
/dev/sdc92             95M     0   95M   0% /media/mike/8763-8FB5
/dev/sdc93             95M     0   95M   0% /media/mike/8765-7A2F
/dev/sdc94             95M     0   95M   0% /media/mike/8767-1DBC
/dev/sdc95             95M     0   95M   0% /media/mike/8768-D314
/dev/sdc96             95M     0   95M   0% /media/mike/876A-A46E
/dev/sdc97             95M     0   95M   0% /media/mike/876B-F064
/dev/sdc98             95M     0   95M   0% /media/mike/876D-9D90
/dev/sdc58             95M     0   95M   0% /media/mike/8666-B9AA
/dev/sdc61             94M     0   94M   0% /media/mike/866B-8EFA
/dev/sdc62             95M     0   95M   0% /media/mike/866D-1726
/dev/sdc64             95M     0   95M   0% /media/mike/8671-5EE1
/dev/sdc66             95M     0   95M   0% /media/mike/8736-C2F5
/dev/sdc67             95M     0   95M   0% /media/mike/8737-EE95
/dev/sdc68             95M     0   95M   0% /media/mike/8739-7213
/dev/sdc69             94M     0   94M   0% /media/mike/873B-181F
/dev/sdc70             95M     0   95M   0% /media/mike/873C-E80C
/dev/sdc73             95M     0   95M   0% /media/mike/8743-11E7
/dev/sdc74             95M     0   95M   0% /media/mike/8745-28A8
/dev/sdc75             95M     0   95M   0% /media/mike/8746-CA94
/dev/sdc77             95M     0   95M   0% /media/mike/874A-1D30
/dev/sdc78             95M     0   95M   0% /media/mike/874B-C1C7
/dev/sdc85             95M     0   95M   0% /media/mike/8757-77A0
/dev/sdc88             94M     0   94M   0% /media/mike/875C-6DF9
/dev/sdc90             95M     0   95M   0% /media/mike/8760-8FD5
/dev/sdc91             94M     0   94M   0% /media/mike/8762-01DA
/dev/sdc99             94M     0   94M   0% /media/mike/8770-0F74
/dev/sdc1              93M     0   93M   0% /media/mike/8609-229A
/dev/sdc17             95M     0   95M   0% /media/mike/8621-921D
/dev/sdc21             95M     0   95M   0% /media/mike/8628-8CDB
/dev/sdc22             95M     0   95M   0% /media/mike/862A-2217
/dev/sdc23             94M     0   94M   0% /media/mike/862B-8EF9
/dev/sdc25             95M     0   95M   0% /media/mike/862F-0BE5
/dev/sdc27             95M     0   95M   0% /media/mike/8633-1F9D
/dev/sdc28             95M     0   95M   0% /media/mike/8634-A26F
/dev/sdc34             95M     0   95M   0% /media/mike/863E-14EB
/dev/sdc37             95M     0   95M   0% /media/mike/8643-1F63
/dev/sdc4              95M     0   95M   0% /media/mike/860D-2753
/dev/sdc40             95M     0   95M   0% /media/mike/8647-8E49
/dev/sdc41             95M     0   95M   0% /media/mike/8649-033D
/dev/sdc42             94M     0   94M   0% /media/mike/864A-A12A
/dev/sdc43             95M     0   95M   0% /media/mike/864C-6EEF
/dev/sdc44             95M     0   95M   0% /media/mike/864E-3469
/dev/sdc45             95M     0   95M   0% /media/mike/8650-8796
/dev/sdc46             95M     0   95M   0% /media/mike/8652-64DF
/dev/sdc47             95M     0   95M   0% /media/mike/8653-F743
/dev/sdc48             95M     0   95M   0% /media/mike/8655-B14B
/dev/sdc49             95M     0   95M   0% /media/mike/8657-34FF
/dev/sdc5              95M     0   95M   0% /media/mike/860E-EBD7
/dev/sdc50             94M     0   94M   0% /media/mike/8658-A04A
/dev/sdc51             95M     0   95M   0% /media/mike/865A-D4D3
/dev/sdc52             95M     0   95M   0% /media/mike/865C-33D1
/dev/sdc53             95M     0   95M   0% /media/mike/865D-FA56
/dev/sdc54             95M     0   95M   0% /media/mike/8660-6C95
/dev/sdc55             95M     0   95M   0% /media/mike/8661-D456
/dev/sdc57             95M     0   95M   0% /media/mike/8665-0AFD
/dev/sdc59             95M     0   95M   0% /media/mike/8668-3D53
/dev/sdc6              95M     0   95M   0% /media/mike/8610-F9B0
/dev/sdc60             95M     0   95M   0% /media/mike/866A-0A0E
/dev/sdc63             95M     0   95M   0% /media/mike/866E-F6E7
/dev/sdc76             95M     0   95M   0% /media/mike/8748-8D02
/dev/sdc10             95M     0   95M   0% /media/mike/8616-B29F
/dev/sdc11             95M     0   95M   0% /media/mike/8618-6462
/dev/sdc12             94M     0   94M   0% /media/mike/861A-5208
/dev/sdc13             95M     0   95M   0% /media/mike/861B-BA6E
/dev/sdc14             95M     0   95M   0% /media/mike/861D-5133
/dev/sdc15             95M     0   95M   0% /media/mike/861E-C384
/dev/sdc18             95M     0   95M   0% /media/mike/8623-BFCF
/dev/sdc19             95M     0   95M   0% /media/mike/8625-9D85
/dev/sdc2              95M     0   95M   0% /media/mike/860A-504E
/dev/sdc20             94M     0   94M   0% /media/mike/8627-1391
/dev/sdc24             95M     0   95M   0% /media/mike/862D-457F
/dev/sdc26             95M     0   95M   0% /media/mike/8631-5F8A
/dev/sdc29             95M     0   95M   0% /media/mike/8636-2F58
/dev/sdc3              95M     0   95M   0% /media/mike/860B-8C77
/dev/sdc30             95M     0   95M   0% /media/mike/8637-F726
/dev/sdc31             94M     0   94M   0% /media/mike/8639-6B19
/dev/sdc32             95M     0   95M   0% /media/mike/863A-FBBC
/dev/sdc33             95M     0   95M   0% /media/mike/863C-AE68
/dev/sdc35             95M     0   95M   0% /media/mike/8640-3A10
/dev/sdc36             95M     0   95M   0% /media/mike/8641-93A6
/dev/sdc38             95M     0   95M   0% /media/mike/8644-AFCF
/dev/sdc39             94M     0   94M   0% /media/mike/8646-1BAE
/dev/sdc7              95M     0   95M   0% /media/mike/8612-54E8
/dev/sdc8              95M     0   95M   0% /media/mike/8613-C38C
/dev/sdc9              95M     0   95M   0% /media/mike/8615-3522

Yes I have cut the “interesting” filesystems out of that output.

Windows (7) does deal¬†quite so well with the situation. After rebooting into Windows with the disk plugged in, the login process seemed to take longer than usual (although I don’t boot Windows enough to be sure).

Once logged in, everything seemed fine including the little popup window by the status bar saying it was configuring the plugged in disk drive. However that took longer than expected – after clicking on it for details, it took around 5 minutes to complete. At which point it stuck a red cross by the “Disk Drive” whilst it popped up an Autoplay window for drives E: through Z:. With an offer to format drive T: – so it could at least use the drive that Linux refused (by default) to format.

However except for that little red cross, there was no clear warning that it failed to do anything with nearly 80 partitions. And closing all those popup Autoplay windows was pretty tedious.

OSX (10.9) dealt a little better with the disk; it at least recognised all of the disks, and stuck up little icons for each one. And mounted them all. ¬†However Finder didn’t seem to respond to attempts to unmount the disks … I had to resort to the command-line. Perhaps I wasn’t patient enough.

And the moral of this little crazy experiment? Whilst we can perhaps throw a little mud at Microsoft, the main lesson learnt is that you too can annoy someone using Windows by handing them an external hard disk with 100 partitions. Especially if the information they want is not in the first 20-odd partitions <Evil Grin>

Oh! And just stating the obvious – it’s a good idea to remove the partitions before putting the spare disk away, or you may encounter a nasty surprise later!


Sep 292013

As some people know, the Linux device for generating random numbers (/dev/random) blocks when there isn’t sufficient entropy to safely generate random numbers. But people will still persist on advising using¬†/dev/urandom as an alternative :-

“To sum up, under both Linux and FreeBSD, you should use¬†/dev/urandom, not¬†/dev/random.”

“Just go ahead and use /dev/urandom as is”

“Oracle wants us to move /dev/random and link /dev/urandom”

“You can remove /dev/random and link that to /dev/urandom to help prevent blocking”

Now it is true that /dev/urandom is usually good enough, but to advise people to use /dev/urandom without considering whether it is sufficient or not is irresponsible. True random numbers can be very important for cryptography, and without knowing it, we use cryptography every day; such as when we browse the web, make ssh connections, check PGP keys, etc. Using a weak random number generator can weaken the cryptographic process fatally.

Aug 202013

Every so often I come across an old Linux box that doesn’t take kindly to being rebooted. Without console access, it is hard to see what is going on, but the Linux kernel gets stuck trying to mount the root file system. There are many possible fixes for this, but they all have one thing in common ‚Ķ a work-around has to be performed to get the box up and running.

The console gets stuck in a “mini-root” environment loaded when the¬†initrd image is loaded and before the real root file system is mounted which means a lot of commands are not available, but¬†lvm should be available. First of all, run¬†lvm lvscan to get a list of the logical volumes that need activating :-

(initramfs) lvm lvscan
  inactive          '/dev/sys/root' [332.00 MiB] inherit
  inactive          '/dev/sys/usr' [8.38 GiB] inherit
  inactive          '/dev/sys/var' [2.79 GiB] inherit
  inactive          '/dev/sys/swap_1' [7.05 GiB] inherit
  inactive          '/dev/sys/tmp' [380.00 MiB] inherit
  inactive          '/dev/sys/home' [16.00 GiB] inherit
  inactive          '/dev/sys/opt' [24.00 GiB] inherit

For each volume group (the second column, middle word), run: lvm lvchange -ay ${volume-group-name}. In the case of my example :-

(initramfs) lvm vgchange -ay /dev/sys
  7 logical volume(s) in volume group "sys" now active

At which point you should be able to press ^D (or enter exit) to continue the boot process.

A slighter better work-around involves changing the Grub configuration to add a delay to the kernel parameters. This sections assumes that you are not using Grub Legacy!

Start by editing /etc/default/grub and changing the variable GRUB_CMDLINE_LINUX to include “rootdelay=20” :-

GRUB_CMDLINE_LINUX='console=tty0 console=ttyS0,19200n8 rootdelay=20'

Finalise by running update-grub. This adds a 20s delay to the boot process so is hardly an ideal solution.

Jul 292013

‚Ķ which is of course¬†massive overkill. But fun. It should increase the raw bandwidth available between the two machines from 1Gbps to 20Gbps (with one link) and 40Gbps with both links bonded. It was a bit of a surprise to me when I looked around at prices of second-hand kit to realise that InfiniBand was so much cheaper to acquire than Fibre Channel; the kit I acquired cost less than ¬£100 all in whereas FC kit would be in the region of ¬£1,000, and InfiniBand is generally quicker. There is of course 16Gb FC and 10Gb InfiniBand, but that is hardly comparing like with like. So what is this overkill for? Networking of course. I’ve acquired two HP InfiniBand dual link cards which means I can connect my workstation to my server :- InfiniBand Network Using dual links is of course overkill on top of overkill, but given that these cards have dual links, why not use them? And it does give a couple of experiments to try later. To prepare in advance, the following network addresses will be used :-

Server Link Number IPv4 Address IPv6 Address
A 1 AAISP:d00d::1
A 2 AAISP:d00f::1
B 1 AAISP:d00d:2
B 1 AAISP:d00f:2

Yes I have cheated for the IPv6 addresses! The first step is to configure each “server” ‚Ķ one is running Debian Linux, and the other is running FreeBSD.

Configuring Linux

This was subject to much delay whilst I believed that I had a problem with the InfiniBand card, but putting the card into a new desktop machine caused it to spring back to life. Either some sort of incompatibility with my old desktop (which was quite old), or some sort of problem with the BIOS settings.

Inserting the card should load the core module (mlx4_core) automatically, and spit out messages similar to the following :-

[    3.678189] mlx4_core 0000:07:00.0: irq 108 for MSI/MSI-X
[    3.678195] mlx4_core 0000:07:00.0: irq 109 for MSI/MSI-X
[    3.678199] mlx4_core 0000:07:00.0: irq 110 for MSI/MSI-X
[    3.678204] mlx4_core 0000:07:00.0: irq 111 for MSI/MSI-X
[    3.678208] mlx4_core 0000:07:00.0: irq 112 for MSI/MSI-X
[    3.678212] mlx4_core 0000:07:00.0: irq 113 for MSI/MSI-X
[    3.678216] mlx4_core 0000:07:00.0: irq 114 for MSI/MSI-X
[    3.678220] mlx4_core 0000:07:00.0: irq 115 for MSI/MSI-X
[    3.678223] mlx4_core 0000:07:00.0: irq 116 for MSI/MSI-X
[    3.678228] mlx4_core 0000:07:00.0: irq 117 for MSI/MSI-X
[    3.678232] mlx4_core 0000:07:00.0: irq 118 for MSI/MSI-X
[    3.678236] mlx4_core 0000:07:00.0: irq 119 for MSI/MSI-X
[    3.678239] mlx4_core 0000:07:00.0: irq 120 for MSI/MSI-X
[    3.678243] mlx4_core 0000:07:00.0: irq 121 for MSI/MSI-X
[    3.678247] mlx4_core 0000:07:00.0: irq 122 for MSI/MSI-X
[    3.678250] mlx4_core 0000:07:00.0: irq 123 for MSI/MSI-X
[    3.678254] mlx4_core 0000:07:00.0: irq 124 for MSI/MSI-X
[    3.678259] mlx4_core 0000:07:00.0: irq 125 for MSI/MSI-X
[    3.678263] mlx4_core 0000:07:00.0: irq 126 for MSI/MSI-X
[    3.678267] mlx4_core 0000:07:00.0: irq 127 for MSI/MSI-X
[    3.678271] mlx4_core 0000:07:00.0: irq 128 for MSI/MSI-X
[    3.678275] mlx4_core 0000:07:00.0: irq 129 for MSI/MSI-X

This is just the core driver; at this point additional modules are needed to do anything useful. You can manually load the modules with¬†modprobe but sooner or later it is better to make sure they’re loaded automatically by adding their names to¬†/etc/modules. The modules you want to load are :-

  1. mlx4_ib
  2. ib_umad
  3. ib_uverbs
  4. ib_ipoib

This is a minimal set necessary for networking (“IP”) rather than additional features such as SCSI. It’s generally better to start with a minimal set of features initially. At this point, it is generally a good idea to reboot to verify that things are getting closer. After a reboot, you should have one or more new network interfaces listed by¬†ifconfig :-

ib0       Link encap:UNSPEC  HWaddr 80-00-00-48-FE-80-00-00-00-00-00-00-00-00-00-00  
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:256 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ib1       Link encap:UNSPEC  HWaddr 80-00-00-49-FE-80-00-00-00-00-00-00-00-00-00-00  
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:256 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Despite the appearance, we still have quite a way to go yet. The next step is to install some additional packages: ibutils,¬†infiniband-diags, and opensm. The last package is for a subnet manager which is unnecessary if you have an InfiniBand switch (but I don’t). The first step is to get opensm up and running. Edit¬†/etc/default/opensm and change the PORTS variable to “ALL” (unless you want to restrict the managed ports, and make things more complicated). And start opensm:¬†/etc/init.d/opensm start;¬†update-rc.d opensm defaults.

At this point, you can configure the network addresses by editing¬†/etc/network/interfaces. If you need help doing this, then you’re in the tech pool beyond your depth! Without something at the other end, these interfaces won’t work (obviously), so it’s time to start work on the other end ‚Ķ

Configuring FreeBSD

See:¬† I hadn’t had cause to build a custom kernel before, so the very first task was to use subversion to checkout a copy of the FreeBSD source code :-

svn co svn:// /usr/src

Updating will of course require just: cd /usr/src && svn update. Once installed, create a symlink from /sys to /usr/src/sys if the link does not already exist: ln -s /usr/src/sys /sys

Go to the kernel configuration directory (/usr/src/sys/amd64/conf), copy the GENERIC configuration file to a new file, and edit the new file to add in certain options :-

# Infiniband stuff (locally added)
options         OFED
options         IPOIB_CM
device          ipoib
device          mlx4ib

Again, this is a minimal set that will not offer full functionality … but should be enough to get IP networking up and running. The next step is to build and install the kernel :-

make buildkernel KERNCONF=${NAME-OF-YOUR-CONFIG}; make installkernel KERNCONF=${NAME-OF-YOUR-CONFIG}

The next step is to build the “world” ¬†:-

  1. Edit¬†/etc/src.conf and add “WITH_OFED=’yes'” to that file.
  2. Change to /usr/src and run: make buildworld
  3. Finalise with make installworld

As it happens I had to build the user-land first, as the kernel compilation needed a new user-land feature.

After a reboot, the new network interface(s) should show up as ib0 upwards. And these can be configured with an address in exactly the same as any other network interface.

Testing The Network

A tip for making sure the interfaces you think are connected together is to configure one of the machines, send a broadcast ping to the relevant network address of each interface in turn, and run tcpdump on the other machine to verify that the packets coming down the wire match what you expect.

Below the level of IP, it is possible to run an InfiniBand ping to verify connectivity. First you need a GUID on “the server”, which can be obtained by running¬†ibstat and looking for the “Port GUID”, which will be something like “0x0002c90200273985”. Next run¬†ibping -S on the server.

Now on the other machine (“the client”), run ibping :-

# ibping -G 0x0002c90200273985
Pong from (Lid 3): time 0.242 ms
Pong from (Lid 3): time 0.153 ms
Pong from (Lid 3): time 0.160 ms

The next step is to run an IP ping to one of the hosts. If that works, it is time to start looking at something that will do a reasonable attempt at a speed test.

This can be done in a variety of different ways, but I chose to use nttcp which is widely available. On one of the hosts, run nttcp -i to act as the “partner” (or server). On the sending server, run nntcp -T ${ip-address-to-test} which will give output something like :-

# nttcp -T
     Bytes  Real s   CPU s Real-MBit/s  CPU-MBit/s   Calls  Real-C/s   CPU-C/s
l  8388608    0.70    0.01     95.7975   5592.4053    2048   2923.51  170666.7
1  8388608    0.71    0.04     94.0667   1878.6950    5444   7630.87  152403.3

According to the documentation, the second line should begin with ‘r’, but for a simple speed test we can simply average the numbers in the “Real-MBit/s” to get an approximate speed. Oddly my gigabit ethernet seems to have mysteriously degraded to 100Mbps! At least it makes the InfiniBand speed slightly more impressive :-

# nttcp -T
     Bytes  Real s   CPU s Real-MBit/s  CPU-MBit/s   Calls  Real-C/s   CPU-C/s
l  8388608    0.03    0.00   2521.9415  16777.2160    2048  76963.55  512000.0
1  8388608    0.03    0.03   2206.6574   2568.6620    4032 132579.25  154329.0

Before getting into a panic over what appears to be a pretty poor result, it is worth bearing in mind that IP over InfiniBand isn’t especially efficient, and InfiniBand seems to suffer from marketing exaggeration. From what I understand, DDR’s 20Gbps signalling rate becomes 16Gbps, which in turn becomes 8.5Gbps when looking at the output of ibstatus (not ibstat) – why the halving here is a bit of a mystery, but that may become apparent later.

There has also been a hint that FreeBSD is due for a significant improvement in InfiniBand performance sometime after the release of 9.2.

As a late addition, it would appear that running OpenSM (the subnet manager) on both hosts means that when one or other is rebooting, the other can take over the duties of the subnet manager. To enable on FreeBSD, simply add opensm_enable=”YES” to the file /etc/rc.conf and reboot.

Oct 312011

According to a couple of articles on The Register, a couple of manufacturers are getting close to releasing ARM-based servers. The interesting thing is that the latest announcement includes details of a 64-bit version of the ARM processor, which according to some people is a precondition for using the ARM in a server.

It is not really true of course, but a 64-bit ARM will make a small number of tasks possible. It is easy to forget that 32-bit servers (of which there are still quite a few older ones around) did a pretty reasonable job whilst they were in service – there is very little that a 64-bit server can do that a 32-bit server cannot. As a concrete example, a rather elderly SPARC-based server I have access to has 8Gbytes of memory available, is running a 64-bit version of Solaris (it’s hard to find a 32-bit version of Solaris for SPARC), but of the 170 processes it is running, none occupies more than 256Mbytes of memory; by coincidence the size of processes is also no more than 256Mb.

The more important development is the introduction of virtualisation support.

The thing is that people Рespecially those used to the x86 world Рtend to over-emphasise the importance of 64-bits. It is important as some applications do require more than 4Gbytes of memory to support Рin particular applications such as large Oracle (or other DBMS) installations. But the overwhelming majority of applications actually suffer a performance penalty if re-compiled to run as 64-bit applications.

The simple fact is that if an application is perfectly happy to run as a 32-bit application with a “limited” memory space and smaller integers, it can run faster because there is less data flying around. And indeed as pointed out in the comments section of the article above, it can also use ever so slightly more electricity.

What is overlooked amongst those whose thinking is dominated by the x86 world, is that the x86-64 architecture offers two¬†benefits over the old x86 world – a 64-bit¬†architecture¬†and¬†an improved architecture with many more CPU registered. This allows for 64-bit applications in the x86 world to perform better than their 32-bit counterparts even¬†if the applications wouldn’t normally benefit from running on a 64-bit architecture.

If the people producing operating systems for the new ARM-based servers have any sense, they will quietly create a 64-bit operating system that can transparently run many applications in 32-bit mode. Not exactly a new thing as this is what Solaris has done on 64-bit SPARC based machines for a decade. This will allow those applications that don’t require 64-bit, to gain the performance benefit of running 32-bit, whilst allowing those applications that require 64-bit to run perfectly well.

There is no real downside in running a dual word sized operating system except a minor amount of added complexity for those developers working at the C-language level.

Jun 182011

This is a series of notes on dealing with PC malware (viruses, worms and the like) gathered because I’m looking into it and published as a way of reminding myself about this stuff. Bear in mind that I’m not an expert but neither am I a complete dunce – I’m normally a Unix or Linux person but I’ve been keeping half an eye on Windows infections for years.

Some links to tools are contained within. However you should be aware that tool recommendations change over time; you will need to check how outdated this document is before following any recommendations blindly.

At present this blog entry is a work in progress … lots of testing needs to be done before being confident this is right.

Cleanup Process

This is not :-

  1. How to approach this forensically – if you’re dealing with an investigation, it’s a whole other ball game and you probably need professional assistance to avoid corrupting evidence.
  2. A technical guide as to which tools to use.

1. For The Ultra Cautious Or When Handling Real Important Data

The process of removal can be destructive, and in the worst cases you can end up cleaning the malware and ending up with a brick. So make an image of the hard disk as it is. Two basic ways this can be done :-

  1. Removing the hard disk from the infected machine, attaching to an appropriate machine (USB->SATA, USB->IDE converters are handy here), and making an image of the disk.
  2. Booting off a “rescue” CD on the infected machine, and imaging the hard disk to a network share of some kind. This is the preferred option.

This will be slow. So be it. Cleaning an infected PC is not going to be a quick job whatever you do. The best you can hope for is that there are many periods where you can leave it churning away and get on with something else.

2. Boot A Rescue CD

There are those who tell you that there is no need to boot off a known uninfected disk to clean an infected machine; their anti-malware/virus product can clean an infected machine “live”. There are others who claim that the only way to be sure is to boot off that disk and clean the machine that way. Both are wrong.

If you are paranoid (and in the presence of malware paranoia is fully justifiable), you will do both.

3. Boot Infected Machine and Clean

As suggested previously after booting off a rescue disk and cleaning, boot the infected machine and clean again.


The following is a list of rescue CD’s that have been suggested :-

  • UBD4Win. Has to be “built” with the assistance of an XP installation; somewhat tedious but it isn’t the end of the world. However it does need preparing in advance – building a rescue CD with the assistance of an infected machine isn’t the most sensible idea!
  • Knoppix. Graphical, pretty, feature packed, but seems to be lacking in anti-malware tools (for instance the only AV tool included is Clam).
  • Trinity Rescue Disk. Menu interface. Virus definitions update over the net; choice of Clam, F-Prot, Bitdefender, Vexira, AVast (need to obtain license key). Various other utilities.
  • F-Secure Rescue CD.

Some of the above are Windows based; some are Linux based. The choice of which to use should be based on results not whether they tickle your prejudices (or mine!).

The following is a list of “live” tools to be installed that have been suggested :-


Nothing to do with the main subject. Merely some notes worth mentioning.

It seems that at least some malware can detect it is running within a virtual environment. In some cases it ceases to do anything, and in others may try to “break out”. This indicates that analysing malware within a virtual environment may not give sensible results, and in some cases may be dangerous! That is not to say that using a virtual environment is no longer of any use, but you may need to take special case such as running the virtual environment under Linux and/or ESX rather than Windows. And be careful about negative results.

May 052011

For my own future reference …

Today I encountered an interesting little issue where I could not send an ABORT signal to a running process to kill it with a core dump because the process had a limit of 0 for the core dump size. Try as I might, I could not find a way to change that process’s core dump limit.

Turns out there is another way of tackling the problem, which is to use gdb to generate a core image :-

>attach PID
>gcore /var/tmp/core.PID

There is of course the gcore shell script wrapper for this, but that may not work if the working directory of the process no longer exists.

Jan 292011

In the dim and distant past when keyboards were enclosed in metal cases and you certainly didn’t tuck one under the arm and walk around with it (actually I don’t do now either), the placement of many keys was continually up for debate. But apart from the main QWERTY section, one of the key placements you could rely on was the Control key next to the “A” key. These days it’s been turned into one of those silly CapsLock keys.

Back when I previously did some keymapping, I neglected to mention how I mapped CapsLock into a Control key. As appropriate punishment, changing window managers has somehow meant that my previous mapping had been lost. So I had to figure out how to do it again.

First thing to do is to switch to a text console – I’ll be mapping this at a very low level.

Next thing to do is to find out the scancode of the key I want to map :-

# showkey -s

One started I have to press the key I am interested in within 10 seconds or the program will edit. I press CapsLock and I get two numbers displayed – 3a and ba (they’re in hexadecimal for the base-16 challenged). The first is the key press, and the second is the key release. We can discard the second as Linux is clever enough to figure out one from the other.

The next thing we want to do is to obtain the keycode of the key that we want to map to – in this case the left control key. It probably doesn’t matter here, but it is worth noting that the left and right control keys are different scancodes and keycodes. So you could map then to different things. Anyway, to obtain the keycode of the key we want, run :-

# showkey -k

And press the key to map to.

Lastly we want to construct the command to actually do the mapping :-

setkeycodes 3a 29

This of course has to be added to a script being run when the system boots – you want this mapped as early as possible.

Facebook Auto Publish Powered By :

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.