Linux: ZFS: Be Careful With Child File Systems

 Computing, Linux  No Responses »
Sep 012016
 

One of the advantages that ZFS brings, is that it is so easy to create file systems, that you can create them for purposes that you would not previously do. For example, I have an additional file system mounted under my home directory for a certain application that generates a lot of data that I do not need backed up. Because the script I use to back up stuff does not cross file system boundaries (i.e. it does not descend into a directory that contains a mounted file system), I can simply exclude a large amount of frequently changing data by making a file system.

Or I might (as it happens I do not, but I could well do) create file systems for large lumps of data to easily see how much space they occupy – perhaps ~/Pictures. You can run a command like du -sh ~/Pictures, but that is an expensive command (it takes a while) and it tells you how large the files are; not how much space they occupy on disk. And on-disk compression can make that a significant difference! So simply run df -h ~/Pictures if that directory is on a separate file system.

But there is a bit of a gotcha with that. If you create such file systems in the normal way (such as zfs create pool/mikes-pictures; zfs set mountpoint=/home/mike/Pictures pool/mikes-pictures) you risk creating a situation that may prevent your home directory from mounting. If the “child” file system is mounted before the parent, it will not be possible for the parent file system to be mounted when booting.

Instead create the hierarchy properly :-

zfs create pool/h2
mkdir /h2
zfs set mountpoint=/h2 pool/h2
zfs create pool/h2/mike
zfs create pool/h2/mike/Pictures
ls /h2/mike/Pictures

You will also have to fix the permissions, but this is a far safer way of organising things suitable for future file system creation.

damascus-unix-prompt

Linux: Fixing The Invisible Mouse Pointer

 Computing, Linux  No Responses »
Aug 292016
 

It seems that occasionally GNOME can go a little screwy and its fancy mouse pointer plugin can result in an invisible mouse pointer. Which makes doing anything just a little bit tricky.

If you can open a terminal, enter the command :-

gsettings set org.gnome.settings-daemon.plugins.cursor active false

And all should be well. At least until it decides to turn itself back on again (so make a note of this fix!).

damascus-unix-prompt

Randomising Stuff At The Command-Line

 Computing, Linux  No Responses »
Mar 252016
 

Recently I have been seeing quite a lot of usage of random.org (to pick out winners of various kinds of competitions; and no I’m not a winner). The documentation on that site are reasonable with regard to pseudo-random number generators but are not quite correct with regard to the source of random numbers under Linux. And for non-cryptographic uses, the following is fine.

The use of random.org momentarily made me wonder how I would do the equivalent at the Unix (or Linux) command-line, and having used the command before, the shuf command came to mind. To be honest shuffling is not what I think of randomisation given how bad I am at shuffling cards, but despite the name, shuf does pretty well at randomising things :-

» seq 1 10 | shuf
4
5
8
7
2
1
10
9
6
3

The seq command generates a sequence from 1-10 as given. It turns out that shuf can do it itself :-

» shuf -i 1-10
7
3
5
6
9
10
8
1
4
2

The most common (relatively) use I have for shuf is to pick out a random line or two from a file. By using the -n option, shuf can do this. The following example makes use of an example file which contains a small number of first names :-

» shuf -n 1 first-names 
Julian
» shuf -n 1 first-names
Ian
» shuf -n 1 first-names
Craig

If you have just a small selection to make, you can provide the list on the command line with the -em option :-

» shuf -n 1 -e Male Female
Female

And that is pretty much all there is to it – a simple tool that does just one thing well.

damascus-unix-prompt

Shell: Efficiency and Clarity

 Computing, Linux  No Responses »
Mar 052016
 

Just to amuse myself, I’ve been re-reading and re-learning the Unix shell’s ${} details, and it occurred to me that whilst these were all very well and cute, they very easily lead to impenetrable code. But they are more efficient.

Take the following two ways of getting the current date :-

✓ mike@pica» print -P "%D" 
16-03-05
✓ mike@pica» echo $(date) 
Sat 5 Mar 13:14:38 GMT 2016

It’s not exactly helpful that they return the date/time in different formats. But glossing over that for the moment, which one is clearer? That is right – the second one clearly says that it is going to “echo” the date. Even if this usage is particularly stupid (as date will echo the date all by itself), the second wins as far as clarity goes.

However it is also less efficient – rather than get the date and show it to the terminal, the shell invokes a sub-process to display the date, captures it and then uses it to show to the terminal. In the old days when terminals consisted of printing mechanisms that actually hit a template of a letter against an inked up ribbon against a roll of paper and hoped that the result was readable, this inefficiency could result in very slow code.

But today this level of inefficiency should not make that much difference, and if it does, then why are you writing code in the shell? There are far better languages out there.

In addition, there is a bit of a gotcha with the print -P “%D” option … it only works if you happen to be using zsh :-

✓ mike@pica» print -P "%D"
16-03-05
✓ mike@pica» /bin/sh
$ print -P "%D"
file: option requires an argument -- 'P'
Usage: file [-bcEhikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type]
            [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
       file -C [-m magicfiles]
       file [--help]
Warning: unknown mime-type for "-P" -- using "application/octet-stream"
Error: no such file "-P"
Error: no such file "%D"
$ 
✗ mike@pica» /bin/ksh
$ print -P "%D"
%D
$ 
✓ mike@pica» /bin/bash
mike@pica:~/.lyx$ print -P "%D"
file: option requires an argument -- 'P'
Usage: file [-bcEhikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type]
            [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
       file -C [-m magicfiles]
       file [--help]
Warning: unknown mime-type for "-P" -- using "application/octet-stream"
Error: no such file "-P"
Error: no such file "%D"
mike@pica:~/.lyx$ exit

Confusing is it not?damascus-unix-prompt

Of course if the shell would intercept common usages such as $(date) and optimise them, that would be perfectly reasonable.

Converting ihex Format To Binary Under Linux

 Computing, Linux, Working Notes  No Responses »
Jan 052016
 

A bit of a simple one this … if you are looking at converting an Intel hex format file that looks like the following :-

2016-01-05_2123

Then it is relatively trivial under Linux (Debian). The relevant tool is probably installed anyway; unless you are not compiling software which may be a marginal activity for weird people but so is converting ihex files. But just in case, you can install it with: sudo apt-get install binutils.

Once installed (or being already present) the conversion process is as simple as :-

» objcopy -I ihex -O binary somefile.hex somefile.bin

Be careful to specify the second file name or objcopy will overwrite the original hex file (don’t ask how I discovered this!).

 Older Entries  Newer Entries