Mar 252016
 

Recently I have been seeing quite a lot of usage of random.org (to pick out winners of various kinds of competitions; and no I’m not a winner). The documentation on that site are reasonable with regard to pseudo-random number generators but are not quite correct with regard to the source of random numbers under Linux. And for non-cryptographic uses, the following is fine.

The use of random.org momentarily made me wonder how I would do the equivalent at the Unix (or Linux) command-line, and having used the command before, the shuf command came to mind. To be honest shuffling is not what I think of randomisation given how bad I am at shuffling cards, but despite the name, shuf does pretty well at randomising things :-

» seq 1 10 | shuf
4
5
8
7
2
1
10
9
6
3

The seq command generates a sequence from 1-10 as given. It turns out that shuf can do it itself :-

» shuf -i 1-10
7
3
5
6
9
10
8
1
4
2

The most common (relatively) use I have for shuf is to pick out a random line or two from a file. By using the -n option, shuf can do this. The following example makes use of an example file which contains a small number of first names :-

» shuf -n 1 first-names 
Julian
» shuf -n 1 first-names
Ian
» shuf -n 1 first-names
Craig

If you have just a small selection to make, you can provide the list on the command line with the -em option :-

» shuf -n 1 -e Male Female
Female

And that is pretty much all there is to it – a simple tool that does just one thing well.

damascus-unix-prompt

Mar 052016
 

Just to amuse myself, I’ve been re-reading and re-learning the Unix shell’s ${} detailsand it occurred to me that whilst these were all very well and cute, they very easily lead to impenetrable code. But they are more efficient.

Take the following two ways of getting the current date :-

✓ mike@pica» print -P "%D" 
16-03-05
✓ mike@pica» echo $(date) 
Sat 5 Mar 13:14:38 GMT 2016

It’s not exactly helpful that they return the date/time in different formats. But glossing over that for the moment, which one is clearer? That is right – the second one clearly says that it is going to “echo” the date. Even if this usage is particularly stupid (as date will echo the date all by itself), the second wins as far as clarity goes.

However it is also less efficient – rather than get the date and show it to the terminal, the shell invokes a sub-process to display the date, captures it and then uses it to show to the terminal. In the old days when terminals consisted of printing mechanisms that actually hit a template of a letter against an inked up ribbon against a roll of paper and hoped that the result was readable, this inefficiency could result in very slow code.

But today this level of inefficiency should not make that much difference, and if it does, then why are you writing code in the shell? There are far better languages out there.

In addition, there is a bit of a gotcha with the print -P “%D” option … it only works if you happen to be using zsh :-

✓ mike@pica» print -P "%D"
16-03-05
✓ mike@pica» /bin/sh
$ print -P "%D"
file: option requires an argument -- 'P'
Usage: file [-bcEhikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type]
            [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
       file -C [-m magicfiles]
       file [--help]
Warning: unknown mime-type for "-P" -- using "application/octet-stream"
Error: no such file "-P"
Error: no such file "%D"
$ 
✗ mike@pica» /bin/ksh
$ print -P "%D"
%D
$ 
✓ mike@pica» /bin/bash
mike@pica:~/.lyx$ print -P "%D"
file: option requires an argument -- 'P'
Usage: file [-bcEhikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type]
            [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
       file -C [-m magicfiles]
       file [--help]
Warning: unknown mime-type for "-P" -- using "application/octet-stream"
Error: no such file "-P"
Error: no such file "%D"
mike@pica:~/.lyx$ exit

Confusing is it not?damascus-unix-prompt

Of course if the shell would intercept common usages such as $(date) and optimise them, that would be perfectly reasonable.

Jan 052016
 

A bit of a simple one this … if you are looking at converting an Intel hex format file that looks like the following :-

2016-01-05_2123

Then it is relatively trivial under Linux (Debian). The relevant tool is probably installed anyway; unless you are not compiling software which may be a marginal activity for weird people but so is converting ihex files. But just in case, you can install it with: sudo apt-get install binutils.

Once installed (or being already present) the conversion process is as simple as :-

» objcopy -I ihex -O binary somefile.hex somefile.bin

Be careful to specify the second file name or objcopy will overwrite the original hex file (don’t ask how I discovered this!).

Dec 102015
 

damascus-unix-prompt

You have a a column of numbers that you have produced in some manner such as :-

$ awk '/clean message/ {print $(NF-1)}' mail.info.log
...
100935
12197
3606
84653
4498
99110
4762
3001
10889
12611
12249
12245
136599
49097
6668

And you want a quick and dirty way of finding the largest number. Well there is a way but it is perhaps the least efficient way to do it, and that is to sort the numbers into numerical order and use “head” to display the first one :-

$ awk '/clean message/ {print $(NF-1)}' mail.info.log | sort -rn | head -1
5476168

But frankly there must be a better method. And yes there is if you happen to be using zsh (or possibly others, but this has been tested with zsh). Simply iterate over the values assigning the current value to the “max” variable if the current variable is larger :-

$ max=0; for x in $(awk '/clean message/ {print $(NF-1)}' mail.info.log); [[ $x -gt $max ]] && max=$x; echo $max
5476168

You may be wondering why I don’t simply use the ability of awk to perform calculations. Well that is certainly possible, but I may not always be using awk to produce the numbers in the first place, and this is supposed to be a generic recipe.

Nov 142015
 

See an updated post here.

I am obviously doing something wrong because computers are not supposed to behave like this, but my Linux containers (despite previous attempts) are booting with IPv6 privacy addresses randomly :-

✓ root@pica» lxc-ls --fancy | grep chagers
chagers   RUNNING  10.0.0.32  2001:8b0:ca2c:dead::5e11, 2001:8b0:ca2c:dead:f42b:6dff:fe16:2f2d  YES        
✓ root@pica» lxc-stop --name chagers; lxc-start --daemon --name chagers
✓ root@pica» lxc-ls --fancy | grep chagers
chagers   RUNNING  10.0.0.32  2001:8b0:ca2c:dead:206b:70ff:fe45:7242, 2001:8b0:ca2c:dead::5e11  YES        
✓ root@pica» lxc-stop --name chagers; lxc-start --daemon --name chagers
✓ root@pica» lxc-ls --fancy | grep chagers
chagers   RUNNING  10.0.0.32  2001:8b0:ca2c:dead::5e11                                         YES        

That is not how computers are supposed to behave!damascus-unix-prompt