Jan 142024
 

Just seen a video title about how Linux defeated UNIX™; it is quite hard to dispute this givennd that that Linux is alive, well, and thriving. But I would argue that it isn’t quite true.

First of all, UNIX™ is technically alive as Solaris, HP-UX and AIX are still active. And there may well be rarer versions out there – and I’m excluding operating systems that meet the trademark requirements but aren’t really “Unix” (we could argue all day about what is and what isn’t “Unix”).

But the market for UNIX™ machines is a great deal smaller than it used to be. And why is that? I would argue that whilst Linux made the transition easier, it isn’t the real reason why many organisations swapped out their high-priced machines for cheaper machines.

And that gives a bit of a clue. Whilst the high-priced machines from Sun, SGI, HP, IBM, Digital, etc. weren’t over-priced they were expensive. The hardware was built to be exceptionally reliable – for example some of the Suns I worked with could deal with a processor failure by simply turning off that processor and letting an engineer replace the board all whilst the system was up and running.

No what “killed” those expensive UNIX™ machines was virtualisation and the use of commodity hardware. If a modern server dies, the virtual servers running on it are simply migrated to a working server suffering at worst a reboot (but probably not).

Plus there was a realisation that not everything needed to be continually available.

Through The Gateway
Jun 112023
 

Ah yes! The eternal debate on how to do storage under Linux (and previously Unix). This debate has been going on since Unix found itself with some disks.

No, but …

First of all, in the simplest case of installing Linux onto a machine with a single disk isn’t the only possibility here. If you are in that situation, you do need to consider a separate partition for the /home file system.

But there are all sorts of other possibilities here – for example my own workstation has a separate /home file system but it comes from another (ZFS) storage pool of disks. So my system disk doesn’t have a separate partition for /home. If you are using extra disks you’ll almost always want a separate /home file system.

But before we get too deep into the technical terms, what exactly are they?

  1. When setting up a new disk, you can divide it up into 1 or more partitions which to the operating system look pretty much like disks – you can use nvme0n1 to create a file system, or you can use nvme0n1p1. On a system disk you will very often have three or more – one each for /boot, /boot/efi, and / (at the least).
  2. Once Linux has taken over a disk and “formatted” it for its use, it has a data structure on it that makes it a file system of one type or another. This file system can be mounted at any point in the hierarchy, so historically (when we had much smaller disks), there could be file systems mounted at /, /usr, /var, /var/spool, /usr/local, etc.

So do we need a separate /home file system? Of course not, but that doesn’t mean it isn’t a wise thing to do.

If you’re setting up a throw-away laptop that won’t ever store anything important, then sure a separate /home file system is probably a waste of time – it’ll probably only ever store some configuration files. If the system you’re setting up is your main machine and /home contains all your files – past, present, and future, then a separate /home file system is worth considering.

  1. If you ever re-install the operating system, your separate /home could be preserved so that you don’t have to restore from backup. That isn’t safe (so you should always have a backup elsewhere) but it can be done fairly easily (with enough practice).
  2. If you upgrade your storage, a separate /home file system can be quicker and safer to copy (at the file system level) to the new storage. Doing it on a file-by-file basis (such as with the excellent rsync) is likely to be very much slower than doing it at the file system level (such as with zfs send).
  3. It should be a great deal easier to take important backups if all the important files are on one file system.
Peering At Each Other
Jul 232022
 

I was following one of those Twitter threads posting their favourite command-line tools (specifically for infosec), and added my own entry – the incomparable tshark. Later it occurred to me that the best command-line tool isn’t really a tool at all as it is built into the shell – the pipe. Many of the command-line tools just wouldn’t be quite the same without it.

For those who aren’t familiar with the command-line, the pipe (“|”) takes the output of one command and feeds it as input to another command. And you can string such pipelines together to add to each other (which can lead to inefficiencies).

For example :-

» ls | wc -l
84

This takes the usual command for listing files and sends the output into the “word count” command to produce a count of the number of files in the current directory. To be more precise, it produces a count of the number of files that ls thinks is in the directory. You can get different results with different variations :-

» echo * | wc -w
89
» ls -a | wc -l
463

If you had a log file containing DHCP requests you could :-

» grep DHCPDISCOVER 2022.07.local0.info.log | head
2022-06-30T23:59:05+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T01:30:04+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T02:53:33+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via eth0
2022-07-01T02:53:33+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via 10.0.0.1
2022-07-01T02:53:39+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from a8:a6:48:92:9d:36 via eth0
2022-07-01T03:01:03+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T04:32:02+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T04:56:53+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 91:06:27:15:EF:DC via 10.72.0.1
2022-07-01T06:03:01+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T07:34:00+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1

List out the first few DHCP DISCOVER requests (the astute may notice that I’ve done some obfuscating). We can then pick out a field using awk to list just the MAC addresses :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | head
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4
DF:69:AF:DC:79:3E
DF:69:AF:DC:79:3E
a8:a6:48:92:9d:36
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4
91:06:27:15:EF:DC
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4 

We can then remove the “head” command and add a sort and uniq command to produce a full list of all MAC addresses that have performed a DHCP DISCOVER :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c
      4 DF:69:AF:DC:79:3E
      3 89:C1:67:B8:9D:6F
      6 F3:55:1E:06:D4:49
      4 F3:55:1E:06:D4:48
     12 4D:6D:4F:55:59:B3
     92 91:06:27:15:EF:DC
     46 85:2C:B4:B3:70:7E
    333 4D:6D:4F:55:59:B4
      2 40:5B:D8:FF:FA:29
     72 FD:D4:00:41:29:BE
      5 36:1E:07:2D:AD:76
     41 44:FD:6E:05:82:21
     81 CC:78:14:BB:E4:3D

We can sort the result into reverse numerical order :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n
    333 4D:6D:4F:55:59:B4
     92 91:06:27:15:EF:DC
     81 CC:78:14:BB:E4:3D
     72 FD:D4:00:41:29:BE
     46 85:2C:B4:B3:70:7E
     41 44:FD:6E:05:82:21
     12 4D:6D:4F:55:59:B3
      6 F3:55:1E:06:D4:49
      5 36:1E:07:2D:AD:76
      4 F3:55:1E:06:D4:48
      4 DF:69:AF:DC:79:3E
      3 89:C1:67:B8:9D:6F
      2 40:5B:D8:FF:FA:29 

And if you have access to the relevant script, you can produce terminal graphics (just to keep innumerate managers happy) :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n | awk '{print $2, $1}' | tbar --replace 1 --max 350
4D:6D:4F:55:59:B4 ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
91:06:27:15:EF:DC ■■■■■■■■■■■■■■■
CC:78:14:BB:E4:3D ■■■■■■■■■■■■■■
FD:D4:00:41:29:BE ■■■■■■■■■■■■
85:2C:B4:B3:70:7E ■■■■■■■
44:FD:6E:05:82:21 ■■■■■■■
4D:6D:4F:55:59:B3 ■■
F3:55:1E:06:D4:49 ■
36:1E:07:2D:AD:76 
F3:55:1E:06:D4:48 
DF:69:AF:DC:79:3E 
89:C1:67:B8:9D:6F 
40:5B:D8:FF:FA:29 

The pipe isn’t so much a tool itself as a mechanism to combine tools into producing interesting results.

It’s Round
Jul 132022
 

Not all shell aliases of course, but some. I’ve just seen a youtube video that suggested creating a shell alias to run rmtrash when rm is invoked :-

alias rm='rmtrash'

Seems sensible enough doesn’t it? This is in fact the classic example of how dangerous shell aliases can be, although the classic example was to turn on “-i” :-

alias rm='rm -i'

The problem is that you get used to “rm” being safe – either it asks before it removes files (“-i”) or it safely preserves what is deleted in the Trash folder. But what happens when the alias doesn’t get created? Perhaps you have a broken .zshrc and Zsh stops interpreting before the alias is declared. Or you’ve logged on to a remote server that doesn’t have your .zshrc installed as yet?

All of a sudden you are running the unadulterated rm command – deleting files without being asked, or preserving them in the Trash folder. See the danger now?

It is better not to replace standard commands but create a new ‘command’ :-

alias del="rmtrash"

Perhaps you regard this as being excessively risk averse – fair enough. But just don’t say you weren’t warned – and I’ve encountered missing aliases every year over the last 30-odd years I’ve been using Linux and Unix.

The Bare Family
Nov 112021
 

If you follow a certain Linux on Youtube, you may well be aware of an incident where Linus was trying to install Steam on a newly installed copy of Pop_OS! and managed to produce a bit of a mess without a desktop environment. What happened?

I think that when he encountered a problem installing Steam with the gooey, he then obtained a command-line “recipe” for installing Steam – potentially for a different distribution (it certainly mentioned removing lots of “stuff” including gnome-desktop).

Is this a problem with Linus being a bit of an idiot or Linux being a bit broken? A bit of both perhaps.

Linus’ idiocy is perhaps an example of a little knowledge being a dangerous thing – he mentioned being comfortable with using the command-line, but would admit that he doesn’t understand everything that goes on within it (to be fair, nobody understands everything even those who’ve been using the Linux command-line for over 20 years). And certainly when apt said “To install this package, I’m going to remove this long list of other packages”, the appropriately cautious should be saying “No” (and yes there is a prompt to allow you to do that).

The Linux command-line follows the principle that if the human wielding it wants to do something dumb, it may warn you but it will let you do whatever you want. That’s handy but scary and dangerous.

Now most users will likely veer away from the command-line – this is where Linus was a bit of an idiot – at least until they have a bit more experience. But perhaps those who make distributions should make the danger a bit more dangerous by adding a warning when opening the terminal (added to ~/.profile so we can remove an unnecessary warning) :-

WARNING !!!!

The command-line can be dangerous if you are not careful. Pasting in "recipes" found on the Internet for solutions to issues can result in serious damage to your Linux installation requiring re-installation.

In particular a recipe should be specific to your distribution and the version of the distribution you are running. 

When looking for solutions on the Internet, always bear in mind that there are idiots out there who will publish “solutions” that are anything but. As mentioned in my hypothetical warning, recipes are very often (especially when dealing with software installation) specific to a particular distribution and version – use it inappropriately and you may well run into serious trouble.

On the subject of gooeys, it would be handy to include a “Solutions” link when an error occurs in a software package manager that takes you to a web page specific to the package you are trying to install. Encounter trouble installing “Steam 6.23”? The solutions link might take you to a page saying “This package is out of date; please run Update”. This would allow links to be specific to the distribution and version in use – a lot more helpful than simply expecting the user to search the Internet for a solution.

King Alfred Looking Down At The Runners