Jul 032024
 

Well, this is kind of a weird one. But most of the systems I run are Linux-based, and over the years I have ‘developed’ a simple script that I run from my main workstation which iterates through all of the systems applying updates.

As non-interactively as possible – it could even be scheduled to run automatically (although I don’t for no good reason).

But it had one great weakness – it didn’t update my Windows 11 virtual machine. Which wasn’t a serious problem because Windows could and did update itself. But it did result in software installed with winget getting left behind.

So I sorted it …

  1. Install OpenSSH server on Windows: PS: Add-WindowsCapability -Online -Name OpenSSH.Server (this might need the version number which is best obtained using Get-WindowsCapability -Online | Where-Object Name -like ‘OpenSSH*’.
  2. Copy your chosen ssh authentication public key into c:\users\${username}\.ssh\authorized_keys file.
  3. Configure c:\programdata\ssh\sshd_config to permit public key authentication (“PubkeyAuthentication yes”).
  4. Whilst in the same file, comment out the section with the line beginning “Match Group administrators” which whilst makes things less secure did at least work! The section does refer to a file: c:/ProgramData/ssh/administrators_authorized_keys but adding to this file didn’t seem to work for me.
  5. Verify that the daemon is running: PS: get-service -name sshd
    • If it shows as not running, enable with: PS: set-service -name sshd -startuptype ‘automatic’
    • And either reboot, or start it manually: PS: start-service -name sshd
  6. At this point you should be able to login with a simple ssh username@hostname command. If not you’ve either left something out, or I have!

At this point you should be able to run the relevant update commands :-

  1. ssh username@hostname UsoClient ScanInstallWait. Operating system updates which may or may not work, so I wouldn’t disable the automatic updates at this point.
  2. ssh username@hostname winget upgrade –all. This updates additional software (something I’ve called “layered products” in the past) installed via winget (or the Microsoft “Store”. This can sometimes stop with a mysterious error but should usually work.

No guarantees with this sort of thing!

Peering At Each Other
Jun 092024
 

So on Friday my workstation blew up … which goes some way to explaining why this server has been down for much of the weekend (it’s a container on my workstation). The operating system boot drive magically went read-only – which as some of us know is a clear indication that an SSD is on its last legs. Or a few steps beyond.

So I re-installed on a new drive, and for various reasons I chose Ubuntu Server again. An upgrade which made things a bit more interesting.

Now whilst I know that most servers are installed in ways other than interactively, but the interactive experience is bloody awful.

For a start, if the text console is a ridiculous large size – perhaps $COLUMNS is greater than 160, then change the bloody console font. It may be a rare case that someone is installing Ubuntu server on a screen where the resolution is so high, but it can (and in my case does) make the text far too small to read.

And secondly, do something about the logical volume manager creation; I wanted to create a logical volume with a name other than “ubuntu-vg” (the old drive was still readable and creating two VGs with the same name struck me as a dumb idea). The default method didn’t seem to offer a way to rename the VG; the custom method kept giving me an installation error.

The later could possibly be my mistake – I was reduced to using a magnifying glass.

And yes everything is working now.

But sort out that installer!

In The Crack
Jan 142024
 

Just seen a video title about how Linux defeated UNIX™; it is quite hard to dispute this givennd that that Linux is alive, well, and thriving. But I would argue that it isn’t quite true.

First of all, UNIX™ is technically alive as Solaris, HP-UX and AIX are still active. And there may well be rarer versions out there – and I’m excluding operating systems that meet the trademark requirements but aren’t really “Unix” (we could argue all day about what is and what isn’t “Unix”).

But the market for UNIX™ machines is a great deal smaller than it used to be. And why is that? I would argue that whilst Linux made the transition easier, it isn’t the real reason why many organisations swapped out their high-priced machines for cheaper machines.

And that gives a bit of a clue. Whilst the high-priced machines from Sun, SGI, HP, IBM, Digital, etc. weren’t over-priced they were expensive. The hardware was built to be exceptionally reliable – for example some of the Suns I worked with could deal with a processor failure by simply turning off that processor and letting an engineer replace the board all whilst the system was up and running.

No what “killed” those expensive UNIX™ machines was virtualisation and the use of commodity hardware. If a modern server dies, the virtual servers running on it are simply migrated to a working server suffering at worst a reboot (but probably not).

Plus there was a realisation that not everything needed to be continually available.

Through The Gateway
Jun 112023
 

Ah yes! The eternal debate on how to do storage under Linux (and previously Unix). This debate has been going on since Unix found itself with some disks.

No, but …

First of all, in the simplest case of installing Linux onto a machine with a single disk isn’t the only possibility here. If you are in that situation, you do need to consider a separate partition for the /home file system.

But there are all sorts of other possibilities here – for example my own workstation has a separate /home file system but it comes from another (ZFS) storage pool of disks. So my system disk doesn’t have a separate partition for /home. If you are using extra disks you’ll almost always want a separate /home file system.

But before we get too deep into the technical terms, what exactly are they?

  1. When setting up a new disk, you can divide it up into 1 or more partitions which to the operating system look pretty much like disks – you can use nvme0n1 to create a file system, or you can use nvme0n1p1. On a system disk you will very often have three or more – one each for /boot, /boot/efi, and / (at the least).
  2. Once Linux has taken over a disk and “formatted” it for its use, it has a data structure on it that makes it a file system of one type or another. This file system can be mounted at any point in the hierarchy, so historically (when we had much smaller disks), there could be file systems mounted at /, /usr, /var, /var/spool, /usr/local, etc.

So do we need a separate /home file system? Of course not, but that doesn’t mean it isn’t a wise thing to do.

If you’re setting up a throw-away laptop that won’t ever store anything important, then sure a separate /home file system is probably a waste of time – it’ll probably only ever store some configuration files. If the system you’re setting up is your main machine and /home contains all your files – past, present, and future, then a separate /home file system is worth considering.

  1. If you ever re-install the operating system, your separate /home could be preserved so that you don’t have to restore from backup. That isn’t safe (so you should always have a backup elsewhere) but it can be done fairly easily (with enough practice).
  2. If you upgrade your storage, a separate /home file system can be quicker and safer to copy (at the file system level) to the new storage. Doing it on a file-by-file basis (such as with the excellent rsync) is likely to be very much slower than doing it at the file system level (such as with zfs send).
  3. It should be a great deal easier to take important backups if all the important files are on one file system.
Peering At Each Other
Jul 232022
 

I was following one of those Twitter threads posting their favourite command-line tools (specifically for infosec), and added my own entry – the incomparable tshark. Later it occurred to me that the best command-line tool isn’t really a tool at all as it is built into the shell – the pipe. Many of the command-line tools just wouldn’t be quite the same without it.

For those who aren’t familiar with the command-line, the pipe (“|”) takes the output of one command and feeds it as input to another command. And you can string such pipelines together to add to each other (which can lead to inefficiencies).

For example :-

» ls | wc -l
84

This takes the usual command for listing files and sends the output into the “word count” command to produce a count of the number of files in the current directory. To be more precise, it produces a count of the number of files that ls thinks is in the directory. You can get different results with different variations :-

» echo * | wc -w
89
» ls -a | wc -l
463

If you had a log file containing DHCP requests you could :-

» grep DHCPDISCOVER 2022.07.local0.info.log | head
2022-06-30T23:59:05+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T01:30:04+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T02:53:33+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via eth0
2022-07-01T02:53:33+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via 10.0.0.1
2022-07-01T02:53:39+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from a8:a6:48:92:9d:36 via eth0
2022-07-01T03:01:03+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T04:32:02+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T04:56:53+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 91:06:27:15:EF:DC via 10.72.0.1
2022-07-01T06:03:01+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T07:34:00+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1

List out the first few DHCP DISCOVER requests (the astute may notice that I’ve done some obfuscating). We can then pick out a field using awk to list just the MAC addresses :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | head
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4
DF:69:AF:DC:79:3E
DF:69:AF:DC:79:3E
a8:a6:48:92:9d:36
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4
91:06:27:15:EF:DC
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4 

We can then remove the “head” command and add a sort and uniq command to produce a full list of all MAC addresses that have performed a DHCP DISCOVER :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c
      4 DF:69:AF:DC:79:3E
      3 89:C1:67:B8:9D:6F
      6 F3:55:1E:06:D4:49
      4 F3:55:1E:06:D4:48
     12 4D:6D:4F:55:59:B3
     92 91:06:27:15:EF:DC
     46 85:2C:B4:B3:70:7E
    333 4D:6D:4F:55:59:B4
      2 40:5B:D8:FF:FA:29
     72 FD:D4:00:41:29:BE
      5 36:1E:07:2D:AD:76
     41 44:FD:6E:05:82:21
     81 CC:78:14:BB:E4:3D

We can sort the result into reverse numerical order :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n
    333 4D:6D:4F:55:59:B4
     92 91:06:27:15:EF:DC
     81 CC:78:14:BB:E4:3D
     72 FD:D4:00:41:29:BE
     46 85:2C:B4:B3:70:7E
     41 44:FD:6E:05:82:21
     12 4D:6D:4F:55:59:B3
      6 F3:55:1E:06:D4:49
      5 36:1E:07:2D:AD:76
      4 F3:55:1E:06:D4:48
      4 DF:69:AF:DC:79:3E
      3 89:C1:67:B8:9D:6F
      2 40:5B:D8:FF:FA:29 

And if you have access to the relevant script, you can produce terminal graphics (just to keep innumerate managers happy) :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n | awk '{print $2, $1}' | tbar --replace 1 --max 350
4D:6D:4F:55:59:B4 ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
91:06:27:15:EF:DC ■■■■■■■■■■■■■■■
CC:78:14:BB:E4:3D ■■■■■■■■■■■■■■
FD:D4:00:41:29:BE ■■■■■■■■■■■■
85:2C:B4:B3:70:7E ■■■■■■■
44:FD:6E:05:82:21 ■■■■■■■
4D:6D:4F:55:59:B3 ■■
F3:55:1E:06:D4:49 ■
36:1E:07:2D:AD:76 
F3:55:1E:06:D4:48 
DF:69:AF:DC:79:3E 
89:C1:67:B8:9D:6F 
40:5B:D8:FF:FA:29 

The pipe isn’t so much a tool itself as a mechanism to combine tools into producing interesting results.

It’s Round