Jul 202024

This is a bit of rant poking fun at the sheer quantity of misinformation about CrowdStrike’s little issue yesterday (to clarify when this post was written – more information will come out).


Some of the earliest symptoms of the issue were some Microsoft services having issues. Oddly enough I wasn’t using many of those yesterday (I usually do) except for Teams which didn’t seem to suffer … at least not as much.

It appears that Microsoft may run CrowdStrike Falcon on at least some of their servers (although the jury is still out on this one – some are saying it was an independent outage). Despite Microsoft having their own security tools (Defender), this isn’t quite as unlikely as it may seem – particularly safety conscious organisations may well run two of more anti-malware products.

And CrowdStrike is more mature than Defender at least in the fancy “behavioural analytics” area.

The Internet

… wasn’t broken at all. Many services were broken true enough, but probably more were working just as well as normal. Microsoft’s platforms are very widely used, and CrowdStrike is a big name in cybersecurity, so it is hardly surprising that there was so much disruption.

But to say this broke the Internet is a bit of an exaggeration. Kind of what you would expect from mainstream media.

Who Are CrowdStrike?

Not surprisingly, many people just haven’t heard this name before. It is very widely known in the cybersecurity community with a wide variety of security focused services, including top-flight anti-malware products.

But they don’t sell to individuals so they are not well known amongst the general community.

The product at the centre of all this is CrowdStrike Falcon, an anti-malware agent that goes a bit beyond “anti-virus” in that it attempts to go beyond blocking known viruses and attempts to block behaviours known to be malicious.

As such, it receives very frequent updates – up to every hour (although probably many hours) which puts this sort of catastrophic failure at a rate of somewhere in the order of 0.001%.

What Went Wrong?

This starts to get a bit technical …

Some of this was informed by CrowdStrike’s update; some by educated (I work in this field although I’m not familiar with CrowdStrike’s product) common sense.

First of all, this was not a kernel driver update (although the relevant filename made it appear so) but a content update. As previously mentioned, these are sent out very frequently. The content update triggered a bug in the kernel driver and caused a “blue screen of death“. This would repeat after every reboot until the relevant update was removed or updated (the crash doesn’t occur immediately which sometimes allows the agent to download a fixed update).

Secondly this update was tested before being released (do you really believe that an approximately 0.001% failure rate is achieved without testing?), but something went wrong with the testing process. We don’t know what, and CrowdStrike don’t either. Yet.

Why Was It So Widespread?

Simply because although generally unknown to the general public, CrowdStrike Falcon is generally regarded as an excellent security product and is very widely used. Perhaps more widely used than previously suspected.

But the whole Internet? Clearly not, but it’s in the mainstream’s media to be a bit ‘click-baity’ in their reports.

As A Statue
Jul 032024

Well, this is kind of a weird one. But most of the systems I run are Linux-based, and over the years I have ‘developed’ a simple script that I run from my main workstation which iterates through all of the systems applying updates.

As non-interactively as possible – it could even be scheduled to run automatically (although I don’t for no good reason).

But it had one great weakness – it didn’t update my Windows 11 virtual machine. Which wasn’t a serious problem because Windows could and did update itself. But it did result in software installed with winget getting left behind.

So I sorted it …

  1. Install OpenSSH server on Windows: PS: Add-WindowsCapability -Online -Name OpenSSH.Server (this might need the version number which is best obtained using Get-WindowsCapability -Online | Where-Object Name -like ‘OpenSSH*’.
  2. Copy your chosen ssh authentication public key into c:\users\${username}\.ssh\authorized_keys file.
  3. Configure c:\programdata\ssh\sshd_config to permit public key authentication (“PubkeyAuthentication yes”).
  4. Whilst in the same file, comment out the section with the line beginning “Match Group administrators” which whilst makes things less secure did at least work! The section does refer to a file: c:/ProgramData/ssh/administrators_authorized_keys but adding to this file didn’t seem to work for me.
  5. Verify that the daemon is running: PS: get-service -name sshd
    • If it shows as not running, enable with: PS: set-service -name sshd -startuptype ‘automatic’
    • And either reboot, or start it manually: PS: start-service -name sshd
  6. At this point you should be able to login with a simple ssh username@hostname command. If not you’ve either left something out, or I have!

At this point you should be able to run the relevant update commands :-

  1. ssh username@hostname UsoClient ScanInstallWait. Operating system updates which may or may not work, so I wouldn’t disable the automatic updates at this point.
  2. ssh username@hostname winget upgrade –all. This updates additional software (something I’ve called “layered products” in the past) installed via winget (or the Microsoft “Store”. This can sometimes stop with a mysterious error but should usually work.

No guarantees with this sort of thing!

Peering At Each Other
Jun 302024

The funny thing is that this post is not about window tiling but about conventional tiling window managers that to a great extent are “do it yourself” window managers. That is they kind of expect you to do a lot of configuration yourself.

This is not about specific tiling window managers even though I use Awesome and Hyprland.

Tiling vs Floating

In the earliest days of gooeys, windows tended to be tiled so that they didn’t overlap; if you added a new window to the screen, the existing windows would shrink to make room for the new window.

So-called (at least in tiling window managers) “floating” windows were hailed as a brilliant new feature where windows were independent and could overlap. Cool right?

And this is certainly the way that most conventional gooeys work – from Windows, MacOS, and even Linux, they all support floating windows by default. On the other hand, tiling window managers support tiling by default (most will support floating windows as well).

So why would you want to go back to the dawn of gooeys? And it is not just us weird geeks running minimalist window managers looking at tiling – many mainstream desktop environment have tiling features.

It maximises screen real-estate by automatically sizing windows so the whole desktop is covered. I can remember carefully dragging windows to resize them to maximise their size with conventional ‘floating’ window managers. Something that now happens automatically.


So tiling can be done with conventional desktop environments – some of which allow support for tiling. KDE has Bismouth; GNOME has the Tiling Assistant, and even Windows has options. Now a tiling window manager does treat tiling as a first class feature, that’s not really why those who prefer them like them.

No, it’s the minimalism and customisation.

Most desktop environments carry with them lots of bonus features – which is fine for most. But if you don’t need those features and/or want to do things in a different way, then a full desktop environment isn’t what you want.

And tiling window managers tend to be minimalistic; even to the extent that some do not include a status bar requiring an optional status bar to be installed. The default configuration (if any) tends to be minimalistic requiring lots of tinkering to get the most out of it.

Which is a positive feature for tinkerers if a negative feature for those who just want to be up and running. But tinkering whilst it takes time, does tend to product a more productive environment – for example my Hyprland configuration includes a custom key binding to grab IP addresses and URLs from a highlighted section of the screen (and optionally “defangs” safely specified dangerous URLs which give you a hint about what I do).

Picture of a Raven
Who Are You Looking At?
Jun 092024

So on Friday my workstation blew up … which goes some way to explaining why this server has been down for much of the weekend (it’s a container on my workstation). The operating system boot drive magically went read-only – which as some of us know is a clear indication that an SSD is on its last legs. Or a few steps beyond.

So I re-installed on a new drive, and for various reasons I chose Ubuntu Server again. An upgrade which made things a bit more interesting.

Now whilst I know that most servers are installed in ways other than interactively, but the interactive experience is bloody awful.

For a start, if the text console is a ridiculous large size – perhaps $COLUMNS is greater than 160, then change the bloody console font. It may be a rare case that someone is installing Ubuntu server on a screen where the resolution is so high, but it can (and in my case does) make the text far too small to read.

And secondly, do something about the logical volume manager creation; I wanted to create a logical volume with a name other than “ubuntu-vg” (the old drive was still readable and creating two VGs with the same name struck me as a dumb idea). The default method didn’t seem to offer a way to rename the VG; the custom method kept giving me an installation error.

The later could possibly be my mistake – I was reduced to using a magnifying glass.

And yes everything is working now.

But sort out that installer!

In The Crack
Apr 172024

Well that was interesting …

So I decided to upgrade the firmware on my ASRock TRX50 WS motherboard tonight. Partially because I had planned on trying it to sort out a mysterious crashing problem (which turned out to be the world’s worst SATA SSD ‘error’), and partially because I’d like to make sure I know how the process works. And funnily enough, finding ASRock’s instructions aren’t so simple.

The first really rather obvious step is to download the firmware from the ASRock support site. This comes down as a ZIP file, which needs to be unpacked :-

  • TRX50-WS_9.03.ROM

This needs to be copied to a USB stick formatted as FAT32, but whilst you’re checking that make sure that the partition type is set to an appropriate value (0x0b is the value I used; the second time), because it turns out that the ASRock firmware won’t recognise a FAT32 filesystem just based on the actual filesystem – it checks the partition types.

But before you shut down and start the upgrade process, record any firmware settings you may have made … for better or worse, the upgrade will reset any changes you have made.

Starting the upgrade is fairly simple – go into Setup, move across to Tools and select the “Instant Flash” option. This will pop up a menu of different firmware version files it has found that are compatible with your motherboard. Select the version you want (in my case it was just one option), and press Return.

After a warning, it’ll start the upgrade process; this consists of :-

  1. A progress bar which slowly progresses to 100%
  2. A reboot which takes you back into the firmware.
  3. A second progress bar which also progresses slowly.
  4. At some point when this has finished, it’ll just sit there for a few minutes and finally start booting with the new firmware.

Of course in my case, the settings reverting to default values resulted in the SlimSAS controllers both being reset to “NVME” rather than “SATA” meaning half my storage array wasn’t present! But it all worked in the end :-

✓ root@pica» dmidecode -s bios-version

Of course ASRock claim you only do a “BIOS Upgrade” (I hate that word “BIOS” – it’s not really appropriate) when it is absolutely necessary, but an upgrade when it isn’t necessary isn’t a bad idea. Just to get practice.

It should be noted that the firmware should be update-able with fwupdmgr so any urgent updates may well come via that route.

The Missing Sign