Mar 102024
 

This is a collection of notes from my upgrade to an ASRock TRX50 WS motherboard fitted with an AMD Threadripper 7970X processor (32 cores) and 256Gbytes of memory. The upgrade meant that I retained the case, drives, graphics card, etc. from the previous system.

Most of the problems encountered were due to user stupidity.

First of all, whilst many of us have heard about the amount of time that DDR5 takes to “calibrate” itself, what I didn’t know was that the firmware status code shows “00” during this process (a dedicated “I’m messing with memory” code would be handy). And whilst it takes a while to do, if it takes longer than about 5m, then something else is wrong.

In my case it turned out that I hadn’t read the instructions properly and I hadn’t connected enough power connectors. To get it to work, I needed the usual 24-pin power connector, an 8-pin connector, and a 6-pin connector all connected on the “drive” side of the motherboard (opposite the side with the PCIe slots). Once that was sorted, the system was up and running.

The remaining notes relate to “tweaking”.

Booting Linux

Of course I use Linux, what the hell else would I use? FreeBSD? Well, that would be a good choice.

The biggest problem I had booting Linux was changing the netplan configuration to pick up the new network interfaces. In my case, the Marvell interface (the 10G one) came up as enp65s0 and the Realtek interface (the 2.5G one) as enp69s0. Because I’m bound to plug the cable into the wrong interface, I simply bonded the two interfaces together; the relevant section of my netplan configuration is as follows :-

network:
  version: 2
  renderer: networkd
  bonds:
    james:
      interfaces: [enp65s0, enp69s0]
  ethernets:
    enp65s0: {}
    enp69s0: {}
  bridges:

Yes, you can choose silly names here. And yes the bonding works fine – just now I swapped the cable over to the “right” NIC with numerous active network connections, and everything stayed alive.

Firmware Upgrade

The motherboard was supplied with version 6.04 of the firmware (I refuse to call this a “BIOS” because it just isn’t “basic” any more) whereas the latest was 7.09. The process is fairly simple :-

  1. Download the relevant firmware version from https://www.asrock.com/mb/AMD/TRX50%20WS/index.asp#BIOS.
  2. Save it to a FAT32 USB disk – I used a vfat formatted disk and I have a sneaking suspicion that exFAT will work too. The “Instant Flash” instructions by ASRock are obviously somewhat dated – it even mentions that saving to a floppy disk will work!
  3. Reboot the system and start the UEFI firmware. Select “Tools” and “Instant Flash”.
  4. Follow the on-screen instructions.

If you’re replacing a motherboard you won’t need detailed instructions here, but it is worth mentioning that the process takes a couple of reboots, and the second involves doing that memory calibration thing, so it takes an unusually long time to start.

I didn’t go to the effort to time the whole process, but my system went down at 18:04 and was back up at 18:15. So roughly 10 minutes.

SlimSAS

This isn’t currently 100% confident as I haven’t plugged anything in yet (ignoring a failed attempt when I assumed it work just work), but the SlimSAS ports can be configured for SATA mode in the firmware. Just go to Advanced, Chipset, go to the end of the list (which involves scrolling) past the settings for the PCIe slot configuration parameters and set :-

  1. SLIMSAS1 Mode: SATA
  2. SLIMSAS2 Mode: SATA

Firmware Settings

The following settings are what I chose to set based on a very quick session search Duckduckgo for explanations. The built-in documentation is somewhat lacking although there are URLs (encoded as QR codes) for more details. This is one area where firmware authors should pay more attention – even if they just hinted which settings work best for Windows, which work best for Linux, and which ones are for compatibility for older hardware.

The choices I’ve made may not be the best, but it seems to be working. Some of the explanations may be off, so I’d welcome corrections. All of these settings are found under the “Advanced” tab of the firmware page :-

CPU Configuration

  1. SMT: Or “hyperthreading”. It is possible some scientific computing workloads might work better with this turned off, but my recommendation is to leave it to “Auto”.
  2. CPB – Core performance boost: presumably allows one core to accelerate when other cores are idle. Left on “Auto”.
  3. Global C-State control: related to power-saving. There’s a suggestion that disabling this may result in extra stability. Disabled.
  4. Local APIC Mode: controls how the APIC appears to the operating system with choices of Auto, Compatible, xAPIC, or 2xAPIC. Supposedly 2xAPIC allows for greater efficiency on higher core counts. Set to 2xAPIC.
  5. L1 Stream HW Prefetcher: Enables or disabled pre-fetching memory into cache. Enabled.
  6. L2 Stream HW Prefetcher: Enables or disabled pre-fetching memory into cache. Enabled.
  7. SMEE (SME?): Secure memory (i.e. encrypted) for virtual machines. Not likely to make much difference in my case as I’m the exclusive owner of both the “host” and all of the virtual machines running on it. Left as “Auto”.
  8. SEV-ES ASID Space Limit Control: More on virtual machine security. Left on Auto.
  9. SVM mode: This option seemed to disappear on the upgrade to 7.09. If this does appear, enable it.
  10. ROM Armor: protection for SPI flash. Left as Enabled.

Chipset

  1. IOMMU: virtual machine I/O virtualisation to allow PCIe pass-through to a virtual machine. Enabled.
  2. ACS: More I/O virtualisation. Suggestions hinting at allowing PCIe←→PCIe transfers. Some hints at better IOMMU set up. Enabled.
  3. Enable AER Cap: PCIe error handling. Presumably disabling Linux AER error handling. Disabled.
  4. PCIe ARI Support: Enables support for ARI which allows a device to more easily support pretending to be multiple devices (so a graphics card could be shared amongst multiple virtual machines). Although card support for this is probably quite rare, I enabled it anyway.
  5. PCIe Ten Bit Tag Support: Allows a supporting device to use greater bandwidth and lower latency. Enabled.
  6. NUMA node(s) per socket: It is suggested that this allows the processor’s CCXes (the ‘core complex’ that appears as individual chiplets in an AMD processor) to operate as separate NUMA nodes. Set to NPS4.
  7. ACPI SRAC L3 Cache as NUMA domain: It is suggested that this also allows each CCX to function as a NUMA node. Enabled.
  8. TSME: Or Transparent SME. Support for SME is done by the firmware rather than the OS. Disabled.
  9. HPET: High Precision Timer. Enables support for a newer way of doing timing. Enabled.
  10. … (missing details because they weren’t of interest to me)
  11. SLIMSAS1 Mode/SLIMSAS2 Mode: As mentioned previously, allows switching the SlimSAS ports from supporting NVME devices to supporting SATA devices. Switched to SATA mode!

PCI

  1. PCI latency timer: How many clock cycles a 32-bit PCIe card can hang onto the bus for. Leave alone (32 cycles).
  2. PCI-X latency timer: How many clock cycles a 64-bit PCIe card can hang onto the bus for. Leave alone.
  3. VGA Palette Snoop: Whether to allow other cards to snoop on the VGA palette which is used by older cards for video encoding and the like. Disabled.
  4. PERR# Generation: Something to do with PCIe card errors. Left alone.
  5. SERR# Generation: Something to do with PCIe card errors. Left alone.
  6. Above 4G Decoding: Allows card to specify a 64-bit address to house their memory window. Enabled.
  7. Re-size BAR Support: Allows a card to negotiate a larger address window than the default of 256Mbytes. Enabled.
  8. SR-IOV Support: Where PCIe cards allow, enables the creation of virtual devices to be allocated to virtual machines. Enabled.
  9. BME DMA Mitigation: Re-enable Bus Master Attribute after SMM is locked. Whatever that means! Left disabled.
Feb 122024
 

So two days ago, I upgraded my main workstation to Ubuntu 23.10; a few little issues (mostly related to my own scripts), but nothing serious. Yet.

On the following day, my smart TV box started misbehaving. It couldn’t see any of the videos NFS mounted from my workstation, ITVX threw up a website error (this should have been a clue), but Youtube worked fine (which showed that the network was working fine).

So I did the obvious thing and started checking the NFS parameters to see if anything had changed. Nothing definite but on the way I noticed that the TV box wasn’t getting an IPv4 address from the dhcp server; IPv6 was working fine but some services don’t work on an IPv6 network.

I foolishly assumed that the TV box had stopped requesting addresses via dhcp – backed by the dhcp logs which showed no requests had been logged since the previous day. Set a static address, and everything sprang into life (except for ITVX who seem to have decided that only approved TV boxes should be allowed to run their code).

Later that same day, I upgraded a switch which failed to come back (“Failed to adopt”) which caused a daisy-chained wireless access point to disappear (“Failed to adopt”). And then a little while later, a second unconnected wireless access point also disappeared.

After a few reboots of the switch (and access points), I finally checked the dhcp server and found that its root filesystem had become ‘read-only’. But that wasn’t the end of the misdiagnosis …

I assumed that the SD card in my dhcp server (a tiny ARM box) was fried, so made arrangements to backup the contents, buy a couple of replacements, and try a spare (which was broken). After the spare turned out to be broken, I ran fsck on the root filesystem of the original and a whole bunch of errors were fixed.

Re-installed into the ARM box, and everything sprang to life again.

I guess the moral of the story is that you should check the basic services before diving into making assumptions.

Upended Cannon
Jan 142024
 

Just seen a video title about how Linux defeated UNIX™; it is quite hard to dispute this givennd that that Linux is alive, well, and thriving. But I would argue that it isn’t quite true.

First of all, UNIX™ is technically alive as Solaris, HP-UX and AIX are still active. And there may well be rarer versions out there – and I’m excluding operating systems that meet the trademark requirements but aren’t really “Unix” (we could argue all day about what is and what isn’t “Unix”).

But the market for UNIX™ machines is a great deal smaller than it used to be. And why is that? I would argue that whilst Linux made the transition easier, it isn’t the real reason why many organisations swapped out their high-priced machines for cheaper machines.

And that gives a bit of a clue. Whilst the high-priced machines from Sun, SGI, HP, IBM, Digital, etc. weren’t over-priced they were expensive. The hardware was built to be exceptionally reliable – for example some of the Suns I worked with could deal with a processor failure by simply turning off that processor and letting an engineer replace the board all whilst the system was up and running.

No what “killed” those expensive UNIX™ machines was virtualisation and the use of commodity hardware. If a modern server dies, the virtual servers running on it are simply migrated to a working server suffering at worst a reboot (but probably not).

Plus there was a realisation that not everything needed to be continually available.

Through The Gateway
Dec 042023
 

Just for fun (I have admittedly a very weird sense of fun), I thought I’d have a look at one of the phishing emails that came into me. I’ll go through this bit by bit, picking out bits that first occurred to me …

Subject: LastPass : Required action needed regarding your account

Eh? Do I even have a LastPass account? I keep my passwords stored somewhere else, but it’s not impossible – I’ve been known to sign up to things just to test them out. Including cloud-based password managers.

But all the same, let’s give it a point on the suspicion scale. Running total: 1.

From: LastPass <yoji-okugawa1975@tg8.so-net.ne.jp>

Well LastPass certainly use a funny looking email domain (the bit to the right of the “@”), but Marketing departments sometimes aren’t aware of how important that email domain really is. On the other hand, “tg8.so-net.net.jp” does look particularly uncorporate, so let us give it a suspicion point.

Running total: 2

On the other hand, it is too easy to fake domains – I could very easily send you an email from the-management@lástpáss.com (and even more subtle equivalents of “a” – “а”, “ạ”, “ą”, “ä”, “à”, “á”, “ą”). And just to demonstrate something that looks identical can actually be quite different :-

In [8]: print(ord('а'))
1072

In [9]: print(ord('a'))
97

Now this isn’t to suggest that you should run your email headers through some Python code, but just that because something looks like lastpass.com doesn’t mean it really is. The next thing that jumped out at me was the body of the email – I may be well trained, but something new and shiny is still distracting :-

Now the first thing that jumps out at me is that red “Confirm my information” box. Screams “click here” doesn’t it? Well don’t click on it! In my email client (something you’re quite likely not using – claws-mail), if I hold the mouse pointed above a link, it’ll tell me where that link goes in the status bar of the client. In this case it shows up as https://tg8.benchurl.com/…. doesn’t look very much like lastpass.com does it? That’s sufficiently suspicious that I’ll award it 3 suspicion points.

Running total: 5

Notice how they don’t add a “Dear ${name}” to the top of the email? Not personally addressing email is ever so convenient to scammers that want to get your details – because they don’t necessarily know your name. That’s a suspicion point all on its own.

Running total: 6

Next note how it tries to rush you … “log in before January 16, 2024”. It’s subtler than many phishing scams, but it’s still trying to rush you. Add another suspicion point.

Running total: 7

There’s further details we could dig into, but that’s more than enough that the Delete button is the only thing this email should attract. That running total? It was just for fun, it’s not intended as a guideline for when to count something as a phishing email.

In the case of doubt, contact the company via other means.

Oct 132023
 

Do you have a disk in your computer to keep data on? Really? It must be quite old then. Most of us are switching to solid-state devices.

And even if your hard disk really is spinning rust, it technically isn’t one disk; it’s a number of them (individually called platters).

IBM terms all appropriate storage devices DASDs (direct-access storage device) which because it refers to what the storage device does rather than describes how it is constructed. Except for the difficulty pronouncing it, it makes a far better name.

How about cheating and referring to them as DASes?

Wooden and Concrete Seating