ASRock TRX50 WS & Threadripper 7970X

 Computing, Working Notes  No Responses »
Mar 102024
 

This is a collection of notes from my upgrade to an ASRock TRX50 WS motherboard fitted with an AMD Threadripper 7970X processor (32 cores) and 256Gbytes of memory. The upgrade meant that I retained the case, drives, graphics card, etc. from the previous system.

Most of the problems encountered were due to user stupidity.

First of all, whilst many of us have heard about the amount of time that DDR5 takes to “calibrate” itself, what I didn’t know was that the firmware status code shows “00” during this process (a dedicated “I’m messing with memory” code would be handy). And whilst it takes a while to do, if it takes longer than about 5m, then something else is wrong.

In my case it turned out that I hadn’t read the instructions properly and I hadn’t connected enough power connectors. To get it to work, I needed the usual 24-pin power connector, an 8-pin connector, and a 6-pin connector all connected on the “drive” side of the motherboard (opposite the side with the PCIe slots). Once that was sorted, the system was up and running.

The remaining notes relate to “tweaking”.

Booting Linux

Of course I use Linux, what the hell else would I use? FreeBSD? Well, that would be a good choice.

The biggest problem I had booting Linux was changing the netplan configuration to pick up the new network interfaces. In my case, the Marvell interface (the 10G one) came up as enp65s0 and the Realtek interface (the 2.5G one) as enp69s0. Because I’m bound to plug the cable into the wrong interface, I simply bonded the two interfaces together; the relevant section of my netplan configuration is as follows :-

network:
  version: 2
  renderer: networkd
  bonds:
    james:
      interfaces: [enp65s0, enp69s0]
  ethernets:
    enp65s0: {}
    enp69s0: {}
  bridges:

Yes, you can choose silly names here. And yes the bonding works fine – just now I swapped the cable over to the “right” NIC with numerous active network connections, and everything stayed alive.

Firmware Upgrade

The motherboard was supplied with version 6.04 of the firmware (I refuse to call this a “BIOS” because it just isn’t “basic” any more) whereas the latest was 7.09. The process is fairly simple :-

  1. Download the relevant firmware version from https://www.asrock.com/mb/AMD/TRX50%20WS/index.asp#BIOS.
  2. Save it to a FAT32 USB disk – I used a vfat formatted disk and I have a sneaking suspicion that exFAT will work too. The “Instant Flash” instructions by ASRock are obviously somewhat dated – it even mentions that saving to a floppy disk will work!
  3. Reboot the system and start the UEFI firmware. Select “Tools” and “Instant Flash”.
  4. Follow the on-screen instructions.

If you’re replacing a motherboard you won’t need detailed instructions here, but it is worth mentioning that the process takes a couple of reboots, and the second involves doing that memory calibration thing, so it takes an unusually long time to start.

I didn’t go to the effort to time the whole process, but my system went down at 18:04 and was back up at 18:15. So roughly 10 minutes.

SlimSAS

This isn’t currently 100% confident as I haven’t plugged anything in yet (ignoring a failed attempt when I assumed it work just work), but the SlimSAS ports can be configured for SATA mode in the firmware. Just go to Advanced, Chipset, go to the end of the list (which involves scrolling) past the settings for the PCIe slot configuration parameters and set :-

  1. SLIMSAS1 Mode: SATA
  2. SLIMSAS2 Mode: SATA

Firmware Settings

The following settings are what I chose to set based on a very quick session search Duckduckgo for explanations. The built-in documentation is somewhat lacking although there are URLs (encoded as QR codes) for more details. This is one area where firmware authors should pay more attention – even if they just hinted which settings work best for Windows, which work best for Linux, and which ones are for compatibility for older hardware.

The choices I’ve made may not be the best, but it seems to be working. Some of the explanations may be off, so I’d welcome corrections. All of these settings are found under the “Advanced” tab of the firmware page :-

CPU Configuration

  1. SMT: Or “hyperthreading”. It is possible some scientific computing workloads might work better with this turned off, but my recommendation is to leave it to “Auto”.
  2. CPB – Core performance boost: presumably allows one core to accelerate when other cores are idle. Left on “Auto”.
  3. Global C-State control: related to power-saving. There’s a suggestion that disabling this may result in extra stability. Disabled.
  4. Local APIC Mode: controls how the APIC appears to the operating system with choices of Auto, Compatible, xAPIC, or 2xAPIC. Supposedly 2xAPIC allows for greater efficiency on higher core counts. Set to 2xAPIC.
  5. L1 Stream HW Prefetcher: Enables or disabled pre-fetching memory into cache. Enabled.
  6. L2 Stream HW Prefetcher: Enables or disabled pre-fetching memory into cache. Enabled.
  7. SMEE (SME?): Secure memory (i.e. encrypted) for virtual machines. Not likely to make much difference in my case as I’m the exclusive owner of both the “host” and all of the virtual machines running on it. Left as “Auto”.
  8. SEV-ES ASID Space Limit Control: More on virtual machine security. Left on Auto.
  9. SVM mode: This option seemed to disappear on the upgrade to 7.09. If this does appear, enable it.
  10. ROM Armor: protection for SPI flash. Left as Enabled.

Chipset

  1. IOMMU: virtual machine I/O virtualisation to allow PCIe pass-through to a virtual machine. Enabled.
  2. ACS: More I/O virtualisation. Suggestions hinting at allowing PCIe←→PCIe transfers. Some hints at better IOMMU set up. Enabled.
  3. Enable AER Cap: PCIe error handling. Presumably disabling Linux AER error handling. Disabled.
  4. PCIe ARI Support: Enables support for ARI which allows a device to more easily support pretending to be multiple devices (so a graphics card could be shared amongst multiple virtual machines). Although card support for this is probably quite rare, I enabled it anyway.
  5. PCIe Ten Bit Tag Support: Allows a supporting device to use greater bandwidth and lower latency. Enabled.
  6. NUMA node(s) per socket: It is suggested that this allows the processor’s CCXes (the ‘core complex’ that appears as individual chiplets in an AMD processor) to operate as separate NUMA nodes. Set to NPS4.
  7. ACPI SRAC L3 Cache as NUMA domain: It is suggested that this also allows each CCX to function as a NUMA node. Enabled.
  8. TSME: Or Transparent SME. Support for SME is done by the firmware rather than the OS. Disabled.
  9. HPET: High Precision Timer. Enables support for a newer way of doing timing. Enabled.
  10. … (missing details because they weren’t of interest to me)
  11. SLIMSAS1 Mode/SLIMSAS2 Mode: As mentioned previously, allows switching the SlimSAS ports from supporting NVME devices to supporting SATA devices. Switched to SATA mode!

PCI

  1. PCI latency timer: How many clock cycles a 32-bit PCIe card can hang onto the bus for. Leave alone (32 cycles).
  2. PCI-X latency timer: How many clock cycles a 64-bit PCIe card can hang onto the bus for. Leave alone.
  3. VGA Palette Snoop: Whether to allow other cards to snoop on the VGA palette which is used by older cards for video encoding and the like. Disabled.
  4. PERR# Generation: Something to do with PCIe card errors. Left alone.
  5. SERR# Generation: Something to do with PCIe card errors. Left alone.
  6. Above 4G Decoding: Allows card to specify a 64-bit address to house their memory window. Enabled.
  7. Re-size BAR Support: Allows a card to negotiate a larger address window than the default of 256Mbytes. Enabled.
  8. SR-IOV Support: Where PCIe cards allow, enables the creation of virtual devices to be allocated to virtual machines. Enabled.
  9. BME DMA Mitigation: Re-enable Bus Master Attribute after SMM is locked. Whatever that means! Left disabled.

Diesel Generating Electricity for Electric Cars

 General, Politics  No Responses »
Mar 042024
 

I recently came across a post talking about how insane it was for a diesel generator to be generating electricity for an electric car charging station. Well, yes but it might not be quite as bad as it seems.

The first thing to bear in mind is that very few cars will charge here unless they really need to – most car users will charge at home and will only use public chargers when they have no other choice. And this one is undoubtedly in a remote location where a suitable electrical feed is not available. As a remote location, it’ll get even fewer users than most public chargers.

Probably only those who haven’t properly charged their car overnight and are ‘caught short’.

The second thing to bear in mind is that although it’s a more polluting solution, it is still almost certainly far less polluting than using a petrol car. For a start, most of the electricity used in an electric car is off the grid which whilst not completely clean is at least getting cleaner (the UK is currently generating 36% with renewables and is in late winter when demand is quite high).

Next, a diesel generator is a very efficient device – far more so than a diesel vehicle because it can always run at its most efficient (assuming a sensible load and it’ll have been picked to supply the load for an electric car).

But hopefully it is just a temporary solution until someone has enough nous to set up a solar farm and a battery backup.

Dawn At The End Of The Pier

The USA Isn’t The Best Country In The World

 General  No Responses »
Feb 182024
 

Nobody likes to hear their country isn’t the best country in the world – especially when said by a filthy foreigner. But the USA isn’t.

  1. The USA is ranked 59th by life expectancy.
  2. The USA is ranked 7th by GDP per capita.
  3. The USA is ranked 123rd by pregnancy mortality ratio (there are 180-odd positions and the lowest rate is best).
  4. The USA is ranked 12th by the World Freedom Index.

I could go on, and probably there’s a metric out there where the USA is #1. But I doubt reeling off metrics showing the USA isn’t the best is going to convince many.

But here’s another reason. Assuming your country is the best leads to complacency, which eventually leads it it not being the best country in the world. So even if your country is the best country in the world, pretend it isn’t and always strive to make it better.

The Bare Family

A Tale of Misdiagnosis

 Computing, Working Notes  No Responses »
Feb 122024
 

So two days ago, I upgraded my main workstation to Ubuntu 23.10; a few little issues (mostly related to my own scripts), but nothing serious. Yet.

On the following day, my smart TV box started misbehaving. It couldn’t see any of the videos NFS mounted from my workstation, ITVX threw up a website error (this should have been a clue), but Youtube worked fine (which showed that the network was working fine).

So I did the obvious thing and started checking the NFS parameters to see if anything had changed. Nothing definite but on the way I noticed that the TV box wasn’t getting an IPv4 address from the dhcp server; IPv6 was working fine but some services don’t work on an IPv6 network.

I foolishly assumed that the TV box had stopped requesting addresses via dhcp – backed by the dhcp logs which showed no requests had been logged since the previous day. Set a static address, and everything sprang into life (except for ITVX who seem to have decided that only approved TV boxes should be allowed to run their code).

Later that same day, I upgraded a switch which failed to come back (“Failed to adopt”) which caused a daisy-chained wireless access point to disappear (“Failed to adopt”). And then a little while later, a second unconnected wireless access point also disappeared.

After a few reboots of the switch (and access points), I finally checked the dhcp server and found that its root filesystem had become ‘read-only’. But that wasn’t the end of the misdiagnosis …

I assumed that the SD card in my dhcp server (a tiny ARM box) was fried, so made arrangements to backup the contents, buy a couple of replacements, and try a spare (which was broken). After the spare turned out to be broken, I ran fsck on the root filesystem of the original and a whole bunch of errors were fixed.

Re-installed into the ARM box, and everything sprang to life again.

I guess the moral of the story is that you should check the basic services before diving into making assumptions.

Upended Cannon

Why Tories Should Vote Against Tories

 Politics  No Responses »
Jan 142024
 

This is going to be a bit of a departure from the normal far-left propaganda and I dare say almost every Tory voter will be ignoring it. But maybe a few will at least listen – at least to me it makes sense. And not because I’ll not be voting Tory myself either.

Today’s Tory party is broken – to at least the extent that the Labour party was broken during the era of the Militant tendency; they have been infiltrated by two (at least) factions of the far right.

The first are your classic proto-fascists – nationalistic to the extreme, anti-immigration, and worried by “woke-ism” to the extent they’ll by really nasty to those the wokeists will say you should be nice to. And I don’t just mean with words – they’re basic thugs.

The second are disaster capitalists who don’t care about the country. They’re after enriching themselves and their rich mates. They’re after tax cuts – not because they believe in small government but because they want to cut taxes for the rich.

The fix for this is to smash the party at the next election; anything less won’t work. The party needs to be reduced to the point that the extremists will give up on the party and leave for greener pastures. Hopefully they won’t find a new home.

It is quite possible that a loss at the next election won’t be quite enough – real Tories need to get out and shout down the extremists at every opportunity. The party needs to look closely at what the public as a whole wants – for example, sabotaging the NHS over the last decade has made much of the public very, very angry.

And the party needs to look very closely at why the young steer clear – the way things are going at the moment, the Tory party will be having an annual conference in a back street pub in a couple of decades. The old adage about young people becoming more right-wing as they age is almost certainly wrong, but even if it’s right, they’ll be looking to the right-wing of the Labour party or the Liberals as their home.

Why am I trying to help the Tories? Because I’ll enjoy seeing the Tories have an election result that’ll make the 1906 disaster look like a tea party of course. But a relatively small loss won’t cause the party to do a proper reflection to the extent they need.

The Lighthouse
 Older Entries  Newer Entries