No ads? Contribute with BitCoins: 16hQid2ddoCwHDWN9NdSnARAfdXc2Shnoa
Jun 122018

This posting is about using the command-line ssh tool for relatively securely copying stuff around, and logging into devices. Many of the tips contained within are things I have had to pry out of the manual page for my own use and these notes are a way of keeping the information around without relying on my brain.

#1: It Comes With Windows

If you are running the latest version of Windows 10, you get the command-line versions of ssh and scp without dropping into the Linux shell :-

Of course you have been able to install ssh clients for Windows for years or even decades, but having it available by default is a big win. Particularly for Windows machines you don’t tweak with your favourite applications.

#2: Public/Private Key Authentication

This the first part of increasing security by only permitting key authentication so that password brute forcing attacks become impossible. With the assistance of an ssh agent (not covered here) or a passphrase-less key pair (not advisable), it is no longer necessary to enter a password.

Of course getting into this sort of thing can be very confusing especially as most instructions tend to get into far too much detail on the cryptography involved. To keep it simple, I shall avoid going on about the cryptography, and concentrate on how to get it to work.

The most important thing to remember about key authentication is that there are two keys – the private key (which should be kept as secure as possible on the client machine) and the public key (which is copied to the devices you want to connect to).

So to get started, you first need to generate a key pair, which can be done with ssh-keygen; this has lots of options, but at this point you can ignore them. After you enter the command, you can simply hit return at all the prompts to generate a key pair :-

Generating public/private rsa key pair.
Enter file in which to save the key (/home/mike/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/mike/.ssh/id_rsa.
Your public key has been saved in /home/mike/.ssh/
The key fingerprint is:
SHA256:REMOVED mike@Michelin
The key's randomart image is:
+---[RSA 2048]----+
|=*+o ..  .B*..=o |
|o+++.  . =+o. o+.|
|.BE.+   + .  =  .|
|o+=& . . .    o  |
|. o +   S    .   |
|     .           |
|    SS           |
|              .  |
|     --          |

Of course this is not ideal because there is no passphrase, but to get started with that’s fine. You can ignore most of this output (except for the first item in the following list) but just in case :-

  1. The key pair is saved in the files ~/.ssh/id_rsa (the private key) and ~/.ssh/ (the public key). The permissions are usually generated properly, but just to be safe you may want to reset the permissions anyway: chmod 0400 ~/.ssh ~/.ssh/id_rsa; chmod u+x ~/.ssh
  2. The key fingerprint can be used to check that when you are connecting that the keys haven’t changed unexpectedly.
  3. Alternatively (and slightly more of a reasonable check) you can check the fingerprint using the “randoart”.

Of course on its own, it doesn’t do much good. You have to copy it into place onto the machine you wish to authenticate to :-

$ ssh username@server mkdir .ssh
$ cat ~/.ssh/ | ssh username@server cat ">>" .ssh/authorized_keys

Note the quotes around the “>>”; these are significant because you do not want the local machine’s shell to interpret them – they need to be interpreted by the remote machine’s shell. Normally I would simply “scp” the file into place, but appending to a supposedly non-existent file is safer – just in case it does exist and does contain public keys that are currently in use.

There are a whole bunch of options to the command, but the two most important ones are :-

  1. The -t option which is used to specify the key type to generate (dsa, rsa, ecdsa, and ed25519). This is mostly unnecessary, but some older and limited devices do not understand certain key types. And as time goes on, more key types will be declared “insecure”. So you may sometimes find the need to generate more secure keys. The simplest (but not very efficient) process for dealing with such situations is to generate a key for each key type and try each one in turn.
  2. The -f option which is used to specify the output filenames – the private key is saved under the name ‘filename’ and the public key under the name ‘’.

#3: SSH Configuration File and Usernames

There are a ton of things that can be done with the ssh configuration file, but for this section I’ll stick with setting the username used to login to specific hosts – not because this is the most interesting thing that can be done, although it is quite useful.

The configuration file can be found (if it has been created) at ~/.ssh/config (with a system-wide version at /etc/ssh/ssh_config). Within that file, you can set global preferences, or host specific preferences :-

Username fred

Host router
  Username admin
Host dns*
  Username fxb
Host ds-* web-*
  Username baileyf
Host *
  Username fred

The first line (Username fred) instructs ssh to use the username ‘fred’ when no username is specified – ssh effectively becomes ssh fred@

If you specify the same username within a Host section, the specified username is used for any hosts that the specification following the Host word. In the first case (“Host router”) the username “admin” will be used for any host called “router” but not “router.some.domain”.

In the case of the second clause, a wildcard is used which is very useful for specifying a range of hosts – the example can match “dns01”, “dns01.some.domain”, or even “dns02”. In fact the first Host section is an example of what you should not do – put in a single hostname without a wildcard because it will only activate if the hostname is specified exactly as given. Put a wildcard in there, and it will work whether you use a single hostname or use the fully qualified domain name.

You can also have more than one host specification – as in the “ds-* web-*” list.

And lastly you can (if you choose) use the Host declaration to specify a set of default values – in much the same way that configuration settings in the global context specify default values. Use whatever method you choose.

#4: Cryptographic Incompatibility

I have commented elsewhere on this, but basically the ssh developers have chosen to disable weak encryption by default. Personally I would prefer that ssh throw up huge warnings about weak cryptography, but what is done is done.

If you need to connect to something with weak cryptography, there are three potential ‘fixes’ to allow connections. Each of these is a keyword to add to a specific host section, followed by a specification of what ‘algorithm’ to add.

In each case, a connection attempt will give an indication of what is wrong together with an indication of what algorithm to include :-

» ssh admin@${someswitch}
Unable to negotiate with ${ip} port 22: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1

In this case, we can see that it is the KexAlgorithms we need to adjust and the algorithm we need to add is “diffie-hellman-group1-sha1” :-

Host someswitch*
  KexAlgorithms +diffie-hellman-group1-sha1

This can be repeated for Ciphers and (rarely) MACs.

#5: X11 and Port Forwarding

Run X11 gooey programs over an ssh connection? Of course .. why not?

This can be enabled on a host-by-host basis (it is off by default because it can be insecure) using the configuration file :-

Host pica*
  ForwardX11 yes

This is just a special case of port forwarding where a network port is connected (via the ssh session) to a remote network port. Port forwarding can be very useful – for example to access an internal web site temporarily that isn’t (and probably shouldn’t be) exposed with a hole through the firewall.

Of course this can be done with a VPN, but ssh may be simpler :-

Host pica*
  LocalForward 8000 8000

When the connection is made, a local port is opened (tcp/8000) and connected to tcp/8000 on the machine you are logging into.

Jun 052018

As the subject says, this blog has been offline for just over a week because of a hardware failure. Just when I wanted to moan about all the GDPR hissy fits that people are throwing.

Noticed some websites are blocking you because of the GDPR?

That’s the hissy fit. Seems that some international web site operators who previously assumed that GDPR didn’t apply to them, are suddenly realising that it does. Which is an indication that they have been impersonating an ostrich for a couple of years now.

Smaller businesses get a free pass on that one, but any reasonably sized company should have been aware of GDPR by now. It was put in place and deliberately put on hold for two years to allow people to get started with complying with GDPR. Anyone involved in the security business has been hearing “GDPR” for over two years now.

So there are those who claim they’ve not heard of it, and are now panicking and trying to catch up, making a mountain out of a molehill, and claiming that it’s a dumb law. Technically it isn’t actually a law but an EU regulation that member states are required to make law.

Anyway onto some of the biggest arguments against the GDPR …

The Whois Question

This is a great example of what happens when you ignore a situation and then panic.

When you register a domain (such as or a netblock (a set of IP addresses), you are expected to provide contact details for the individual(s) involved in the registration process – to allow for billing, and contact to be made in the event of operational issues.

Storing that information is perfectly reasonable.

Publishing that information is perfectly reasonable given informed consent.

Ideally the domain registration would offer a choice to the registrant – public listing of personal details, public listing of role contact information, or public listing of indirect contacts (i.e. keeping the contact details private).

There is a German court case decision saying that it isn’t necessary to have contact information for registering a domain; all I can say is that the German court obviously didn’t have the full facts.

GDPR’s “Right To Be Forgotten”

One of the misconceptions is that the “right to be forgotten” is an absolute human right; for a start it’s not a a human right, but a right under the law. And it is not absolute; the text of the GDPR includes numerous exceptions to the right to be forgotten, such as :-

  • A legal or regulatory obligation to keep the personal information.
  • An overriding public interest.
  • Ongoing legitimate business processes still require that personal information.

The key is that if you are an ethical business (in particular don’t plan to sell personal information and/or keep spamming people) then the right to be forgotten isn’t anything to worry about.

GDPR: The Fines

The strange thing is that there is doubt over the level of fines that can be levied under the GDPR which is remarkable as the language is quite clear – the lower level of breach can be fine of up to either €10 million or 2% of annual turnover.

Or to put it another way, for the lower level of breach, the maximum fine is whichever is greater €10 million or 2% of annual turnover. The maximum.

Do you know how often the ICO has imposed the maximum level of fine under existing legislation? Never.

The Jurisdiction Issue

Now here there is some legitimate grounds for grievance; after all whenever the US starts imposing its laws outside of the US, people outside the US start jumping up and down. And yes, the EU does expect non-EU companies to obey the GDPR regulation if they store data on EU citizens.

In practice, the EU isn’t going to try going after small companies outside the EU; particularly not small companies that are just ordinary business and not engaged in Cambridge Analytica type business.

The other way of looking at the global reach of the GDPR is whether it would be a good idea for there to be a world-wide law in relation to the protection of personal information. The Internet means that world-wide laws are necessary in this area, or those abusing personal information will merely move to the jurisdiction with the weakest protection of personal information.

Rusty Handrail

May 042018

I had the pleasure of upgrading a server today which involved fixing a number of little niggles; one of which was that connecting to switches suddenly stopped working :-

✗ msm@${server}» ssh admin@${someswitch}
Unable to negotiate with ${ip} port 22: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1

This was relatively easily fixed :-

✗ msm@${server}» ssh -o KexAlgorithms=+diffie-hellman-group1-sha1 admin@${someswitch}

Of course doing this command-by-command is a little tedious, so a more permanent solution is to re-enable all the supported key exchange algorithms. The relevant algorithms can be listed with ssh -Q kex, and they can be listed in the server-wide client configuration in /etc/ssh/ssh_config :-

Host *
    KexAlgorithms ${comma-separated-list}

But Why?

According the OpenSSH developers, the latest version of ssh are refusing to use certain key exchange algorithms (and other cryptographic ‘functions’).

Their intention is perfectly reasonable – by default the software refuses to use known weak crypto. I’m fully behind the idea of discouraging the use of weak crypto.

But the effect of disabling weak crypto in the client is unfortunate – all of a sudden people are unable to connect to certain devices. The developers suggest that the best way of fixing the problem is to upgrade the server so that it supports strong cryptography.

I fully agree, but there are problems with that :-

  1. Some of the devices may very well be unsupported with no means to upgrade the ssh dæmon. Now in an ideal world, these devices wouldn’t be on the network, but in the real world there are such devices on the network.
  2. Some devices may not be capable of being upgraded because of processor or memory limitations. Network switches are notorious for having slow processors and tiny amounts of memory, and it is entirely possible that such a device would not be capable of running more exotic and modern crypto. Similarly lights out management processors are often severely limited.
  3. Even if a device is capable of being upgraded, there are the standard problems – the vendor may be slow at releasing updates, change control gets in the way, and lastly resourcing may be an issue – upgrading several hundred switches manually with just one or two people doing it is not going to be a quick job.

Lastly, whilst security is important, breaking things just to make a point is a little extreme. Whilst it is possible to fix the problem, it is something that isn’t immediately obvious to someone who doesn’t routinely configure ssh. And someone, somewhere has had this breakage occur just before they really need to fiddle with a switch Right Now.

There is a far better option available – leave the weak crypto enabled, but warn noisily about its use :-

WARNING!!!!! (2 second delay)
WARNING!!!!! (2 second delay)

The device you are connecting to only supports known weak crypto which means this connection
is subject to interception by an attacker.

You should look at upgrading the device as soon as possible.

Telling people what is wrong noisily and continuing to work is far better than simply breaking with a rather terse message.

Foggy Reflection


Apr 012018

This is a continuation of an earlier post regarding ECC memory under Linux, and is how I added a little widget to display the current ECC memory status. Because I don’t really know lua, most of the work is carried out with a shell script that is run via cron on a frequent basis.

The shell script simply runs edac-util to obtain the number of correctable errors and uncorrectable errors, and formats the numbers in a way suitable for setting the text of a widget :-

# Use edac-util to report some numbers to display ...

correctables=$(edac-util --report=ce | awk '{print $NF}')
uncorrectables=$(edac-util --report=ue | awk '{print $NF}')

if [[ "$correctables" != "0" ]]
if [[ "$uncorrectables" != "0" ]]

echo "ECC: $correctables/$uncorrectables "

This is run with a crontab entry :-

*/7 * * * * /site/scripts/gen-ecc-wtext > /home/mike/lib/awesome/widget-texts/ecc-status

Once the file is being generated, the Awesome configuration can take effect :-

-- The following function does what it says and is used in a number of dumb widgets
-- to gather strings from shell scripts
function readfiletostring (filename)
  file =, "r")
  s =
  return s

eccstatus = wibox.widget.textbox()
eccstatus:set_markup(readfiletostring(homedir .. "/lib/awesome/widget-texts/ecc-status"))
eccstatustimer = timer({ timeout = 60 })
      eccstatus:set_markup(readfiletostring(homedir .. "/lib/awesome/widget-texts/ecc-status"))
layout = wibox.layout.fixed.horizontal, ... eccstatus, ...

There plenty of ways this could be improved – there’s nothing really that requires a separate shell script, but this works which is good enough for now.

Mar 292018

For some reason when I look at RADIUS packet captures using Wireshark, the attribute Operator_Name is instead interpreted as Multi-Link-Flag (an integer rather than a string). I’m not sure what this is, but it is much more useful to me to be able to see the Operator_Name properly – and for example, filter on it.

It turns out this is easy to “fix” (if it is a fix) :-

  1. Find the file radius/dictionary.usr (mine was /usr/share/wireshark/radius/dictionary.usr)
  2. Edit that file, and comment out three lines containing “Multi-Link-Flag” which in my case appeared like :-
    1. ATTRIBUTE Multi-Link-Flag 126 integer
    2. VALUE Multi-Link-Flag True 1
    3. VALUE Multi-Link-Flag False 0
  3. Save the modified file.

After a restart, Wireshark now understands it.

It is possible that later versions of Wireshark have fixed this, or not – it is possible that the bug is down to whoever assigned RADIUS attribute codes!

Mar 252018

It seems likely that the company Cambridge Analytica paid Facebook for access to data and using it’s access, downloaded as much data as possible for nefarious purposes. Nobody should be that surprised at this.

Facebook does not host an enormously expensive social network just because it is fun; it does it to make money. It probably does this primarily through advertising, but selling access to social network data is always going to take place.

And from time to time, scandals when companies like Cambridge Analytica are going to take place. At which point Facebook will protest saying that it didn’t realise that the associated firm was doing such naughty things. And once the story drops out of the news, Facebook will carry on leaking data.

As the saying goes: “If you are not paying for it, you are the product.”

In the end, the only solution to something like this, is to produce some kind of peer-to-peer application that is as easy to use as Facebook, uses strong end-to-end encryption, and keeps our data private to those people and groups we choose to share it with.

The Hole

Mar 092018

One of the things that annoys me about pagers such as lessmore, most, etc. is that they are dumb in the sense that they cannot detect the format of the text file they are displaying. For example, all of a sudden I find myself reading lots of markdown-formatted files, and I find myself using most to display it – never remembering that it is mdv I want.

As it happens, when I invoke a pager at the shell prompt, I typically use an alias (page or pg) to invoke a preferred pager, and by extending this functionality into a function I can start to approach what I want :-

function extension {
  printf "%s\n" ${argv/*\./}

function page {
 if [[ -z $argv ]]
   case $(extension $argv) in
       mdv -A $argv | $PAGER
       groff -m mandoc -Tutf8 $argv | $PAGER
       $PAGER $argv

Of course there are undoubtedly umpteen errors in that, and probably better ways to do it too. And it won’t work properly on its own ($PAGER hasn’t been set).
But it’s the start of something I can use to display all sorts of text files in a terminal window without having to remember all those commands. But as for ‘intelligent’, nope it’s not that – just a bit smarter than the average pager.

Feb 082018

Some time ago, I wrote about using new (for the time) partition tables to create a memory stick with 100 partitions; each with a mountable file system on. And decided the time was right to have another look to see if things have improved … or degraded. After all, things have moved on, and everything has been updated.

I also improved the creation script slightly :-



parted $disk mklabel gpt
for x in {1..99}     
  echo Partition: $x
  parted -s $disk mkpart FAT $(($x * 100)) $((x * 100 + 99))
  sleep 0.2
  mkfs -t vfat -n DOOM${x} ${disk}${x} 
  sleep 0.2

And I used a zsh-ism – so shoot me.

The script ran fairly well, but :-

  1. The load average shot up through the roof as copies of systemd-udevd started, worked, and closed.
  2. Strangely the links in /dev/disk/by-label (and presumably elsewhere) kept disappearing and re-appearing. As if on each partition change to the disk, all of the disk’s devices were removed and re-created. This is probably not dangerous, but harmful to performance.
  3. Given that I used sleep within my script, it is hard to criticise performance, but it did seem slow. However this is not an area worth optimising for.
  4. Unlike last time, Linux did not refuse to create any file systems.

Now onto trying to stick the memory stick of doom into various systems…

Ubuntu 17.10

This was of course the machine I ran the script on initially.

This did not go so well, with the machine initially freezing momentarily (although it is a cheap and nasty laptop), apparently silently refusing to mount half the file systems, and “Files” (or Nautilus) getting wedged at 100% processor usage.

After some 10 minutes, Nautilus was still stuck with no signs of making any progress.

After I lost patience and restarted “Files”, it came up okay showing the mounted file systems and showing the file systems it had failed to mount. On one occasion the additional file systems were shown as unmounted (and could be mounted) and on another they were shown as mounted (even though they weren’t).

So both “Files” gets a thumb down for getting stuck, and whatever else gets a thumb down for trying and failing (silently) to mount all the file systems.

This is definitely a serious degradation from the previous try, although probably GNOME-specific rather than Linux-specific. Especially as a later mounted all the file systems from the command-line on a different system without an issue.

Windows 10

Windows 10 became unusually sluggish, although it may have been in the mysterious “we’ll run Windows update at the most inconvenient time possible” mode. It did attempt to mount the file systems, and failed miserably – it mounted the first set until it ran out of drive letters.

Which is just about understandable, as there aren’t 100 drive letters. However :-

  1. Where was the message saying “There are 100 partitions in this silly USB stick. You can see the first 22; additional ones can be mounted within folders if there is important data on them.”.
  2. Why is Windows still limiting itself with single letter device names? Okay it is what we’re used to, but when you run out of drive letters, start using the file system label – “DOOM99:”. Hell, I’d like all my removable disks treated that way under Windows.

As for the whole “ran out of drive letters, so don’t bother with the rest”, how many people are aware that drives can be mounted (as Unix does) in directories?

macOS 10.13 (OSX)

Oddly enough (but perhaps sensibly), macOS refused to have anything to do with the memory stick. Indeed it popped up a dialog suggesting initialising the disk, which is perhaps not particularly sensible with a disk that could contain data.

The “Disk Utility” happily showed the disk – increasing the size of the window inconveniently wide in the process – and happily indicated 99 partitions.

At the Terminal prompt, it was apparent that the operating system had created device files for each of the partitions, but for some reason wouldn’t mount them.


Inserting a “stick of doom” with 100 partitions on it into any machine is still a risky thing to do. It’s also a dumb thing to do, but something operating system developers should be doing.

Linux (or rather GNOME) performs significant worse this time around than previously, and my suspicions are that systemd is to blame.

But however bad Linux does, none of the operating systems actually do sensible things with the “stick of doom”. macOS arguably comes closest with refusing to have anything to do with the disk, but it also encourages you to reformat the disk without saying that it could be erasing data.

Ideally, a gooey would pop up a window listing the file system labels and ask you which you want to mount. That’s not even a bad idea for a more sensibly set up memory stick.

Pebble On Steel

Feb 022018

On occasions, I have run into issues where mounting a filesystem from /etc/fstab fails on a reboot because it depends on something else happening first. The easiest example to recall is when mounting a conventional filesystem constructed from a ZPool block device – the block device isn’t ready until ZFS has finished starting which often occurs after the filesystem mounts are attempted.

The fix is dead simple; just add the option “_netdev” to the options field in /etc/fstab and the problem is sorted :-

/dev/zvol/pool1/vol-splunk      /opt/splunk     ext2    noatime,_netdev         0 2

Yes the reason I am using a block device is that Splunk doesn’t support being installed on a ZFS filesystem.

Jan 292018

I recently dived into the rabbit hole of educational computers and came across a site which made a big song and dance about how Python is a great deal more complicated than BASIC. Well that is perhaps arguably correct, but the comparison they made was grossly unfair :-

#!/usr/bin/env python
#-*- coding: UTF-8 -*-

from random import randint
from time import sleep
import sys

string = "Hello World!"
while true:
  attr = str(randint(30,48))
  out = "\x1b[%sm%s\x1b[0m" % (attr, string)


Now for the criticisms :-

  1. The first line (“#!/usr/bin/env …”) is nothing to do with Python; and in fact a BASIC program should also include this if it wants to run in the same way as a Python program under Linux. The “#!” is in fact a directive to the Linux kernel to tell it what script to pass the rest of the file through.
  2. The second line (“# -*-…”) also has nothing to do with Python; it is a directive to an editor to tell it to use the UTF-8 character set. Why doesn’t the basic equivalent also include this?
  3. Now onto the Python itself … first of all there are a whole bunch of imports which are done in the verbose way just so that you can call sleep rather than time.sleep; I generally prefer the later (which would result in the import time rather than from time import sleep). But yes, in Python you have to import lots of stuff to get anything done, and it would be helpful for quick and dirty scripts if you could just import lots to get a fair amount of ordinary stuff loaded.
  4. The rest of the code is … um … obviously designed to make Python look bad and glossing over the fact that Python runs in the Linux runtime environment whereas the BASIC equivalent does not – it has a BASIC runtime environment.

That last point is worth going into more detail on – the BASIC code was written for a BASIC runtime environment, and one method of sending output to the screen. Linux has many ways of writing to the screen, and the chosen method above is perhaps historically the worst (it only works for devices that understand the escape sequences; there is a curses library for doing this properly).

So is Python unsuited to a quick and easy learning environment? A quick hackers language? As it is, perhaps not, but that is not quite what Python is designed to be. And with a suitable set of modules, Python could be suitable :-

import lots

white True:
  screen.print("Hello World!")


(That’s entirely hypothetical of course as there is no “screen” module)

I’m not qualified to judge whether BASIC or Python are better languages for beginners – I’ve been programming for around 35 years, and the BASIC I remember was very primitive. But at least when you compare the two languages, make the comparison a fair one.

WP Facebook Auto Publish Powered By :

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.