No ads? Contribute with BitCoins: 16hQid2ddoCwHDWN9NdSnARAfdXc2Shnoa
Dec 082018

I recently bought a second-hand camera – but this is not specific to photography (but perhaps particularly relevant). The seller threw in an old SD card which was nice of them (although unnecessary for me).

After doing the photo thing with the new-to-me camera, and having carefully replaced the SD card, it occurred to me that I could test a file recovery tool to see if there was any previously shot photos on the card.

Using photorec, I fired it off and came back 30m later – not because it’s particularly slow but I have spent far too much time watching the equivalent of a progress bar, and I would rather get on and do something useful.

By the time I came back, it had recovered in excess of 1,000 images and videos. It turns out to be probably the most boring collection of photos you can imagine – an ordinary collection of family (not your own) photos would be interesting in comparison.

I won’t be including any of those recovered photos here because that would be unprofessional and potentially embarrassing to the camera seller (although they would most likely never find out). 

But you can easily imagine how such a recovery could be potentially embarrassing; even distressing. We usually choose whether a photo should be made public or not.

So how do you protect such things from happening? Is it sufficient to format a card in camera?

No it isn’t. Tools such as photorec are designed to recover images from cards where the images have been deleted or when the card has been formatted. Surprisingly enough, formatting a card does not overwrite all of the data blocks on a storage device; it merely replaces the data structures that allows an operating system to find files with a new blank structure.

So what are the solutions to keep your private photos to yourself?

It should be emphasised that this is advice intended to protect you from personal embarrassment; if there are legal or risk to life issues involved, seek professional advice.

The first rather obvious solution is to never give away or sell old cards; if you want to dispose of the cards, destroy them. It is not as if you could recover much by selling them – who wants a 5-year old 512Mbyte SD card?

If you do want to let others use your old cards, then use a special utility to destroy the contents completely; optionally (but nice for the recipient) is to then format the cards afterwards.

If you are using Windows (or macOS although the following Linux recipe can be adapted), then you will need a tool such as SafeWiper. There are those who claim that Windows format can do the job, but I wouldn’t trust it – the “quick format” option is the default which definitely doesn’t erase the data from the disk, and I have not personally checked that a “slow format” really removes the data beyond recovery with normal tools.

Whatever method you choose, check, double-check, and triple-check that the device you are erasing really

The first step under Linux is to identify the block device path to erase. You may well find that your SD card is automatically mounted when you plug it in. So running df from the command-line will give you a device path (/dev/sdb

But to double check, run lsblk

✓ mike@Michelin» lsblk -o NAME,FSTYPE,MOUNTPOINT,VENDOR,MODEL,SIZE | grep -v loop 
NAME                    FSTYPE      MOUNTPOINT                      VENDOR   MODEL              SIZE
sda                                                                 ATA      SAMSUNG MZNTY128 119.2G
├─sda1                  vfat        /boot/efi                                                   512M
├─sda2                  ext4        /boot                                                       732M
└─sda3                  crypto_LUKS                                                             118G
  └─sda3_crypt          LVM2_member                                                             118G
    ├─ubuntu--vg-root   ext4        /                                                         114.1G
    └─ubuntu--vg-swap_1 swap        [SWAP]                                                      3.9G
sdb                                                                 Generic  USB  SD Reader     3.8G
└─sdb1                  vfat        /media/mike/disk                                            3.8G

Note that how we have “USB SD Reader” alongside /dev/sdb and that it’s size is just 4Gbytes. So we have three confirmations that this is the device we want to erase.

To erase it, first we unmount it, run a hdparm command to erase it, and erase it a second time :-

✓ mike@Michelin» umount /dev/sdb1
✓ mike@Michelin» sudo hdparm --security-erase NULL /dev/sdb
security_password: ""

 Issuing SECURITY_ERASE command, password="", user=user
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
✓ mike@Michelin» sudo dd if=/dev/zero of=/dev/sdb bs=64M

Whilst we’re waiting for the “dd” command to finish writing zeros all over the SD card, why are we erasing this twice? 

We’re using hdparm

And I then suggest using the old slow method of “dd” as well because there is nothing wrong with being cautious in this area.

Misty Trees
Nov 112018

If you use the Unix or Linux command-line, you may very well wonder about the origins of some of the “special” characters. One of those is tilde (~) which is expanded by the shell into “home” :-

✓ mike@Michelin» echo $HOME                        
✓ mike@Michelin» echo ~
✓ mike@Michelin» echo ~root

This doesn’t of course work in general; just in the shell.

But where did this usage originate?

As it turns out, it was the markings on the keyboard of the ADM3A terminal :-

If you used Unix in the late 1970s/1980s, you may very well have used the ADM3A terminal and it seems that those who added the tilde feature to the Unix shell were amongst the users.

Oct 092018

And yes that includes me

For those who don’t know, RFC1918 is the Internet standard that allocates the addresses used for private networks – 10/8, 192.168/16, or 172.16/12. 

And in reference, specifically :-

To minimize the risk it is strongly recommended that an organization using private IP addresses choose randomly from the reserved pool of private addresses

(Apologies for the incorrect spelling above; it’s a quote from an American source)

This was reinforced to me yesterday whilst I was working from home, and I had trouble with a site-to-site VPN joining my network to work’s. As it happens there was no addressing conflict, but I had to install many routes more than should be necessary.

And I keep seeing this sort of thing; joining multiple networks when everybody is using 10/8 is a continual game of chicken – when are we going to find ourselves in conflict? 

Of course there is a “fix” for this – NAT. The real fix of course is to use global IPv6 addresses even for devices and networks that will never be on the global Internet.

The Window
Oct 032018

I have a Python script that over-simplifying, reads very large log files and runs a whole bunch of regular expressions on each line. As it had started running inconveniently slowly, I had a look at improving the performance.

The conventional wisdom is that if you are reading a file (or standard input), then the simplest method is probably almost always the fastest :-

for line in logstream:

But being stubborn, I looked at possible improvements and came up with :-

from itertools import islice
while True:
    buffer = list(islice(logstream, islicecount))
    if buffer != []:
        for line in buffer:

This code has been updated twice because the first version added a splat to the output and the second version (which was far more elegant) didn’t work. The final version 

This I benchmarked as being nearly 5% quicker – not bad, but nowhere near enough for my purposes.

The next step was to improve the regular expressions – I read somewhere that .* can be expensive and that [^\s]* was far quicker and often gave the same result. I replaced a number of .* occurrences in the “patterns” file and re-ran the benchmark to find (in a case with lots of regular expressions) the time had dropped nearly 25%.

The last step was to install nuitka to compile the Python script into a binary executable. This showed a further 25% drop – a script that started the day taking 15 minutes to run through one particular run ended the day taking just under 8 minutes.

The funny thing is that the optimisation that took the longest and had the biggest effect on the code showed the smallest improvement!

Four Posts
Sep 102018

If you have not heard, Steam have added a compatibility layer to Steam which allows a limited number of Windows games to run. The “compatibility layer” is in fact a fork of WINE called Proton.

Peered at from 500 metres away, Proton allows Windows software to run (or not infrequently crash and burn) by translating the Win32 API into Linux APIs, and translating the variety of graphics APIs into Vulkan. That is a really difficult thing to do.

I have taken a very quick look at the new Steam client (and “Proton” is no longer part of a beta release of the Steam client – it’s in the standard client). It works perfectly adequately, although you will have variable experiences running Windows software.

For some reason this news has captured the imagination of a number of ‘tubers who are more gamers than Linux users, which has lead to some misunderstanding :-

  1. This is not Linux gaming; it is Windows gaming under Linux. If you have a bad experience with Steam under Linux, you are not experiencing a bad time with Linux gaming. Linux gaming involves native Linux software, and yes there is some out there.
  2. Problems with Steam could well be down to the Proton compatibility layer with unsupported API calls or buggy usage of the Win32 API which relies on Windows behaving in a certain way for undefined parameters.
  3. In addition problems with Steam could be due to the hardware you are running; take a game that works perfectly fine with an Nvidia card. It may behave problematically with an AMD card or even a different Nvidia card. Or the other way around.

The important thing to remember when looking at videos about Steam is that the person looking at Steam may not be the most experienced Linux user out there. That is not necessarily bad – the whole purpose of Steam is to be able to run games easily without a whole lot of Linux experience.

But they may not be understanding properly what is going on – for example the first thing I would do as a professional game-orientated ‘tuber would be to try out a selection of games with an nvidia card, and then repeat using an AMD card – just to see if things work better, worse, or at least differently.

And again, this is not about Linux gaming but about allowing easy access to old Windows titles that someone may have bought in the past. 

Pentland Hills
Sep 082018

Having used Linux for well over 20 years (yes it is that old), and Unix before that, I’m often puzzled by how scary some people seem to find Linux. Why should it be scary? It’s just a computer – you’re the human in charge of it.

Yes There Are Gooeys

(graphical user interfaces – GUIs – gooeys)

Yes there is plenty of software with a graphical user interface – I use plenty on a daily basis including a standard web browser, an email client, a password manager, an office package.

On a slightly less frequent basis there are many more that I use. Indeed providing that you accept the use of alternatives, you can find Linux software to do just about anything.

But Don’t Ignore The Command-Line

Yes, Linux has a command-line, and for those of us familiar with it, it can be very powerful. And there is no harm in learning how to use the command-line just to the point where you can follow instructions on how to “get something done” there.

Because if I have a fix for some niggle that you are having, it is easier and less error-prone to pass instructions for a command-line incantation than instructions for a gooey (and yes I have done both).

The Birds

Sep 062018

I recently put together a new PC (or mostly) and had occasion to look at what PC cases are like these days. In the end I kept my existing case, but did spend enough time looking to have certain opinions.

And they suck.

All about the glass windows to let the silly lights show through, but how about useful features?

  • Tool-less case panels? Or at least the top panel (to access the expansion cards).
  • Built-in cable runs so things (fans, SATA drives, etc) can be plugged in next to where they are installed.
  • On the subject of fans, servers often have easily removable fan trays; fans are mounted to a plastic frame which in turn slots into position together with power and control signals. A doddle to clean, which would be handy for a desktop workstation.
  • A front panel display to show fault messages during startup – firmware fault codes (some motherboards have a two-digit display but they’re optional and usually not visible when the case is closed). Post-boot it could be used for other things. If it breaks the clean lines of the case, put it behind a sliding panel or something.
  • Handles. And wheels. 
Signs Of The Sea

There are probably a whole bunch more that could usefully be considered, and some of these are inherited from cases known to me (the old Mac Pro case is a good place to start from).

Aug 092018

Well that was a weird error; I recently discovered that ntpd had mysteriously stopped working; specifically it was not able to resolve NTP “pool” names :-

ntpd: error resolving pool Name or service not known (-2)

After some time spent blundering around down dead ends with the help of an appropriate search engine, I ended up resorting to strace. This is a tool most commonly used by developers but can be surprisingly useful for diagnosing system problems too.

As long as you can look past all the inscrutable output!

The strace tool runs a command and records every system call that the command calls together with the results. And of course most commands make zillions of system calls so you’re likely to end up with a huge output file.

To generate the output file, I ran the modern equivalent of ntpdate (ntpd -d) which tries to do the same thing using the actual NTP daemon. Usefully in this case because the command starts, configures itself (which is where the error occurs), and then exits (unlike the normal dæmon). It is important to redirect the output to have a file to trawl through later :-

strace ntpd -d > /var/tmp/ntpd.strace 2>&1

Once the output was generated, it was necessary to trawl through it to look for clues. The first thing was to search for “europe” (as I use as one of my NTP servers). The first occurrence was the error claiming that the name didn’t exist :-

write(2, "error resolving pool europe.pool"..., 73error resolving pool Name or service not known (-2)

Which was somewhat odd because you would expect the string “europe” to occur within an instructable attempt resolve the name. Yet it appears as though the error occurs without any attempt to resolve the name!

As a bit of a guess I searched for “resolv.conf” which revealed :-

stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=362, ...}) = 0
openat(AT_FDCWD, "/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied)

Apparently ntpd is unable to open the file due to a permissions problem!

Looking at my /etc/resolv.conf revealed an oddity dating back to when I tried configuring /etc/resolv.conf as a symbolic link to a file on a separate file system. The file itself was a symbolic link to /etc/resolv.conf.file.

For some reason ntpd didn’t like the symbolic link, which is a bit odd but changing it to an ordinary file fixed the problem.

Jul 302018

Alternatively, why does Windows use drive letters? Because if you are coming from an old unix background, drive letters are just as weird as the lack of them if you are coming from a Windows background.

I mean, why is Windows installed on drive C? What ever happened to drives A and B?

Technically Linux does have the equivalent of drive letters but they are rarely used directly (unless you’re weird like I am). For example I currently have an SD card plugged into my desktop system, and it has the path /dev/disk/by-label/EOS_DIGITAL (or /dev/sdo1).

Historically, Unix (which is loosely the predecessor of Linux) ran on large minicomputers where system administrators would decide what disks were “mounted” where.  The Linux equivalent of drive C is effectively “/” (root), and you can attach (or “mount”) disks at any point underneath that – for example /home.

This allowed people to use an old Unix machine without worrying where this disks were; and allowed system administrators to add and remove disks as and where they were needed. These days we are all system administrators as well as users – that little voice you hear from time to time saying things like “When would be a good time to update the operating system?” and “I must clean up those temporary files all over the place” are your inner system administrator speaking up.

And if you don’t hear that inner voice, cultivate it!

With device paths, Linux has the opportunity to create sensible friendly names for disks, but a historical accident has resulted in almost every kind of disk being identified as a SCSI disk – SATA disks (a normal hard disk), SAS disks (server hard disks), Fiber Channel disks (SAN hard disks), and even USB storage devices all use SCSI commands.

So nearly all Linux disks are identified as /dev/sd followed by a letter (a “drive letter” – we can’t get away from them) and a number indicating the partition. Fortunately there is also the relatively new /dev/disks directory that has slightly friendlier names for disk devices. If you are getting into low-level disk management, learn these directories; in particular if you are looking into enterprise disk management look at WWNs (each disk has a unique “world-wide-number”).

Now back to Windows. Windows is the descendent of DOS, which goes back to the time when PCs may not have had hard disks and by default would have booted off a floppy disk in drive A with a data disk in drive B. Later PCs came with hard disks which used drive C on the assumption that you would have one or two floppy drives.

Windows has been updated over the years and there is a great deal of sophistication under the surface, but it does act a bit conservatively when it comes to drive letters – A and B are by default reserved for floppy drives even though I haven’t seen one of those on an ordinary system for years. You can use A and B for other purposes such as mapping network drives – A makes a good drive for a NAS drive.

If we get away from the terminology of “drive letters” and “device paths” and instead refer to them as “storage device names”, both Linux and Windows have “storage device names” but Linux prefers to hide that level of detail.

Personally I prefer the Linux way, but whatever floats your boat.

Jul 242018

As someone who has spent far too much time dealing with the Domain Name System, I get kind of miffed when people insist on creating names that conflict with the DNS ordering. You see the DNS naming works from right-to-left (the wrong way around if you’re reading this in English).

Take the name for this site – – which is admittedly a rather quirky name. The most significant part of the name is at the right (org – and yes I’m ignoring the really significant and invisible “dot”). The next most significant part (zonky) specifies what organisation has registered the site (me), and the least significant part (really) points to one service at that organisation.

So when people ask for names that break that ordering it is ever so slightly irritating – for example if you have a service called and wanted a test service you might request which breaks the ordering of things. As an alternative, doesn’t break the naming, looks a bit nicer, and ultimately more reasonably flexible.

Let us look at a slightly more complex example; let’s assume that we have a domain called and want to register a service name for each database. We could register names such as, and, or we could register them instead as and In the later case, I can very quickly write a firewall rule that allows access to * (whereas db-* would not work).

Ultimately suggest names in DNS naming order unless you can justify why it is not suitable.


WP Facebook Auto Publish Powered By :

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.