Jun 052014
 

There are many different ways to listen to music, and many different ways to listen. The old way to record music is on huge discs containing an analogue recording of the music on both sides – what is now called the Long-Playing record (or LP). It’s called long-playing because it is an evolution of other disc-based formats that were capable of holding less music.

The not quite so old method of holding music is the Compact-Disc (or CD) which uses stores the music digitally rather than in analogue form.

The newest method is the “digital download” and is itself as complex as all the previous methods put together. Because the playback of digital music recordings is basically software, it is possible to introduce new “formats” every so often – so we have MP3, Ogg, FLAC, etc. Most of these formats compromise on music quality in favour of smaller file size so they can be downloaded quicker and you can fit more of them on the average portable music player.

However for the purposes of comparing with LPs and CDs, I will only consider high-resolution FLAC files which do not compromise on quality over file size. Specifically FLAC files with a better than CD quality, and yes they do exist.

When considering music playback, it’s worth remembering that good music always sounds better than bad music; an improved playback merely makes good music sound better. And there are diminishing returns.

Having said that, which of the three is better: LPs, CDs, or digital downloads? The answer is complex, and depends upon the what you want to get out of an album. It also depends on the manufacturing quality – a poorly pressed CD on inferior materials does not compare well with a good CD; and perhaps even more so for an LP.

The Physical Experience and Longevity

In terms of an object to be appreciated in addition to the music nothing beats the LP. Excluding the disc itself, the album art is large enough to be fully appreciated. This is for many an important consideration, but personally I buy recorded music for the music and not for the object d’art that it comes on.

In terms of longevity, nothing lasts forever, but a DRM-free digital download is perhaps somewhat surprisingly in the lead here. Providing that you have a good backup regime, there is no reason why a digital download cannot be passed on to your off-spring.

On the other hand, LPs are subject to various forms of damage that can occur by chance every time you play one – even the playback causes a very tiny amount of damage unless you use a laser-based record player. And every bit of damage (or dust) causes a deterioration in the quality of the playback.

CDs have certain advantages in this respect because it is digital and there is built-in error correction so smaller errors get corrected.

However neither physical medium will last forever, and a poorly treated LP is likely to deteriorate significantly in a relatively short amount of time.

Analog vs Digital

very strongly suspect that the preference that some people have for analog has nothing to do with the fact that it is analog. If you play back a quality LP through a high-end system, it may seem to sound slightly better than a CD. But I have listened to very high quality digital rips of LPs played back on a high quality digital music player, and have heard the same effect – even including the added “warmth” that LP playback supposedly gives you.

analog-digital

The graph above is a very exaggerated demonstration of the difference between a “perfect” analog signal (which frankly doesn’t exist) and a very rough and ready digital signal. The distance between the green lines is representative of the sample rate – how many times per second a digital value is measured. The vertical scale of the green lines is representative of the number of bits used to measure each sample – in this case it is far too few leading to a dramatic staircase effect.

The digital version can be improved by increasing the sample rate and increasing the number of bits for each sample. Increase it enough, and it is impossible to distinguish between the digital and analog versions.

But increase the sample rate to 192KHz and the resolution to 32-bits, and you are talking about some really rather large files. Which is why CDs are just 44KHz and 16-bits; at the time CDs were developed, it was felt to be a reasonable compromise between quality and the need to keep storage requirements down.

One thing that is often overlooked is that CDs contain a lot of error correcting data – 20% of the storage capacity is error correction data. This allows the CD player to correct any read errors that occur (up to a certain level). Nothing else has this level of error correction with the exception of a digital download if it is held on an error-correcting file system (such as ZFS).

This section implies that analog is “perfect”. In case anyone believes that, try reading up on wow, and flutter. And indeed the full details of analog vs. digital. Whilst analog in general is no worse than digital, the specific analog method used for LPs is inherently less detailed than the amount of detail available in CD quality audio and indeed high-resolution digital downloads.

The interesting thing about the so-called “warmth” that LPs provide is that this is effectively making a virtue of the weakness of LPs. The old terms for LP warmth are “muddy bass” and “rolled off high end”. Listening to the warmth of LPs may well be a better experience but that does not make LPs better for music playback.

Digital Compression

Judging by some of the snake oil around digital music, compression is not very well understood – for example. there is a fairly popular myth that the audio from FLAC can be improved by uncompressing into WAV format and then playing it back. There is possibly a very tiny advantage in doing that if you happen to have very poor quality digital equipment, but the advantage would be miniscule. That advantage would be simply down to the hardware/software being unable to uncompress and play back the audio stream at the correct speed all of the time.

There are essentially two different forms of digital compression – lossless and lossy. The former can be uncompressed to produce a digital stream exactly the same as the original, and the latter does not. Lossy compression is much smaller in size, so it is better suited to small portable music players, although there is no reason why small portable music players have to have such small capacities for storage (my current portable player has 256Gbytes of storage and my previous portable player had 240Gbytes of storage).

Because lossy compression loses some of the original data, it approximates the original data – ideally in a way that is indistinguishable from the original. However it is possible that extreme compression – as found in 128kbps MP3 files – will introduce artifacts that sound “wrong”.

Summary

When you come down to it, music is not about digital or analog, it is about the music. And to many quality is also a desirable aspect. This can be obtained with analog or digital, but to a great extent digital is more convenient.

If you are after quality, there is no need to switch to analog to get it – digital music can be just as filled with quality as analog providing that you do it properly.

But most importantly, enjoy the music!

May 192014
 

So there I was, contemplating whether I could produce a nice and simple bar chart showing the number of films I’ve watched per month. Using just MySQL.

I knew it was easy to produce a simple numerical count with count(*) and a group by clause, but a bar chart? Turns out it was easy with the repeat string function.

mysql> select date_format(whence, "%Y-%m") "Month", 
          repeat('*', count(*)) "Number of Films" 
          from films group by date_format(whence, "%Y-%m");
+---------+------------------------------------------------+
| Month   | Number of Films                                |
+---------+------------------------------------------------+
| 2007-05 | ****                                           |
| 2007-06 | ******                                         |
| 2007-07 | ******                                         |
| 2007-08 | *****                                          |
| 2007-09 | ******                                         |
| 2007-10 | ***                                            |
| 2007-12 | ****                                           |
| 2008-01 | *****                                          |
| 2008-02 | ****                                           |
| 2008-03 | *                                              |
| 2008-04 | *****                                          |
| 2008-05 | *****                                          |
| 2008-06 | ******                                         |
| 2008-07 | ****                                           |
| 2008-08 | *****                                          |
| 2008-09 | *****                                          |
| 2008-10 | ****                                           |
| 2008-11 | *****                                          |
| 2008-12 | ***                                            |
| 2009-01 | ***                                            |
| 2009-03 | **                                             |
| 2009-04 | ***                                            |
| 2009-05 | **                                             |
| 2009-06 | ****                                           |
| 2009-07 | ****                                           |
| 2009-08 | *                                              |
| 2009-09 | **                                             |
| 2009-10 | ****                                           |
| 2009-11 | **                                             |
| 2009-12 | **                                             |
| 2010-01 | **********                                     |
| 2010-02 | **********                                     |
| 2010-03 | *************************                      |
| 2010-04 | *****************************                  |
| 2010-05 | ********************************************** |
| 2010-06 | ***********************                        |
| 2010-07 | ****************                               |
| 2010-08 | **********                                     |
| 2010-09 | ************                                   |
| 2010-10 | **********                                     |
| 2010-11 | ********                                       |
| 2010-12 | *********                                      |
| 2011-01 | *******************                            |
| 2011-02 | *************                                  |
| 2011-03 | **                                             |
| 2011-04 | ************                                   |
| 2011-05 | ********                                       |
| 2011-06 | ***                                            |
| 2011-07 | ****                                           |
| 2011-08 | *********************                          |
| 2011-09 | *                                              |
| 2011-10 | **                                             |
| 2011-11 | ***************                                |
| 2011-12 | ********************                           |
| 2012-01 | ********************                           |
| 2012-02 | *******                                        |
| 2012-03 | *******                                        |
| 2012-04 | *****                                          |
| 2012-05 | ******                                         |
| 2012-06 | *******                                        |
| 2012-07 | **************                                 |
| 2012-08 | ************                                   |
| 2012-09 | ***************                                |
| 2012-10 | *******************                            |
| 2012-11 | ****************                               |
| 2012-12 | *******                                        |
| 2013-01 | *************                                  |
| 2013-02 | *************                                  |
| 2013-03 | ******************                             |
| 2013-04 | ******************                             |
| 2013-05 | ********                                       |
| 2013-06 | ************                                   |
| 2013-07 | **************                                 |
| 2013-08 | ***                                            |
| 2013-09 | *************                                  |
| 2013-10 | ********                                       |
| 2013-11 | ***************                                |
| 2013-12 | ***********************                        |
| 2014-01 | *************************                      |
| 2014-02 | ********                                       |
| 2014-03 | *************                                  |
| 2014-04 | ****************                               |
| 2014-05 | ************                                   |
+---------+------------------------------------------------+
83 rows in set (0.02 sec)
Apr 092014
 

The interwebs are all a flutter over the latest vulnerability announcement – an OpenSSL vulnerability that has been termed the heartbleed vulnerability. But is it that serious? And what is it anyway?

What Is It?

OpenSSL is a very widely used software component that adds encryption – a web server will very likely use OpenSSL to allow it to encrypt communications between yourself and it. The vulnerable versions of OpenSSL come equipped with new functionality – a “heart beat” that is used to keep connections alive and open.

When this functionality is not disabled and you are using a vulnerable version of OpenSSL, an attacker can make a connection to your server and ready up to 64Kbytes of the process memory. For each and every request.

This is a classic information leakage issue, and the attacker can trawl through a collection of 64Kbyte “chunks” of binary data looking for interesting information. In theory, these chunks of information can contain anything the process (the web server, the mail server, etc) contains within itself. Some examples include :-

  1. A researcher has used this vulnerability to expose Yahoo Mail account passwords.
  2. It is believed to be possible to extract a server’s private key to allow an attacker to decrypt communications traffic and/or impersonate the server.

Whilst trawling through binary chunks of data looking for interesting data is the sort of activity that seems to normal people to be so difficult that it would be almost impossible for someone. However it is possible, and for something like passwords is even easy. And for private keys, there are hints out there on how to do it.

But How Does This Affect Me?

If you are not a server administrator, this will all seem a bit geeky and not have much meaning for you.

It is probably better to ask: What should I do about this? And the answer is to do nothing unless you are advised to do so by a trusted source. Whatever damage has taken place already and service providers will be busy fixing the vulnerability.

The only addition to that is to make sure you update your software on your computers – your laptop, phone, tablet, etc. Whilst the media is concentrating on the server side of the problem, OpenSSL is also used on client machines, and that means that your computers are vulnerable in some way – whilst no exploits are known to exist today, it is still worth being proactive in making sure you apply updates.

Because sooner or later, attackers will use this vulnerability to attack you directly rather than via servers.

But How Serious Is This?

Very.

But perhaps not as much as some of the more extreme possibilities might suggest.

There is a great deal of probability involved here. For example, was it possible that this vulnerability was known to the “bad people” before the announcement this week? The vulnerability has existed for a year or two so it is possible it was known about. But probably not widely known.

Was it exploited? Possibly, but it’s probable that it wasn’t widely exploited – the activities of “bad people” tends to leak. If it was exploited, it was quite possible that it was limited to the NSA and GCHQ.

As to over-reaction, there was a comment on a blog entry about this that claimed that his Yahoo Mail account password had been compromised three times in the last month by this method. Well, possibly but it seems far more likely that his password had been compromised via other methods – such as using a weak password. Using this method against Yahoo’s servers may reveal some account passwords, but it is likely to reveal random account passwords each time. Meaning that an attacker will find it quite hard to compromise the password for a single account more than once.

Going forwards, it is very likely that this vulnerability will be used by “bad people” – there are already indications that they may be starting to try this.

So it is important and urgent for server administrators to look at this problem and fix :-

  1. Update vulnerable OpenSSL versions.
  2. Revoke the old SSL certificates
  3. Issue new SSL certificates.
  4. If passwords are known to have been compromised, issue a notice to suggest people change their passwords.

It is also important that client machines are updated as and when fixes are released.

Mar 132014
 

This blog posting talks about machine code and whether 32-bit code or 64-bit code is more appropriate. When AMD released the Opteron way back in 2003, it was the first processor to support the x86-64 instruction set for supporting 64-bit code whilst maintaining backward compatibility with the old IA32 code. Or in other words, the Opteron could run both 32-bit code and 64-bit code.

Everybody leaped onto the 64-bit bandwagon without thinking too much about it – it was faster.

But if we look at the other processors that made the transition from 32-bit code to 64-bit code – such as the SPARC, MIPS, etc., we find something interesting. Much of the code running on the relevant operating systems remained 32-bit – not as a transitional measure, but because the 32-bit code was faster. If you look at a relatively modern Solaris system the contents of /bin contain 500-odd binaries that are 32-bit and just 11 that are 64-bit (most of which aren’t in fact part of Solaris but another add-on package).

It turns out that in general, 64-bit code is slower than 32-bit code. In the case of x86-64, 64-bit code is faster not because it is 64-bit, but because of the architectural changes that were also introduced – including (probably most significantly) extra registers.

How do we know this? Apart from it being obvious to those who have lived through the 32-bit to 64-bit transition multiple times, it turns out that people have been experimenting. As it turns out, the x86-64 architecture does allow for 32-bit code to be run with all the features of the x86-64 architecture and that architecture has been labelled as X32.

It turns out that X32 code can be anywhere from 5-40% faster than 64-bit code. The largest increases come from code that makes very heavy use of pointers, and at present no benchmarks of “ordinary” software have been released.

The “downside” of X32 is of course that the software is limited to 4Gbytes of memory, but most programmes don’t need that much memory because 4Gbytes is a lot. Forget that huge video editor you’re playing with – that will quite possibly need 64-bit pointers, but what about all the other software running on your machine?

There are over 400 processes running on my workstation, and none of those processes really requires more than 4Gbytes of memory. Sure I run software that does require more than 4Gbytes of memory, but not all the time.

And running things 10% quicker would be useful … or alternatively running things quicker means the processor can spend more time asleep making battery life longer.

Feb 232014
 

Just an experiment in producing a video :-

Not especially good because I’m too lazy to go back and even out the exposure (so the lighting doesn’t keep changing).