Internet – More Zonkyness

How CrowdStrike Broke The Internet

Jul 202024

This is a bit of rant poking fun at the sheer quantity of misinformation about CrowdStrike’s little issue yesterday (to clarify when this post was written – more information will come out).

Microsoft

Some of the earliest symptoms of the issue were some Microsoft services having issues. Oddly enough I wasn’t using many of those yesterday (I usually do) except for Teams which didn’t seem to suffer … at least not as much.

It appears that Microsoft may run CrowdStrike Falcon on at least some of their servers (although the jury is still out on this one – some are saying it was an independent outage). Despite Microsoft having their own security tools (Defender), this isn’t quite as unlikely as it may seem – particularly safety conscious organisations may well run two of more anti-malware products.

And CrowdStrike is more mature than Defender at least in the fancy “behavioural analytics” area.

The Internet

… wasn’t broken at all. Many services were broken true enough, but probably more were working just as well as normal. Microsoft’s platforms are very widely used, and CrowdStrike is a big name in cybersecurity, so it is hardly surprising that there was so much disruption.

But to say this broke the Internet is a bit of an exaggeration. Kind of what you would expect from mainstream media.

Who Are CrowdStrike?

Not surprisingly, many people just haven’t heard this name before. It is very widely known in the cybersecurity community with a wide variety of security focused services, including top-flight anti-malware products.

But they don’t sell to individuals so they are not well known amongst the general community.

The product at the centre of all this is CrowdStrike Falcon, an anti-malware agent that goes a bit beyond “anti-virus” in that it attempts to go beyond blocking known viruses and attempts to block behaviours known to be malicious.

As such, it receives very frequent updates – up to every hour (although probably many hours) which puts this sort of catastrophic failure at a rate of somewhere in the order of 0.001%.

What Went Wrong?

This starts to get a bit technical …

Some of this was informed by CrowdStrike’s update; some by educated (I work in this field although I’m not familiar with CrowdStrike’s product) common sense.

First of all, this was not a kernel driver update (although the relevant filename made it appear so) but a content update. As previously mentioned, these are sent out very frequently. The content update triggered a bug in the kernel driver and caused a “blue screen of death“. This would repeat after every reboot until the relevant update was removed or updated (the crash doesn’t occur immediately which sometimes allows the agent to download a fixed update).

Secondly this update was tested before being released (do you really believe that an approximately 0.001% failure rate is achieved without testing?), but something went wrong with the testing process. We don’t know what, and CrowdStrike don’t either. Yet.

Why Was It So Widespread?

Simply because although generally unknown to the general public, CrowdStrike Falcon is generally regarded as an excellent security product and is very widely used. Perhaps more widely used than previously suspected.

But the whole Internet? Clearly not, but it’s in the mainstream’s media to be a bit ‘click-baity’ in their reports.

DNS Naming

Computing No Responses »

Jul 242018

As someone who has spent far too much time dealing with the Domain Name System, I get kind of miffed when people insist on creating names that conflict with the DNS ordering. You see the DNS naming works from right-to-left (the wrong way around if you’re reading this in English).

Take the name for this site – really.zonky.org – which is admittedly a rather quirky name. The most significant part of the name is at the right (org – and yes I’m ignoring the really significant and invisible “dot”). The next most significant part (zonky) specifies what organisation has registered the site (me), and the least significant part (really) points to one service at that organisation.

So when people ask for names that break that ordering it is ever so slightly irritating – for example if you have a service called mail.zonky.org and wanted a test service you might request mail-test.zonky.org which breaks the ordering of things. As an alternative, test.mail.zonky.org doesn’t break the naming, looks a bit nicer, and ultimately more reasonably flexible.

Let us look at a slightly more complex example; let’s assume that we have a domain called db.zonky.org and want to register a service name for each database. We could register names such as db-addresses.zonky.org, and db-orders.zonky.org, or we could register them instead as addresses.db.zonky.org and orders.db.zonky.org. In the later case, I can very quickly write a firewall rule that allows access to *.db.zonky.org (whereas db-*.zonky.org would not work).

Ultimately suggest names in DNS naming order unless you can justify why it is not suitable.

Decentralising The Internet To Keep Speech Free

Computing No Responses »

Jan 112016

Watching the 32c3 conference videos for free (which is relevant), and coming across the inevitable “the Internet is dead”, “corporations have bored the spirit of the Internet to death”, etc. It’s a pretty common meme amongst those who somehow believe that the Internet used to be free.

The Internet was never free, but it did have the appearance of being free.

Of course we have become used to paying for access to the Internet, but that monthly payment to the ISP doesn’t pay for the Internet as a whole. As an example none of the money you pay your ISP reaches me to help me pay for the server this web page is on. Supposedly I can ‘monetise’ your visits by publishing adverts on my web site; in practice it doesn’t. At least not for low traffic sites.

And things like Facebook or Twitter do cost lots of money to run; enough that many of the large successful companies took a long time before they became profitable.

Of course I’ve been concentrating on the monetary meaning of “free” but this applies to a certain extent to the other meaning of free – you can’t post content to Facebook that they don’t agree with (although in practice very little is censored with the main victim being pictures of breastfeeding). A company like Facebook is in this game to make money and whilst they are not going to censor your content for no reason, neither are they going to fight too hard for your free speech.

In the end you can only exercise your freedom of speech on your own servers. But with the exception of a few weirdos like me, most of us are not keen on running servers.

All normal people want to do is run an application that lets them “do stuff” and the conventional way to implement an Internet application is for an application running on the person’s desktop to speak across the Internet to a server – for example the web works this way. The big problem with such an approach particularly when something like Facebook becomes almost ubiquitous is that you are giving a large central organisation a lot of data about yourself.

Of course everyone who is not up to anything nefarious is not bothered by that, right? Well perhaps, but there are other aspects of sending all your data to a company who desperately needs to monetise your data and your eyeballs. Such as targeted advertising. And worse.

The conventional way. There are of course what could be called unconventional applications that communicate across the Internet without a centralised server managing it all. These are commonly called “peer-to-peer” (or P2P) applications and are commonly used to share files; very commonly files that the copyright owner would rather not be shared (films, applications, music, etc.). So P2P has a bit of a rogue reputation.

But it is merely a means for communicating and does not dictate what is communicated. There is nothing to stop someone from implementing a P2P-based application that communicates “posts” that are the equivalent of Facebook posts. Such an application :-

Would continue to use the web browser as a display engine.
Run as a separate “service” on the desktop to send and receive P2P posts in the background; displaying relevant ones on request.
Automatically encrypt all postings so that they can only be decrypted by the relevant audience. Keep the automatic encryption hidden to avoid scaring those who just can’t be bothered with all that.

Of course as I am not going to be writing this (I simply don’t have the time), I have no right to say how it should be written! But writing such an application would be very beneficial if we could persuade people to use it rather than the monolith that is Facebook. Unlike some people, I don’t believe that Facebook is intentionally evil, but because of the centralisation of social networking in the hands of Facebook, it has the potential to be evil.

Those who want the Internet to be free (as in freedom) need to put their money where their mouth is and write the code.

The Latest Map Of The Internet

Computing, Security No Responses »

Mar 242013

The above links to an interesting browser which allows zooming and selection of different data sets. It’s worth a look if you’re into that sort of thing. Although it’s rather surprising that it doesn’t like IPv6 addresses!

The most controversial thing about this map of the Internet gathered during 2012, is that it was produced with the aid of a botnet or in other words this researcher stole the resources they needed. Which is obviously wrong – no matter how good the cause – but now that it has been done, there is no reason not to look at the results (whilst wrong this isn’t really evil).

The first interesting discovery here is that this anonymous researcher managed to write a simple virus that would load the Internet scanner onto many devices with default passwords set – admin accounts with “admin” as the password, root accounts with “root” as the password, etc. You would have thought that such insecure devices would have been driven off the Internet by now, but it turns out not to be the case – there are at least 420,000 of them!

You could even argue that the owners of such machines are asking to have their devices controlled by anyone who wants to. Perhaps a little extreme, but certainly some people think so or this Internet survey wouldn’t exist.

But now the results. If you look at the default settings in the browser above, you will encounter large swathes of black squares where apparently nothing is in use. The trouble is that whilst it is true that an IP address that is pingable, or has ports open is “in use”, there is no guarantee that an IP address that is just registered in the DNS is in use or not, and finally unregistered IP addresses that do not appear to do anything may very well still be in use.

Essentially the whole exercise hasn’t really said much about how much of the Internet address space is in use, although that is not to say that the results are not useful.

One special point to make is that many of the large black squares that appear unused, are allocated to organisations that may very well want to have proper IP addresses that are not connected to the global Internet. That is not wrong in any way – before the wide spread adoption of NAT, it was common and indeed recommended that organisations obtain a public IP address before they were connected to the Internet to avoid duplicate network addresses appearing. And an organisation that legitimately obtained an old “class A” has no obligation to return the “unused” network addresses back to the unallocated pool. And even if they did, it would not make a big difference; we would still run out of addresses.

The answer to the shortage of IPv4 addresses is IPv6.

The Governance of the Internet – Should We Allow The ITU To Take Over?

Computing No Responses »

Nov 242012

As could be expected, when there are yet again moves made to pass the job of Internet Governance into the hands of the ITU, there is a huge wave of objections from the Americans; some of whom are objecting more from a reflex anti-UN position (or a wish to see the US remain “in control” of the Internet) rather than a more considered objection.

What is perhaps more surprising is the EU’s objections to the ITU taking control.

What Is Internet Governance?

In a very real sense, there is no such thing as the Internet; there are merely a large number of different networks that agree to use the Internet standards – protocol numbers, network addresses, names, etc. With the exception of names this is all pretty invisible to ordinary users of the Internet; at least when it works.

There is nothing to stop different networks from changing the Internet standards, or coming up with their own networking standards. Except of course that a network’s customers might very well object if they suddenly can’t reach Google because of different standards. Historically there has been a migration towards Internet standards rather than away from them.

In a very real sense, this is governance by consent. At least by the network operators.

It may be worthwhile to list those things that the current Internet Governance doesn’t do :-

It does not control network traffic flows or peering arrangements. Such control is exercised by individual networks and/or governments.
It does not control the content of the Internet. Not only is censureship not part of the current governance mission; it isn’t even within their power. Any current censureship is exercised by the individual networks and/or governments.
It does not control access, pricing, or any other form of network control. Your access to the Internet is controlled by your ISP and any laws enacted by your government.

There is probably a long, long list of other things that the current Internet Governance does not do. To a very great extent, the current governance is about technical governance.

What’s So Bad About The Status Quo?

“The Internet” is currently governed by ICANN (the “Internet Corporation for Assigned Names and Numbers”) which is a US-based (and controlled) non-profit corporation. Whilst there are plenty of those who complain about ICANN and how it performs it’s work, the key metric of how well they have performed is that just one of their areas of responsibility – the control of the top-level domains in the DNS – has resulted in any alternatives.

And those alternatives are really not very successful; as someone who runs an institutional DNS infrastructure, I would be under pressure to support alternative roots if they were successful enough to interest normal people. No such requests have reached me.

So you could very well argue that technically ICANN has done a perfectly reasonable job.

But politically, it is a far more difficult situation. ICANN is a US-based corporation whose authority over the Internet standards is effectively granted to it by the US Department of Commerce. This grates with anyone who is not a US citizen, which is now by far a majority of the Internet population.

Historically the Internet is a US invention (although the historical details are quite a bit more complex than that; it is widely acknowledged that the packet switching nature of the ARPAnet was inspired by work done by a British computer scientist), so it is not unreasonable that Internet governance started as a US organisation.

But in the long term, if it remains so, it will be undemocratic and tyrannical; whilst the US is a democratic government it is only US citizens that can hold their government to account with a vote. The rest of us have no say in how the US government surpervises ICANN which is an untenable situation.

What About The ITU ?

The key to any change in how Internet governance is managed, is to make as few changes as possible. If we accept that ICANN has managed reasonably well at the technical governance, there is no overriding reason to take that away from them. If we accept that control of ICANN has to be passed to an international body, then what about the ITU ?

Many people object to the idea of the ITU being in charge for a variety of reasons, but probably the biggest reason of all is that it is a UN body and certain people start frothing at the mouth at the mere mention of the UN.

But if you look at the history of the ITU, you will see that despite the beaurocratic nature of the organisation (which predates the UN by a considerable number of years), it has managed to maintain international telecommunications through two world wars. A not inconsiderable achievement even if it succeeded because it had to succeed.

Time For A Compromise

International agreement is all about making all parties equally satisfied … or at the very least equally disastisfied, with a solution that comes as close as possible to giving everyone what they want. A seemingly impossible task.

But despite spending nowhere near enough time studying the issues, one solution does occur to me. Hand over the authority by which ICANN operates to the ITU with the proviso that any changes to the mandate of ICANN (in particular giving it additional authority) should be subject to oversite by the UN as a whole; and of course subject to UN Security Council vetos.

Of course this is not a decision that should be made hastily; given that the main issue at stake is “political” rather than technical, there is no reason why the decision to do something has to be made quickly. But it does need to be made within 10 years.

Older Entries