Ubuntu ZFS: Why All The Filesystems?

 Computing, Linux  No Responses »
Apr 262020
 

A number of those who have experimented with Ubuntu’s ZFS install option (which as of 20.04 is marked as “experimental”) have expressed bewilderment over the number of filesystems created :-

The short answer as to why is that there are two schools of thought amongst grizzled old Unix veterans as to whether one big filesystem should be the way to go or lots of little ones. There are pros and cons to both approaches, and whilst I have a preference for lots of filesystems (especially on servers), I don’t care enough to change it on a laptop install.

Even though those who insist on one big filesystem are wrong.

As to the longer explanation …

Some History

A long time ago – the 1970s or the 1980s – Unix systems lacked sophisticated disk management software, and the disks were very much smaller (I started off with 80Mbyte disks and no that isn’t a typo, and many started with much smaller disks). On larger Unix servers, you couldn’t fit everything onto one disk, so we got used to splitting up the filesystem into many separate filesystem – / on one disk partition (or slice), /usr on another, /var on a third, /home on yet another, etc.

These very frequently got further subdivided – /var/mail, /var/tmp, /var/spool, etc. as Unix servers got larger and busier.

Those days are long past, and nobody is keen to go back to those days so why do some still like to split things up?

The Fringe Benefits of Splitting

It turns out that there was a fringe benefit to splitting up the filesystems – disk space exhaustion on one wouldn’t cause a problem elsewhere. For example if a mail server had a separate /var/spool/mail filesystem for operating within it would still continue to operate if /var filled up; similarly a DNS server wouldn’t crash and burn if it had a /var/named filesystem and /var filled up.

Both of those examples are known to me personally – and there are many other examples.

Of course there is also a downside – if you create a separate /var/spool/mail filesystem you need to make sure it is large enough to operate not just normally but in reasonable exceptional circumstances. Or your mail server crashes and burns.

On the other hand, if you don’t separate things out then when something goes berserk and fills up all the disk space then you will have a good deal of trouble actually logging in to fix things.

In a sense, the “everything in one” camp and “lots of little filesystems” camp are determined by what troubles we’ve seen over the years (and in some cases decades).

With something like ZFS you can set quotas to limit the size of any filesystem so managing the sizes of these separate filesystems is a great deal easier than it ever was in the past! Ubuntu does not set quotas by default on a desktop installation; for a server it may well be worth checking quotas and setting them appropriately.

And Snapshots …

One of the other things that Ubuntu does with ZFS and filesystem snapshots (we’ll worry about what those are another time) is to offer to rollback a broken update. People worry that upgrading their system will break things and the ability to quickly revert to the previous state is very comforting.

But the Unix file layout “standard” and the later Linux file layout standard were not designed with snapshots in mind, and simply rolling back the whole of “/” would have negative effects – not least you would lose any file changes you had made in /home and any mail stashed away in /var/mail.

So to implement the ability to rollback updates requires numerous separate filesystems to avoid losing important data.

It is also likely that it would be beneficial to tune separate filesystems for different requirements.

Finally

In short, don’t worry about it. It’ll have very little effect on your operation of a normal Ubuntu machine unless you choose to take advantage of it. And it makes possible certain features that you will probably like – such as the ability to revert updates.

Messing With Rofi: Inserting Unicode

 Computing, Linux, Working Notes  No Responses »
Feb 292020
 

I used to be able to remember all of the keyboard short-cuts I’d set up to insert ‘fun’ (or useful) Unicode characters … but my memory isn’t quite what it used to be, and I happened to catch a video that mentioned a menu to insert Unicode characters.

So I set out to create my own …

#!/bin/zsh
#
# Run a menu of Unicode characters to put into the clipboard

read -r -d '' menu << END
Þ       Thorn (Sym-t)
Þ	Capital thorn (Sym-T)
✓	Tick (Sym-y)
✔       Alternate tick (Sym-Y)
π	pi (Sym-p)
Π       PI (Sym-P)
★       Star
🖕      Finger
END

selectedchar=$(echo $menu |\
  rofi -dmenu -l 20 -fn misc-24 -p "Unicode inserter" |\
  awk '{print $1}')
if [ -z "$selectedchar" ]
then
  notify-send "Unicode Inserter" "Cancelled"
else
  printf "%s" $selectedchar | xclip -selection clipboard
  notify-send "Unicode Inserter" \
    "Character $selectedchar now in clipboard"
fi

This script :-

  1. Creates a variable with a whole pile of text inside it. The original script contains a lot longer list, but including it would be a) boring and b) give too much away. As it stands, the format of each line is pretty much anything you want as long as the first character is the Unicode character followed by whitespace.
  2. Runs rofi with the variable as input and selects the first field of the response.
  3. Guards against the empty selection (when the menu was cancelled) for neatness mostly.
  4. Prints the selected character (without a newline) into xclip so it can be pasted in. I did try using xdotool to type it directly into the active window, but this didn’t always work so well (i.e. xdotool couldn’t “type” some of the more esoteric characters).
  5. And uses notify-send to alert the dumb user (me!) that something has happened.

Lastly, to make this useful, I added an entry to my sxhkd configuration :-

super + F11
  /site/scripts/m-unicode-insert

Linux Steam: A Fix for Civilization V

 Computing, Linux  No Responses »
Jun 292019
 

(And apologies for the misspelling; that words is spelt “civilisation” around these parts)

So I’m a Linux user and whilst I don’t often play games, an occasional break spending time slaughtering pedestrians (in GTA V) or conquering the world (in ‘Civilization’) can be fun.

Except that I have not been able to play Civilization V since I bought it through Steam – usually instant crashes although sometimes it worked well enough for a few turns.

Today I sat down and tried out various suggestions on fixing it until I found one that worked for me :-

  1. Right-click the game name in Steam.
  2. Select “Properties”
  3. Select “Startup options”
  4. Specify one of the following :-
    1. (Failed for me) LD_PRELOAD=’./libcxxrt.so:/usr/$LIB/libstdc++.so.6′ %command%
    2. (Failed for me) LD_PRELOAD=/usr/lib32/libopenal.so.1 %command%
    3. taskset –cpu-list 0-3 %command%

As implied, the “taskset” startup option appears to have worked for me. According to the site I nicked it from, Civilization has trouble running on systems with more than 8 core threads.

Tower Stonework

Zsh: Renewed Hostname Completion

 Computing, Linux  No Responses »
Jun 082019
 

Quite a while ago, I “borrowed” some inscrutable zsh magic to automatically add the contents of ~/.ssh/known_hosts to a known_hostsi variable and used that variable to perform host name completion for certain commands. Once ssh started hashing the known_hosts file, this broke and I was busy at the time and stopped using it.

Ages later, I’ve fixed it and enhanced it a bit (and arguably made it a bit simpler). Not only does it pick up host names from the known_hosts file but also adds a list from lxc-ls and adds a few static host names (with one exception, not shown). This is done by adding the following to .zshrc :-

knownhosts=( $(sudo -b lxc-ls) )
#       Pick up a list of hosts from lxc-ls
knownhosts+="localhost"
#       Add static hostnames
for x in $(grep -v "|" ~/.ssh/known_hosts | awk '{print $1}' | awk -F, '{printf "%s ", $1}')
do
  knownhosts+=$x
done
#       Pull a list of hosts out of ~/.ssh/known_hosts excluding the Hashed hosts.
zstyle ':completion:*:(ssh|scp|sftp|ping|nmap):*' hosts $knownhosts
#       Commands to use a list of known hosts with

That probably is not the most efficient code, but does have the advantage that it is relatively easy to follow.

One addition is to add the option HashKnownHosts no to ~/.ssh/config.

There is of course a risk associated with disabling the hashing of host names within the known_hosts file. If your host becomes compromised, malicious code can use that file to obtain a list of hosts with which there is a trust relationship making it easier for an attacker to pivot through your network.

Linux Containers: Configuring Network Address (part 636)

 Computing, Linux, Working Notes  3 Responses »
Jun 052019
 

On previous occasions (yes that does mean more than once) I have messed around with the network configuration of containers to get :-

  1. A consistent behaviour.
  2. A fixed IPv4 address with no DHCP configuration (this one is easy).
  3. A fixed IPv6 address with no autoconfigured global addresses (this one has been tricky)

This turns out to be relatively easy providing that you configure the addresses within the container rather than within the container configuration. At least it looks good to go so far (I’ve been mistaken in the past).

The container configuration is quite simple :-

lxc.net.0.type = veth
lxc.net.0.flags = down
lxc.net.0.link = br0

Note that the bridge interface (br0) may be different. Also note that there is no lxc.net.0.ipv4.address, lxc.net.0.ipv4.gateway, lxc.net.0.ipv6.address, or lxc.net.0.ipv6.gateway.

The configuration within the container is dependent on what userland you are running, but for Debian (and Ubuntu if you’re not using Netplan) :-

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 10.0.0.34/16
    gateway 10.0.0.1

iface eth0 inet6 static
    address 2001:0db8:cafe:dead:0000:0000:0000:3eb/64
    scope global
    gateway 2001:0db8:cafe:dead:0000:0000:0000:0001
    privext 0
    accept_ra 0
    autoconf 0

Not sure quite which options are required but having all of “privext 0”, “accept_ra 0” and “autoconf 0” does mean no additional autoconfigured IPv6 addresses.

(And no the part number of this post isn’t anything more than a joke)

 Older Entries  Newer Entries