CLI Archives - More Zonkyness

Amazing Command-Line Tools: The Pipe

Jul 232022

I was following one of those Twitter threads posting their favourite command-line tools (specifically for infosec), and added my own entry – the incomparable tshark. Later it occurred to me that the best command-line tool isn’t really a tool at all as it is built into the shell – the pipe. Many of the command-line tools just wouldn’t be quite the same without it.

For those who aren’t familiar with the command-line, the pipe (“|”) takes the output of one command and feeds it as input to another command. And you can string such pipelines together to add to each other (which can lead to inefficiencies).

For example :-

» ls | wc -l
84

This takes the usual command for listing files and sends the output into the “word count” command to produce a count of the number of files in the current directory. To be more precise, it produces a count of the number of files that ls thinks is in the directory. You can get different results with different variations :-

» echo * | wc -w
89
» ls -a | wc -l
463

If you had a log file containing DHCP requests you could :-

» grep DHCPDISCOVER 2022.07.local0.info.log | head
2022-06-30T23:59:05+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T01:30:04+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T02:53:33+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via eth0
2022-07-01T02:53:33+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via 10.0.0.1
2022-07-01T02:53:39+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from a8:a6:48:92:9d:36 via eth0
2022-07-01T03:01:03+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T04:32:02+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T04:56:53+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 91:06:27:15:EF:DC via 10.72.0.1
2022-07-01T06:03:01+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1
2022-07-01T07:34:00+00:00 <local0.info> 2001:db8:bad:cafe::b/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1

List out the first few DHCP DISCOVER requests (the astute may notice that I’ve done some obfuscating). We can then pick out a field using awk to list just the MAC addresses :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | head
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4
DF:69:AF:DC:79:3E
DF:69:AF:DC:79:3E
a8:a6:48:92:9d:36
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4
91:06:27:15:EF:DC
4D:6D:4F:55:59:B4
4D:6D:4F:55:59:B4

We can then remove the “head” command and add a sort and uniq command to produce a full list of all MAC addresses that have performed a DHCP DISCOVER :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c
      4 DF:69:AF:DC:79:3E
      3 89:C1:67:B8:9D:6F
      6 F3:55:1E:06:D4:49
      4 F3:55:1E:06:D4:48
     12 4D:6D:4F:55:59:B3
     92 91:06:27:15:EF:DC
     46 85:2C:B4:B3:70:7E
    333 4D:6D:4F:55:59:B4
      2 40:5B:D8:FF:FA:29
     72 FD:D4:00:41:29:BE
      5 36:1E:07:2D:AD:76
     41 44:FD:6E:05:82:21
     81 CC:78:14:BB:E4:3D

We can sort the result into reverse numerical order :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n
    333 4D:6D:4F:55:59:B4
     92 91:06:27:15:EF:DC
     81 CC:78:14:BB:E4:3D
     72 FD:D4:00:41:29:BE
     46 85:2C:B4:B3:70:7E
     41 44:FD:6E:05:82:21
     12 4D:6D:4F:55:59:B3
      6 F3:55:1E:06:D4:49
      5 36:1E:07:2D:AD:76
      4 F3:55:1E:06:D4:48
      4 DF:69:AF:DC:79:3E
      3 89:C1:67:B8:9D:6F
      2 40:5B:D8:FF:FA:29

And if you have access to the relevant script, you can produce terminal graphics (just to keep innumerate managers happy) :-

» grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n | awk '{print $2, $1}' | tbar --replace 1 --max 350
4D:6D:4F:55:59:B4 ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
91:06:27:15:EF:DC ■■■■■■■■■■■■■■■
CC:78:14:BB:E4:3D ■■■■■■■■■■■■■■
FD:D4:00:41:29:BE ■■■■■■■■■■■■
85:2C:B4:B3:70:7E ■■■■■■■
44:FD:6E:05:82:21 ■■■■■■■
4D:6D:4F:55:59:B3 ■■
F3:55:1E:06:D4:49 ■
36:1E:07:2D:AD:76 
F3:55:1E:06:D4:48 
DF:69:AF:DC:79:3E 
89:C1:67:B8:9D:6F 
40:5B:D8:FF:FA:29

The pipe isn’t so much a tool itself as a mechanism to combine tools into producing interesting results.

Shell Alias May Be Dangerous

Computing, Linux No Responses »

Jul 132022

Not all shell aliases of course, but some. I’ve just seen a youtube video that suggested creating a shell alias to run rmtrash when rm is invoked :-

alias rm='rmtrash'

Seems sensible enough doesn’t it? This is in fact the classic example of how dangerous shell aliases can be, although the classic example was to turn on “-i” :-

alias rm='rm -i'

The problem is that you get used to “rm” being safe – either it asks before it removes files (“-i”) or it safely preserves what is deleted in the Trash folder. But what happens when the alias doesn’t get created? Perhaps you have a broken .zshrc and Zsh stops interpreting before the alias is declared. Or you’ve logged on to a remote server that doesn’t have your .zshrc installed as yet?

All of a sudden you are running the unadulterated rm command – deleting files without being asked, or preserving them in the Trash folder. See the danger now?

It is better not to replace standard commands but create a new ‘command’ :-

alias del="rmtrash"

Perhaps you regard this as being excessively risk averse – fair enough. But just don’t say you weren’t warned – and I’ve encountered missing aliases every year over the last 30-odd years I’ve been using Linux and Unix.

When Is a Command-Line Application Not?

Computing, Linux No Responses »

Jun 022022

It sometimes seems that every time I dive into a Youtube video promising “${N} Awesome CLI Applications” (or equivalent), that most of the suggested applications are not command-line applications. They’re TUI-applications – text user interface as opposed to graphical user interface – or to align with my bad habit of referring to GUI applications as gooey applications, perhaps tooey applications.

Now there’s nothing wrong with tooey applications; I use them every day. Especially nmon (just because I got used to it before I discovered htop). Or btop

But none of these are really command-line applications; by which I mean they aren’t used at the command-line even if they are (optionally) invoked there. A command-line application allows you to use the shell including pipes to produce an aggregate result. For example :-

» grep mike /etc/passwd | awk -F: '{print $5}'
Mike Meredith

That uses two command-line “applications” to turn a username (“mike”) into a full name (“Mike Meredith”). Yes it can be optimised into a single command :-

» awk -F: '/^mike:/ {print $5}' /etc/passwd
Mike Meredith

… which even improves the search, but makes the point less well. And we can do slightly fancier things too :-

Screenshot of a random URL being picked and turned into a QR code.

(don’t assume that QR code takes you somewhere nice)

I’m not suggest Youtubers should stop making videos about terminal-based applications; I’m not even suggesting they should concentrate on “proper” command-line applications. Just don’t call terminal-based applications “command-line” because they really are not.

Linux: No You Don’t Need The Command-Line, But It Isn’t That Scary

Computing, Linux No Responses »

Sep 082018

Having used Linux for well over 20 years (yes it is that old), and Unix before that, I’m often puzzled by how scary some people seem to find Linux. Why should it be scary? It’s just a computer – you’re the human in charge of it.

Yes There Are Gooeys

(graphical user interfaces – GUIs – gooeys)

Yes there is plenty of software with a graphical user interface – I use plenty on a daily basis including a standard web browser, an email client, a password manager, an office package.

On a slightly less frequent basis there are many more that I use. Indeed providing that you accept the use of alternatives, you can find Linux software to do just about anything.

But Don’t Ignore The Command-Line

Yes, Linux has a command-line, and for those of us familiar with it, it can be very powerful. And there is no harm in learning how to use the command-line just to the point where you can follow instructions on how to “get something done” there.

Because if I have a fix for some niggle that you are having, it is easier and less error-prone to pass instructions for a command-line incantation than instructions for a gooey (and yes I have done both).