{"id":6194,"date":"2022-07-23T11:34:11","date_gmt":"2022-07-23T11:34:11","guid":{"rendered":"https:\/\/really.zonky.org\/?p=6194"},"modified":"2022-07-23T11:34:11","modified_gmt":"2022-07-23T11:34:11","slug":"amazing-command-line-tools-the-pipe","status":"publish","type":"post","link":"https:\/\/really.zonky.org\/?p=6194","title":{"rendered":"Amazing Command-Line Tools: The Pipe"},"content":{"rendered":"\n<p>I was following one of those Twitter threads posting their favourite command-line tools (specifically for infosec), and added my own entry &#8211; the incomparable <em>tshark<\/em>. Later it occurred to me that the <em>best<\/em> command-line tool isn&#8217;t really a tool at all as it is built into the shell &#8211; the pipe. Many of the command-line tools just wouldn&#8217;t be quite the same without it. <\/p>\n\n\n\n<p>For those who aren&#8217;t familiar with the command-line, the pipe (&#8220;|&#8221;) takes the output of one command and feeds it as input to another command. And you can string such <em>pipelines<\/em> together to add to each other (which can lead to inefficiencies).<\/p>\n\n\n\n<p>For example :-<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u00bb ls | wc -l\n84<\/code><\/pre>\n\n\n\n<p>This takes the usual command for listing files and sends the output into the &#8220;word count&#8221; command to produce a count of the number of files in the current directory. To be more precise, it produces a count of the number of files that <em>ls<\/em> thinks is in the directory. You can get different results with different variations :-<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u00bb echo * | wc -w\n89\n\u00bb ls -a | wc -l\n463<\/code><\/pre>\n\n\n\n<p>If you had a log file containing DHCP requests you could :-<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u00bb grep DHCPDISCOVER 2022.07.local0.info.log | head\n2022-06-30T23:59:05+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1\n2022-07-01T01:30:04+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1\n2022-07-01T02:53:33+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via eth0\n2022-07-01T02:53:33+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from DF:69:AF:DC:79:3E via 10.0.0.1\n2022-07-01T02:53:39+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from a8:a6:48:92:9d:36 via eth0\n2022-07-01T03:01:03+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1\n2022-07-01T04:32:02+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1\n2022-07-01T04:56:53+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from 91:06:27:15:EF:DC via 10.72.0.1\n2022-07-01T06:03:01+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1\n2022-07-01T07:34:00+00:00 &lt;local0.info> 2001:db8:bad:cafe::b\/d-FCB dhcpd: DHCPDISCOVER from 4D:6D:4F:55:59:B4 (esp32-D04CCC) via 10.72.0.1<\/code><\/pre>\n\n\n\n<p>List out the first few DHCP DISCOVER requests (the astute may notice that I&#8217;ve done some obfuscating). We can then pick out a field using <em>awk<\/em> to list just the MAC addresses :-<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u00bb grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | head\n4D:6D:4F:55:59:B4\n4D:6D:4F:55:59:B4\nDF:69:AF:DC:79:3E\nDF:69:AF:DC:79:3E\na8:a6:48:92:9d:36\n4D:6D:4F:55:59:B4\n4D:6D:4F:55:59:B4\n91:06:27:15:EF:DC\n4D:6D:4F:55:59:B4\n4D:6D:4F:55:59:B4 <\/code><\/pre>\n\n\n\n<p>We can then remove the &#8220;head&#8221; command and add a sort and uniq command to produce a full list of all MAC addresses that have performed a DHCP DISCOVER :-<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u00bb grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c\n      4 DF:69:AF:DC:79:3E\n      3 89:C1:67:B8:9D:6F\n      6 F3:55:1E:06:D4:49\n      4 F3:55:1E:06:D4:48\n     12 4D:6D:4F:55:59:B3\n     92 91:06:27:15:EF:DC\n     46 85:2C:B4:B3:70:7E\n    333 4D:6D:4F:55:59:B4\n      2 40:5B:D8:FF:FA:29\n     72 FD:D4:00:41:29:BE\n      5 36:1E:07:2D:AD:76\n     41 44:FD:6E:05:82:21\n     81 CC:78:14:BB:E4:3D<\/code><\/pre>\n\n\n\n<p>We can sort the result into reverse numerical order :-<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u00bb grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n\n    333 4D:6D:4F:55:59:B4\n     92 91:06:27:15:EF:DC\n     81 CC:78:14:BB:E4:3D\n     72 FD:D4:00:41:29:BE\n     46 85:2C:B4:B3:70:7E\n     41 44:FD:6E:05:82:21\n     12 4D:6D:4F:55:59:B3\n      6 F3:55:1E:06:D4:49\n      5 36:1E:07:2D:AD:76\n      4 F3:55:1E:06:D4:48\n      4 DF:69:AF:DC:79:3E\n      3 89:C1:67:B8:9D:6F\n      2 40:5B:D8:FF:FA:29 <\/code><\/pre>\n\n\n\n<p>And if you have access to the relevant script, you can produce terminal graphics (just to keep innumerate managers happy) :-<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u00bb grep DHCPDISCOVER 2022.07.local0.info.log | awk '{print $7}' | sort | uniq -c | sort -r -n | awk '{print $2, $1}' | tbar --replace 1 --max 350\n4D:6D:4F:55:59:B4 \u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\n91:06:27:15:EF:DC \u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\nCC:78:14:BB:E4:3D \u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\nFD:D4:00:41:29:BE \u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\n85:2C:B4:B3:70:7E \u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\n44:FD:6E:05:82:21 \u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\n4D:6D:4F:55:59:B3 \u25a0\u25a0\nF3:55:1E:06:D4:49 \u25a0\n36:1E:07:2D:AD:76 \nF3:55:1E:06:D4:48 \nDF:69:AF:DC:79:3E \n89:C1:67:B8:9D:6F \n40:5B:D8:FF:FA:29 <\/code><\/pre>\n\n\n\n<p>The pipe isn&#8217;t so much a tool itself as a mechanism to combine tools into producing interesting results. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"695\" height=\"521\" src=\"https:\/\/i0.wp.com\/really.zonky.org\/wp-content\/uploads\/2019-03-17-its-round.jpeg?resize=695%2C521&#038;ssl=1\" alt=\"\" class=\"wp-image-5526\" srcset=\"https:\/\/i0.wp.com\/really.zonky.org\/wp-content\/uploads\/2019-03-17-its-round.jpeg?resize=1024%2C768&amp;ssl=1 1024w, https:\/\/i0.wp.com\/really.zonky.org\/wp-content\/uploads\/2019-03-17-its-round.jpeg?resize=300%2C225&amp;ssl=1 300w, https:\/\/i0.wp.com\/really.zonky.org\/wp-content\/uploads\/2019-03-17-its-round.jpeg?resize=768%2C576&amp;ssl=1 768w, https:\/\/i0.wp.com\/really.zonky.org\/wp-content\/uploads\/2019-03-17-its-round.jpeg?w=1584&amp;ssl=1 1584w, https:\/\/i0.wp.com\/really.zonky.org\/wp-content\/uploads\/2019-03-17-its-round.jpeg?w=1390&amp;ssl=1 1390w\" sizes=\"auto, (max-width: 695px) 100vw, 695px\" \/><figcaption>It&#8217;s Round<\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>I was following one of those Twitter threads posting their favourite command-line tools (specifically for infosec), and added my own entry &#8211; the incomparable tshark. Later it occurred to me that the best command-line tool isn&#8217;t really a tool at all as it is built into the shell &#8211; the pipe. Many of the command-line <a href='https:\/\/really.zonky.org\/?p=6194' class='excerpt-more'>[&#8230;]<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false,"_share_on_mastodon":"1"},"categories":[4,209],"tags":[1793,43,2117,1686,1386],"class_list":["post-6194","post","type-post","status-publish","format-standard","hentry","category-it","category-linux-it","tag-cli","tag-linux","tag-pipe","tag-pipeline","tag-shell","category-4-id","category-209-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"share_on_mastodon":{"url":"https:\/\/mstdn.social\/@grumpygrimnir\/108696440144144433","error":""},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1f2KI-1BU","_links":{"self":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts\/6194","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6194"}],"version-history":[{"count":2,"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts\/6194\/revisions"}],"predecessor-version":[{"id":6200,"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts\/6194\/revisions\/6200"}],"wp:attachment":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6194"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6194"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6194"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}