Post History

Decomposing compound questions

The basis of the Unix philosophy is to decompose complex tasks into simple sub-tasks, so that one can easily choose a combination of simple Unix tools to solve the problem in an intuitive way.

Often I see newbies ask questions along the lines of "how do I X complicated and specific thing". The greybeards then oblige with an even more complicated and impenetrable one liner. The newbie is left with a renewed conviction that the Unix command line is magic, and only a wizened old shell-master, who has spent his life memorizing every man page, can hope to use its powers.

The Unix command line is not a magician's riddle. You're not supposed to "just know" exactly the right obscure incantation for every situation, and if you don't then it's your punishment to do things by hand. It's supposed to be a powerful, intuitive toolkit that works together with the natural problem solving and analysis abilities of your brain.

Whenever someone asks how to do X in the shell, it should be a valid and preferred answer to explain that:

* X is actually a combination of tasks Y and Z
* Link to questions about doing Y, and questions about doing Z
* Explain how to combine them together (eg. with pipes)

The alternative is that for `N` types of problems, we will end up answering `N + N*N + N*N*N + ...` questions for every combination of them. That is silly, clogs up the site with tedious junk, and does a great pedagogical disservice to new Unix users. Instead we should simply have `N` questions explaining each simple task, and a handful more to explain how to compose them together.

As an example, imagine questions like:

* How do I delete every file that meets X condition
* How do run a conversion program on every file of type Y
* How do I get the file counts of every file in these directories
* Infinitely many variations

These all come down to the same thing:

* How can I obtain a list of paths? (refer to `find`, `fd`, `ls`...)
* How do I filter a list of paths based on criteria? (`find` arguments, shell `if`, many others)
* How can I run a program on every file? (refer to pipes, `xargs`, Shell expansion, GNU `parallel`, Shell `for`...)
* How can I do X task on Y file? (obvious, but refer to `rm`, etc)

The only real answer that needs to be given to such questions is to explain how to decompose the task into more obvious sub-tasks, how to pick appropriate Unix tools for those sub-tasks and how to combine them.

As a FOSS advocate, I find it frankly shameful and reprehensible that a community for helping Unix users would keep the wonderful concept of the Unix philosophy a secret from newbies, for the sole reason that the newbies did not think to explicitly ask about it without knowing its existence. Maybe on codidact we can do better?