Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Post History

77%
+5 −0
Q&A How to run a command on a list of files?

There are several options, like xargs and for. I'll leave those for other answers and only describe my favorite, GNU Parallel. You will have to install it separately, because unlike the inferior xa...

posted 12mo ago by matthewsnyder‭  ·  edited 11mo ago by matthewsnyder‭

Answer
#5: Post edited by user avatar matthewsnyder‭ · 2023-06-14T17:42:15Z (11 months ago)
  • There are several options, like `xargs` and `for`. I'll leave those for other answers and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
  • -----
  • Note that normally, these tools assume 1 file per line (delimited by `
  • `). If you have exotic filenames, such as those with `
  • ` in the name, this won't work right. There are various workarounds for this, for example `find -print0`, `fd --print0` or `parallel --null`. You can find these in the man pages of the tools you use (note that in case of `find`, this requires the GNU variant).
  • I personally prefer a workaround called "don't put newlines in filenames, and if you find any, delete them on sight, uninstall the program that made them, and yell at the developer in the issue tracker".
  • There are several options, like `xargs` and `for`. I'll leave those for other answers and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
  • -----
  • Note that normally, these tools assume 1 file per line (delimited by `
  • `). If you have exotic filenames, such as those with `
  • ` in the name, this won't work right. There are various workarounds for this, for example GNU `find -print0`, `fd --print0` or `parallel --null`. You can find these in the man pages of the tools you use.
  • I personally prefer a workaround called "don't put newlines in filenames, and if you find any, delete them on sight, uninstall the program that made them, and yell at the developer in the issue tracker".
#4: Post edited by user avatar AdminBee‭ · 2023-06-14T17:41:45Z (11 months ago)
Minor clarification, as "print0" is not a POSIX-mandated option for find
  • There are several options, like `xargs` and `for`. I'll leave those for other answers and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
  • -----
  • Note that normally, these tools assume 1 file per line (delimited by `
  • `). If you have exotic filenames, such as those with `
  • ` in the name, this won't work right. There are various workarounds for this, for example `find -print0`, `fd --print0` or `parallel --null`. You can find these in the man pages of the tools you use.
  • I personally prefer a workaround called "don't put newlines in filenames, and if you find any, delete them on sight, uninstall the program that made them, and yell at the developer in the issue tracker".
  • There are several options, like `xargs` and `for`. I'll leave those for other answers and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
  • -----
  • Note that normally, these tools assume 1 file per line (delimited by `
  • `). If you have exotic filenames, such as those with `
  • ` in the name, this won't work right. There are various workarounds for this, for example `find -print0`, `fd --print0` or `parallel --null`. You can find these in the man pages of the tools you use (note that in case of `find`, this requires the GNU variant).
  • I personally prefer a workaround called "don't put newlines in filenames, and if you find any, delete them on sight, uninstall the program that made them, and yell at the developer in the issue tracker".
#3: Post edited by user avatar matthewsnyder‭ · 2023-06-13T17:35:39Z (12 months ago)
  • There are several options, like `xargs` and `for`. I'll leave those for other answers and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
  • There are several options, like `xargs` and `for`. I'll leave those for other answers and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
  • -----
  • Note that normally, these tools assume 1 file per line (delimited by `\n`). If you have exotic filenames, such as those with `\n` in the name, this won't work right. There are various workarounds for this, for example `find -print0`, `fd --print0` or `parallel --null`. You can find these in the man pages of the tools you use.
  • I personally prefer a workaround called "don't put newlines in filenames, and if you find any, delete them on sight, uninstall the program that made them, and yell at the developer in the issue tracker".
#2: Post edited by user avatar matthewsnyder‭ · 2023-06-11T23:30:19Z (12 months ago)
  • There are several options, like `xargs` and `for`. I'll leave those for other questions and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
  • There are several options, like `xargs` and `for`. I'll leave those for other answers and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.
  • Parallel can take a list of files on STDIN and run a command on them.
  • ```
  • # Count lines in each file under current directory using "wc FILENAME"
  • find | parallel wc {}
  • ```
  • Actually, it can also take the list from other sources, like CLI arguments.
  • ```
  • # Apply wc to only txt files in current dir (not subdirs)
  • parallel wc {} ::: *.txt
  • ```
  • It supports various ways of manipulating the filename:
  • ```
  • # Rename foo.JPG files to foo.jpg
  • find | grep '.JPG$' | parallel mv {} {.}.jpg
  • ```
  • As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.
  • Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.
#1: Initial revision by user avatar matthewsnyder‭ · 2023-06-11T23:26:42Z (12 months ago)
There are several options, like `xargs` and `for`. I'll leave those for other questions and only describe my favorite, [GNU Parallel](https://www.gnu.org/software/parallel/). You will have to install it separately, because unlike the inferior `xargs` it usually does not come pre-installed.

Parallel can take a list of files on STDIN and run a command on them. 

```
# Count lines in each file under current directory using "wc FILENAME"
find | parallel wc {}
```

Actually, it can also take the list from other sources, like CLI arguments.

```
# Apply wc to only txt files in current dir (not subdirs)
parallel wc {} ::: *.txt
```

It supports various ways of manipulating the filename:

```
# Rename foo.JPG files to foo.jpg
find | grep '.JPG$' | parallel mv {} {.}.jpg
```

As you can see, you can do any kind of manipulation you want on the file list before you pipe it to Parallel.

Parallel also supports many other features like parallelization, grouping output with color, a dry run mode and so on. See `man parallel` for these.