Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Comments on In a bash shell script, how to filter the command line argument list to unique entries only, for processing each?

Parent

In a bash shell script, how to filter the command line argument list to unique entries only, for processing each?

+11
−0

I have a handful of shell scripts that accept any number of command line arguments, then do some relatively expensive processing based on each command line argument in turn. The general format for these goes along the lines of

#!/bin/bash

# preliminary set-up goes here

# main loop:
while test -n "$1"
do
    do_expensive_processing_for "$1"
    shift
done

# tear-down goes here

This works mostly well. However, it has the downside that if I for some reason pass the same argument twice during an invocation, that argument gets processed twice. Since the processing is expensive, I want to avoid that.

How can I ensure that each command line argument is processed only once during a single invocation of the script, while still allowing arbitrary command line argument contents (or at least not restricting them more than the above type of bash script already would)?

I'm happy with any one instance being the one that gets processed; the order of processing is not important.

To the extent that it matters, I'm using GNU bash 5.1.

History
Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+8
−0

Bash

Here Bash's associative arrays come handy. The idea is to put every argument as a key in a separate array, and then only process arguments that are not keys to that array.

#!/bin/bash
declare -A processed  #Declare that "processed" is an associative array
for e in "$@"; do     #Loop over each argument
    if [ -z "${processed["$e"]}" ]; then
        echo "Expensively processing $e"
    fi
    processed["$e"]=1
done

Note the above script does not shift its arguments, so if you need the argument list to be empty after the processing is done, you can add a set -- line. Or use the script below, which has a syntax more along the lines of your sample anyway.

#!/bin/bash
declare -A processed
while test -n "$1"
do
    test -z "${processed["$1"]}" && echo "Expensively processing $1"
    processed["$1"]=1
    shift
done

POSIX shell

Since you don't require the order to be preserved....

  • Pop the head of the argument list.
  • Look at the remainder of the list for a duplicate of the decapitated head. If found,
    • Suppress running the expensive command on the head.
#!/bin/sh
while [ -n "$1" ]; do
    unset suppress
    head=$1
    shift
    for e in "$@"; do
        if [ "$head" = "$e" ]; then
            suppress=1
            break
        fi
    done
    [ -n "$suppress" ] || echo "Expensively processing $head"
done
History
Why does this post require moderator attention?
You might want to add some details to your flag.

1 comment thread

Works fine, made a slight change (1 comment)
Works fine, made a slight change
Canina‭ wrote over 2 years ago

The middle example works fine. I inverted the logic, though, to avoid having to put pretty much the entire main body of the loop inside an if statement, adding test -z "${processed[$1]}" || { shift; continue; } right near the top of the loop. (A caution to anyone adapting this for their own use: don't forget the shift for each iteration of the loop, however you do it, or the script will get stuck in an endless loop the moment there are duplicate entries.)