Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Comments on In a bash shell script, how to filter the command line argument list to unique entries only, for processing each?

Parent

In a bash shell script, how to filter the command line argument list to unique entries only, for processing each?

+11
−0

I have a handful of shell scripts that accept any number of command line arguments, then do some relatively expensive processing based on each command line argument in turn. The general format for these goes along the lines of

#!/bin/bash

# preliminary set-up goes here

# main loop:
while test -n "$1"
do
    do_expensive_processing_for "$1"
    shift
done

# tear-down goes here

This works mostly well. However, it has the downside that if I for some reason pass the same argument twice during an invocation, that argument gets processed twice. Since the processing is expensive, I want to avoid that.

How can I ensure that each command line argument is processed only once during a single invocation of the script, while still allowing arbitrary command line argument contents (or at least not restricting them more than the above type of bash script already would)?

I'm happy with any one instance being the one that gets processed; the order of processing is not important.

To the extent that it matters, I'm using GNU bash 5.1.

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+5
−0

The solution is to use an associative array to store what you've already seen:

#!/bin/bash

unset seen
declare -A seen

for arg in "$@"
do
    if [[ -z "${seen[$arg]}" ]]
    then
        echo "Doing something omplicated with $arg"
        seen["$arg"]=1
    fi
done

The unset seen is just in case the caller had an exported variable named seen. The declare -A seen tells bash to treat seen as an associative array, that is, it takes arbitrary strings as index.

The loop then tests for each argument whether it has not yet been seen (in which case "${seen[arg]}" is empty). If so, it processes it and then marks it as seen by storing something in seen["$arg"] (what it stores doesn't really matter as long as it is not empty; if you want, you can store additional information about the argument here, e.g. the result of processing this argument).

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.

1 comment thread

Works fine, made a slight change (1 comment)
Works fine, made a slight change
Canina‭ wrote about 3 years ago

This works fine. I inverted the logic, though, to avoid having to put pretty much the entire main body of the loop inside an if statement, adding test -z "${processed[$1]}" || { shift; continue; } right near the top of the loop. (A caution to anyone adapting this for their own use: don't forget the shift for each iteration of the loop, however you do it, or the script will get stuck in an endless loop the moment there are duplicate entries.)