Post History

83%

+8 −0

Q&A In a bash shell script, how to filter the command line argument list to unique entries only, for processing each?

Bash Here Bash's associative arrays come handy. The idea is to put every argument as a key in a separate array, and then only process arguments that are not keys to that array. #!/bin/bash decla...

posted 2y ago by Quasímodo‭ · edited 2y ago by Quasímodo‭

Answer

#2: Post edited by

Quasímodo‭ · 2021-09-29T13:26:06Z (over 2 years ago)
Cleanup comments in first script; Add POSIX shell alternative.

Copy Link

Raw

Markdown

Here [Bash's associative arrays](https://www.gnu.org/software/bash/manual/html_node/Arrays.html) come handy. The idea is to put every argument as a key in a separate array, and then only process arguments that are not keys to that array.
```bash
#!/bin/bash
declare -A processed #Declare that "processed" is an associative array
~~for e in "$@"; do #Loop over each argument~~
~~#Test if the argument $e is a key to the "processed" array. If not,~~
if [ -z "${processed["$e"]}" ]; then
~~echo "$e" #Do the expensive processing~~
fi
~~processed["$e"]=1 #Put $e in the "processed" array~~
done
```
Note the above script does not shift its arguments, so if you need the argument list to be empty after the processing is done, you can add a `set --` line. Or use the script below, which has a syntax more along the lines of your sample anyway.
```bash
#!/bin/bash
declare -A processed
while test -n "$1"
do
~~test -z "${processed["$1"]}" && echo "$1"~~
processed["$1"]=1
shift
done
```

### Bash
Here [Bash's associative arrays](https://www.gnu.org/software/bash/manual/html_node/Arrays.html) come handy. The idea is to put every argument as a key in a separate array, and then only process arguments that are not keys to that array.
```bash
#!/bin/bash
declare -A processed #Declare that "processed" is an associative array
for e in "$@"; do #Loop over each argument
if [ -z "${processed["$e"]}" ]; then
echo "Expensively processing $e"
fi
processed["$e"]=1
done
```
Note the above script does not shift its arguments, so if you need the argument list to be empty after the processing is done, you can add a `set --` line. Or use the script below, which has a syntax more along the lines of your sample anyway.
```bash
#!/bin/bash
declare -A processed
while test -n "$1"
do
test -z "${processed["$1"]}" && echo "Expensively processing $1"
processed["$1"]=1
shift
done
```
### POSIX shell
Since you don't require the order to be preserved....
- Pop the head of the argument list.
- Look at the remainder of the list for a duplicate of the
decapitated head. If found,
- Suppress running the expensive command on the head.
```sh
#!/bin/sh
while [ -n "$1" ]; do
unset suppress
head=$1
shift
for e in "$@"; do
if [ "$head" = "$e" ]; then
suppress=1
break
fi
done
[ -n "$suppress" ] || echo "Expensively processing $head"
done
```

#1: Initial revision by

Quasímodo‭ · 2021-09-28T17:39:54Z (over 2 years ago)

Copy Link

Raw

Markdown

Here [Bash's associative arrays](https://www.gnu.org/software/bash/manual/html_node/Arrays.html) come handy. The idea is to put every argument as a key in a separate array, and then only process arguments that are not keys to that array.

```bash
#!/bin/bash
declare -A processed  #Declare that "processed" is an associative array
for e in "$@"; do  #Loop over each argument
    #Test if the argument $e is a key to the "processed" array. If not,
    if [ -z "${processed["$e"]}" ]; then
        echo "$e" #Do the expensive processing
    fi
    processed["$e"]=1 #Put $e in the "processed" array
done
```

Note the above script does not shift its arguments, so if you need the argument list to be empty after the processing is done, you can add a `set --` line. Or use the script below, which has a syntax more along the lines of your sample anyway.

```bash
#!/bin/bash
declare -A processed
while test -n "$1"
do
    test -z "${processed["$1"]}" && echo "$1"
    processed["$1"]=1
    shift
done
```

Communities

Post History