Commands like the AWS CLI may return a list of values all in one line, where each item in the list is separated by the nearby items with spaces. Using a plain read command doesn’t really work: read will read all the values in one go into the variable. You need to change the delimiter that read uses to split the input. No need to pipe the output through Perl or other tools, read got you covered with the -d option.
In this example I get the list of the ARNs of all target groups in an AWS account, and then iterate over those ARNs to list all the instances in each target group. The ouput will also be saved into a file through the tee command:
aws elbv2 describe-target-groups \
--query 'TargetGroups[].TargetGroupArn' \
--output text | \
while read -d ' ' ARN ; do \
echo -n "$ARN: " ; \
aws elbv2 describe-target-health \
--target-group-arn "$ARN" \
--query 'TargetHealthDescriptions[].Target.Id' \
--output text ; sleep 1 ; \
done | \
tee tg-instances.txt
The ouput of this one liner will be in the format:
ARN: instance_ID [instance_ID...]
Things to notice:
- the AWS CLI’s
describe-target-groupscommand will list all target groups’ ARNs thanks to the--queryoption and list as many as possible on single lines, according to the shell’s output buffer capacity; the ouput is piped through awhileloop; - the while loop uses
read -d ' 'to split each line at spaces and save each item in the$ARNvariable, one per cycle; - the
echocommand prints the value of$ARNfollowed by a colon, a space, but will not output a newline sequence due to the-noption; - the AWS CLI’s
describe-target-healthcommand will list all target IDs thanks to the--queryoption and print them out in a single line; it will also provide a newline sequence, so that the next loop will start on a new line; - the
sleep 1command slows down the loop, so that we don’t hammer the API to the point that they will rate limit us; - finally, the
teecommand will duplicate the output of the while loop to both the standard output and the filetg-instances.txt.
This is mostly a note to self. When I need an EC2 instance to run a quick test, it may be overly annoying to provision one through the web console, or it may feel a bit overkill to do that using large frameworks like terraform. Using the AWS command line is just fine, if you know what command to run with which parameters, and it pays off quickly if, to run your tests, you use the settings often (AMI, subnet, security groups…) or if during the same test session you need to scrap and rebuild test instances a few times. Here is an example on how to do so with the AWS command line client.
A sysadmin's logbook 