News HashiCorp to join IBM to accelerate multi-cloud automation. Read the blog →
Boundary
Host management

Dynamic host catalogs on AWS

  • 20min
  • |
  • Boundary

Dynamic updates to host catalogs is an important differentiator for Boundary from traditional access methods that rely on manual target configuration, and enables even tighter integrations with major cloud providers for seamlessly onboarding cloud tenant identities, roles, and targets.

Boundary 0.7 introduces a key Boundary component by enabling automated discovery of target hosts and services. Starting with support for Azure and AWS, dynamic connections to any service registry ensures that hosts and host catalogs are consistently up-to-date. This critical workflow offers access-on-demand and eliminates the need to manually configure targets for dynamic, cloud-based infrastructure.

This tutorial demonstrates configuring a dynamic host catalog using Amazon Web Services.

Dynamic hosts catalog overview

  1. Get setup
  2. Dynamic host catalogs background
  3. Set up cloud hosts
  4. Build a host catalog
  5. Verify catalog membership

Prerequisites

Get setup

In this tutorial, you will test dynamic host catalog integrations using HCP Boundary or by running a Boundary controller locally using dev mode.

The HCP Quickstart tutorials provide an overview of getting started with an HCP Boundary cluster.

If you have an HCP Boundary cluster deployed, the Access HCP Boundary tutorial provides an overview of configuring your local machine to authenticate with your HCP cluster.

This tutorial provides both CLI and UI workflows for setting up Dynamic updates to host catalogs on AWS.

To proceed with the CLI workflow:

  1. Log in to the Boundary web UI and copy your org ID by clicking the copy icon.

    Open a terminal session and set a environment variable for the org ID.

    $ export ORG_ID=<org-id>

  2. In the Boundary web UI, click Orgs in the left navigation menu to return to the global scope, and then click Auth Methods.

    Click the copy icon for the Password auth method.

    In your terminal set an environment variable named BOUNDARY_AUTH_METHOD_ID to the copied ID.

    $ export BOUNDARY_AUTH_METHOD_ID=<auth-id>

  3. Click Projects in the left navigation and then click the project card.

    Click the copy icon for the project.

    In your terminal set an environment variable named PROJECT_ID to the copied ID.

    $ export PROJECT_ID=<project-id>

    Close the Boundary web UI.

  4. Return to the HCP web Portal Boundary page, then click the copy icon for the Cluster URL in the Getting started with Boundary section.

    In your terminal, set the BOUNDARY_ADDR environment variable to the copied URL.

    $ export BOUNDARY_ADDR=<actual-boundary-address>

  5. Log in with the administrator credentials you created when you deployed the HCP Boundary instance. Ensure that the BOUNDARY_ADDR and BOUNDARY_AUTH_METHOD_ID environment variables are set, then run boundary authenticate. Enter your password at the Please enter the password (it will be hidden): prompt.

    $ boundary authenticatePlease enter the login name (it will be hidden):Please enter the password (it will be hidden):Authentication information:  Account ID:      acctpw_VOeNSFX8pQ  Auth Method ID:  ampw_wxzojlKJLN  Expiration Time: Mon, 13 Feb 2023 12:35:32 MST  User ID:         u_1vUkf5fPs9The token was successfully stored in the chosen keyring and is not displayed here.

You are now logged into your HCP Boundary instance's Global scope via the CLI. This is the default scope for all new Boundary clusters.

Dynamic host catalogs background

In a cloud operating model infrastructure resources are highly dynamic and ephemeral. Boundary lacks an on-target agent or daemon, and as a result cannot recognize when a host service migrates or is redeployed. Instead, Boundary relies on an external entity, such as manual configuration by an administrator or IaC application (such as Terraform), to ensure host definitions route to the appropriate network location. This is a pattern followed by many other secure access solutions.

Dynamic host catalog plugins provide an alternative mechanism for automating the discovery and configuration of Boundary hosts and targets by delegating the host registry and their connection information to a cloud infrastructure provider. Administrators provide credentials to the catalog provider and a set of tag-based rules for discovery resources in the catalog. For example, "this catalog contains all EC2 instance types in AWS’s US-West-2 region within the marketing subscription". This model differs from other that rely on either IaC target discovery or agent-based target discovery.

To accommodate this model, Boundary 0.7.0 introduces plugin support via Go-Plugin for expanding the dynamic host catalog ecosystem. While this initial release of Boundary plug-ins are limited to dynamic host catalogs, plugins enable a future ecosystem of partner and community contributed integrations across each step in the Boundary access workflow.

Host tag filtering

To successfully maintain a dynamic host catalog, hosts should be tagged in a logical way that enables sorting into the appropriate host sets identifiable by filters.

For example, this tutorial configures hosts on AWS using the following tags:

Hosts tagged accordingly will be sorted into any host catalogs and host sets configured using these filtering attributes.

Set up cloud hosts

Warning

This tutorial deploys cloud machines to test host catalog plugin configuration. You are responsible for any costs incurred by following the steps in this tutorial. Recommendations for destroying the associated cloud resources are detailed in the Cleanup and teardown section.

An Amazon Web Services test account is required to set up the Boundary AWS host plugin.

This tutorial enables configuration of the test hosts using the AWS CLI, Terraform, or the AWS Console UI.

An AWS account and sample application are required to setup the AWS hosts plugin for Boundary. If you don't have an account, sign up for AWS. A free account is suitable for the steps outlined in this tutorial, but please note that you are responsible for any charges incurred by following the steps in this tutorial.

This tutorial sets up permissions and hosts for host catalogs using the AWS CLI and CloudFormation.

There are four tasks necessary to set up hosts using the AWS CLI.

  1. Deploy and tag the host set members appropriately
  2. Configure user permissions for Boundary
  3. Assign ec2:DescribeInstances IAM privileges to the configured IAM user (may be skipped)
  4. Generate a Secret Access Key for the IAM user, enabling Boundary to authenticate and access the tagged hosts

First, ensure that the AWS CLI is configured correct with your account credentials. You will need the AWS Access Key, Secret Access Key, and (if needed) a Session Token.

Common ways to get set up include:

  • Execute aws configure and pass the access values
  • Export the access values as environment variables
  • Configure the AWS credentials file

For more information on setting up the AWS CLI to interact with AWS, check the Configuring the AWS CLI documentation.

Create hosts

This tutorial deploys tagged hosts to test the dynamic host catalog integration.

To easily deploy a set of pre-configured hosts, a CloudFormation template file is available for use with AWS. CloudFormation is used to easily deploy the EC2 instances in this tutorial.

Clone down the sample template repository.

$ git clone https://github.com/hashicorp-education/learn-boundary-cloud-host-catalogs

Navigate into the aws directory.

$ cd learn-boundary-cloud-host-catalogs/aws/

The provided aws-dynamic-hosts.json template will be used to deploy a CloudFormation resource stack that contains the hosts to be included in our Boundary host catalog.

This tutorial uses the us-east-1a availability zone to deploy the EC2 instances, but you can use any region you want. To change the region, open the aws-dynamic-hosts.json file and update the AvailabilityZone.

Next, create a keypair to be used for the instances. Alternatively, you may open the aws-dynamic-hosts.json file and update the boundary-keypair to match the name of an existing EC2 keypair.

$ aws ec2 create-key-pair \    --key-name boundary-keypair \    --key-type rsa \    --query "KeyMaterial" \    --output text > boundary-keypair.pem

Note

The boundary-keypair.pem file was created within your working directory. This tutorial does not log into the associated VMs, but it is best practice to retain the private key for the duration of the tutorial. The tutorial deletes this keypair in the Cleanup and teardown section. If you choose to keep this keypair, consider moving it into another directory like ~/.aws/. Do not check this key into source control.

Ensure you are within the learn-boundary-cloud-host-catalogs/aws/ directory where the aws-dynamic-hosts.json file is located.

Deploy a new resource stack using the provided aws-dynamic-hosts.json template.

$ aws cloudformation create-stack \  --stack-name boundary-dynamic-hosts \  --template-body file://./aws-dynamic-hosts.json

Example output:

$ aws cloudformation create-stack \  --stack-name boundary-dynamic-hosts \  --template-body file://./aws-dynamic-hosts.json{    "StackId": "arn:aws:cloudformation:us-east-1:157470686136:stack/boundary-dynamic-hosts/c29af940-78b6-11ec-849d-12bc7613d5b9"}

The deployment may take a few minutes.

Check that the stack was successfully created.

$ aws cloudformation list-stacks{    "StackSummaries": [        {            "StackId": "arn:aws:cloudformation:us-east-1:157470686136:stack/boundary-dynamic-hosts/da9ebb30-78a7-11ec-bb47-0eddcb3d6bf3",            "StackName": "boundary-dynamic-hosts",            "TemplateDescription": "AWS CloudFormation template for Boundary Dynamic Hosts tutorial. Deploying this template will incur costs to your AWS account.",            "CreationTime": "2022-01-18T22:43:11.249000+00:00",            "StackStatus": "CREATE_COMPLETE",            "DriftInformation": {                "StackDriftStatus": "NOT_CHECKED"            }        }    ]}

Lastly, verify that the hosts were successfully created by printing their "Name" tag using aws ec2 describe-instances. The following command will only print running instances, and will also report the availability zone and instance ID in table format.

$ aws ec2 describe-instances \  --output table \  --filters Name=instance-state-name,Values=running \  --filters Name=tag-key,Values=Name \  --query 'Reservations[*].Instances[*].{Instance:InstanceId,AZ:Placement.AvailabilityZone,Name:Tags[?Key==`Name`]|[0].Value}'
$ aws ec2 describe-instances \  --output table \  --filters Name=instance-state-name,Values=running \  --filters Name=tag-key,Values=Name \  --query 'Reservations[*].Instances[*].{Instance:InstanceId,AZ:Placement.AvailabilityZone,Name:Tags[?Key==`Name`]|[0].Value}'----------------------------------------------------------------|                       DescribeInstances                      |+------------+-----------------------+-------------------------+|     AZ     |       Instance        |          Name           |+------------+-----------------------+-------------------------+|  us-east-1a|  i-06f5a1240a2e0c3fb  |  boundary-3-production  ||  us-east-1a|  i-0657b0de7662f9863  |  boundary-1-dev         ||  us-east-1a|  i-0ac5cfcae66bdacf5  |  boundary-2-dev         ||  us-east-1a|  i-0b6d3ad435c586783  |  boundary-4-production  |+------------+-----------------------+-------------------------+

Configure an IAM user

Usually, you might configure an IAM user with the correct policies assigned for keeping the dynamic host catalog up-to-date. You may also use an existing IAM user and assign it the policy, or an IAM instance profile.

This tutorial demonstrates creating a new IAM user, but the tutorial can also be continued using root credentials. To continue using root credentials, skip to the Gather plugin details section.

Next, create a new IAM user for Boundary.

$ aws iam create-user --user-name boundary{    "User": {        "Path": "/",        "UserName": "boundary",        "UserId": "AIDASWVU2XLZFFIP6P4IN",        "Arn": "arn:aws:iam::157470686136:user/boundary",        "CreateDate": "2022-01-12T19:35:32+00:00"    }}

Create a new file called boundary-describe-instances-policy.json, and fill it with the following policy:

boundary-describe-instances-policy.json

{  "Version": "2012-10-17",  "Statement": [    {      "Action": [        "ec2:DescribeInstances"      ],      "Effect": "Allow",      "Resource": "*"    }  ]}

This policy allows the boundary IAM user to run the DescribeInstances API call, similar to running the aws ec2 describe-instances command using the CLI. Boundary will be able to list these details, including the host's tags. This will allow Boundary to sort hosts into their appropriate catalogs.

Next, attach this as an inline policy to the boundary user, giving it the name BoundaryDescribeInstances. This command assume the json policy file is located in the same directory the command is executed from.

$ aws iam put-user-policy \  --user-name boundary \  --policy-name BoundaryDescribeInstances \  --policy-document file://./boundary-describe-instances-policy.json

Check your work by listing the policies attached to the user.

$ aws iam get-user-policy --user-name boundary --policy-name BoundaryDescribeInstances{    "UserName": "boundary",    "PolicyName": "BoundaryDescribeInstances",    "PolicyDocument": {        "Version": "2012-10-17",        "Statement": [            {                "Action": [                    "ec2:DescribeInstances"                ],                "Effect": "Allow",                "Resource": "*"            }        ]    }}

The last step is to create an access key for the user.

$ aws iam create-access-key --user-name boundary{    "AccessKey": {        "UserName": "boundary",        "AccessKeyId": "AKIASWVU2XLZLFLIDMVW",        "Status": "Active",        "SecretAccessKey": "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",        "CreateDate": "2022-01-19T02:26:11+00:00"    }}

Keep the AccessKeyId and SecretAccessKey in a safe location for the remainder of this tutorial.

To get ready to configure Boundary, export these values as environment variables.

$ export BOUNDARY_ACCESS_KEY_ID=<AWS Access Key ID> \  export BOUNDARY_SECRET_ACCESS_KEY=<AWS Secret Access Key>

Note

This tutorial disables credential rotation for the host catalog. To learn more about the required IAM permissions for enabling credential rotation, refer to the boundary-plugin-host-aws documentation.

Gather plugin details

To use the Boundary AWS hosts plugin, the following details must be available for a user that has access to the DescribeInstances API.

  • AWS Access Key ID
  • AWS Secret Access Key

These values should be available as environment variables within the terminal session, or copied to a safe location for use when setting up Boundary. This tutorial created the IAM user boundary and exported these values as environment variables.

Note

If using root credentials instead of creating an IAM user, you should export these values as the environment variables defined below. These values will be passed to Boundary when creating the host catalog later on.

$ export BOUNDARY_ACCESS_KEY_ID=<AWS Access Key ID> \  export BOUNDARY_SECRET_ACCESS_KEY=<AWS Secret Access Key>

Host catalog plugins

For Boundary, the process for creating a dynamic host catalog has two steps:

  1. Create a plugin-type host catalog
  2. Create a host set that defines membership using filters

A plugin-type host catalog can be created using some cloud provider resource details, and the host set is then defined using a filter that selects hosts for membership based on the tags defined when setting up the hosts.

Host set filter expressions are defined by the plugin provider, in this case AWS. The AWS plugin uses simple filter queries to specify tags associated with hosts based on tag:Name=Value.

For example, a host set filter that selects all hosts tagged with "service-type": "database" is written as:

Resources within AWS can generally be filtered by tag names and values, and filters can be designed to use either/or selectors for tag values. This process is described in the Boundary AWS Host Plugin documentation.

To learn more about AWS filters for listing resources, visit the describe-instances CLI documentation page and the Describe Instances AWS EC2 API docs.

Build a host catalog

With the cloud provider details gathered, a plugin host catalog can now be created that will contain the respective host sets for the database and application filters.

Create a host catalog plugin

Check that the AWS parameter environment variable values were set correctly in your session.

$ echo $BOUNDARY_ACCESS_KEY_ID; echo $BOUNDARY_SECRET_ACCESS_KEYAKIASWVU2XLZLFLIDMVW8BnyuNv7egZG9/k/+d79JGLoJXcqXGEZiUPEcx0O

Note

If you proceeded without setting up the boundary IAM user, expand the accordion to proceed.

Boundary simply requires user credentials that can execute the aws ec2 describe-instances command.

To proceed with different user credentials (such as root account credentials for testing purposes) simply export these values and provide them to Boundary in the following section. These values may already be available in your terminal session as the $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY environment variables.

$ export BOUNDARY_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID>
$ export BOUNDARY_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY>

If necessary, authenticate to Boundary as the admin user.

$ boundary authenticatePlease enter the login name (it will be hidden):Please enter the password (it will be hidden):Authentication information:  Account ID:      acctpw_VOeNSFX8pQ  Auth Method ID:  ampw_wxzojlKJLN  Expiration Time: Mon, 13 Feb 2023 12:35:32 MST  User ID:         u_1vUkf5fPs9The token was successfully stored in the chosen keyring and is not displayed here.

Next, create a new plugin-type host catalog with a -plugin-name of aws, providing the Boundary Access Key ID and Boundary Secret Access Key using the -secret flag. These values should map to the environment variables defined above. Additionally, ensure disable_credential_rotation and region are set using the -attr flag.

$ boundary host-catalogs create plugin \  -scope-id $PROJECT_ID \  -plugin-name aws \  -attr disable_credential_rotation=true \  -attr region=us-east-1 \  -secret access_key_id=env://BOUNDARY_ACCESS_KEY_ID \  -secret secret_access_key=env://BOUNDARY_SECRET_ACCESS_KEY

Command flags:

  • -plugin-name: This corresponds to the host catalog plugin's name, such as azure or aws
  • disable_credential_rotation: This tutorial utilizes a static secret by setting this value to true
  • region: The region to configure the host catalog for. All host sets in this catalog will be configured for this region
  • access_key_id: The access key ID for the IAM user to use with this host catalog
  • secret_access_key: The secret access key for the IAM user to use with this host catalog

Note

Although credentials are stored encrypted within Boundary, by default this plugin attempts to rotate credentials supplied through the secrets object during a create or update call to the host catalog resource. The given credentials will be used to create a new credential, and then the given credential will be revoked. In this way, after rotation, only Boundary knows the client secret in use by this plugin. Credential rotation will be generally available in a future release of Boundary.

Sample output:

$ boundary host-catalogs create plugin \  -scope-id p_1234567890 \  -plugin-name aws \  -attr disable_credential_rotation=true \  -attr region=us-east-1 \  -secret access_key_id=env://BOUNDARY_ACCESS_KEY_ID \  -secret secret_access_key=env://BOUNDARY_SECRET_ACCESS_KEY Host Catalog information:  Created Time:                  Tue, 18 Jul 2023 10:08:12 MDT  ID:                            hcplg_N9p1Woaq8l  Plugin ID:                     pl_1PRTDb78iY  Secrets HMAC:                  3AcXDqAEALitc7WKM2KUw8dSUVEyYG2391x1DZZCS32z  Type:                          plugin  Updated Time:                  Tue, 18 Jul 2023 10:08:12 MDT  Version:                       1   Scope:    ID:                          p_1234567890    Name:                        Generated project scope    Parent Scope ID:             o_1234567890    Type:                        project   Plugin:    ID:                          pl_1PRTDb78iY    Name:                        aws   Attributes:    disable_credential_rotation: true    region:                      us-east-1   Authorized Actions:    no-op    read    update    delete   Authorized Actions on Host Catalog's Collections:    host-sets:      create      list    hosts:      list

Copy the host catalog ID from the output (hcplg_N9p1Woaq8l in this example) and store it in the HOST_CATALOG_ID environment variable. 

$ export HOST_CATALOG_ID=hcplg_N9p1Woaq8l

Create the host sets

With the dynamic host catalog created, host sets can now be defined that correspond to the service-type and application tags added to the hosts.

Recall the three host sets we wish to create:

  • All hosts with a service-type tag of database
  • All hosts with an application tag of dev
  • All hosts with an application tag of production

The respective host set filters can be constructed as:

Create the first plugin host set containing hosts tagged with a service-type of database, supplying the host catalog ID copied above and the needed filter using the -attr flag.

$ boundary host-sets create plugin \  -name database \  -host-catalog-id $HOST_CATALOG_ID \  -attr filters=tag:service-type=database

Sample output:

$ boundary host-sets create plugin \  -name database \  -host-catalog-id $HOST_CATALOG_ID \  -attr filters=tag:service-type=database Host Set information:  Created Time:        Tue, 18 Jul 2023 10:20:19 MDT  Host Catalog ID:     hcplg_N9p1Woaq8l  ID:                  hsplg_sSCs67KYGD  Name:                database  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:20:19 MDT  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_5pQekoPRJt    Name:              aws   Attributes:    filters:           tag:service-type=database   Authorized Actions:    no-op    read    update    delete

Copy the database host set ID from the output (hsplg_sSCs67KYGD in this example) and store it in the DATABASE_HOST_SET_ID environment variable. 

$ export DATABASE_HOST_SET_ID=hsplg_sSCs67KYGD

Wait a moment, then list all available hosts within the aws host catalog, which contains the newly created database host set.

Note

It may take up to five minutes for the host catalog to sync with the cloud provider.

$ boundary hosts list -host-catalog-id $HOST_CATALOG_ID Host information:  ID:                    hplg_AAdFY3zsdP    External ID:         i-0c482cd40f62309e2    Version:             1    Type:                plugin    Authorized Actions:      no-op      read   ID:                    hplg_6PlZJpEzgx    External ID:         i-0767b38ec388cf165    Version:             1    Type:                plugin    Authorized Actions:      no-op      read   ID:                    hplg_7KjFRrcLxD    External ID:         i-08c588f2d58120192    Version:             1    Type:                plugin    Authorized Actions:      no-op      read   ID:                    hplg_FQzmhOmDqn    External ID:         i-0dac079e5f8760b9c    Version:             1    Type:                plugin    Authorized Actions:      no-op      read

Troubleshooting

If the boundary hosts list command returns No hosts found, expand the accordion below to check your work.

If the host catalog is misconfigured, hosts will not be discoverable by Boundary. There are three issues to check:

  1. The host set filter is defined correctly
  2. The host catalog and host set ID's are exported correctly as environment variables
  3. The IAM profile assigning DescribeInstances permissions has been assigned to the IAM boundary user

Note

Depending on the type of configuration issue, you will need to wait approximately 5 - 10 minutes for the existing host catalog or host sets to sync with the provider and refresh their values. If you do not want to wait a new host catalog and host set can be created from scratch, but these will also take several minutes to sync upon creation.

If incorrect, update the managed group filter. This process can also be used to update the managed group filter criteria in the future for any existing managed groups.

First, check the environment variables defined when creating a host catalog plugin. Ensure these are the correct values gathered when setting up the cloud hosts.

If these are incorrectly defined, set the environment variables again, and update the host catalog:

$ boundary host-catalogs update plugin \  -id $HOST_CATALOG_ID \  -plugin-name aws \  -attr disable_credential_rotation=true \  -attr region=us-east-1 \  -secret access_key_id=env://BOUNDARY_ACCESS_KEY_ID \  -secret secret_access_key=env://BOUNDARY_SECRET_ACCESS_KEY

Second, check is the DescribeInstances profile was assigned to the boundary IAM user. If incorrect permissions are assigned or the wrong user is defined, Boundary will not be able to view the hosts.

Review the steps for configuring an IAM user.

After correcting the profile, give Boundary up to five minutes to refresh the connection to AWS, and list the available hosts again.

Now create a host set that correspond to the application tag of dev.

$ boundary host-sets create plugin \  -name dev \  -host-catalog-id $HOST_CATALOG_ID \  -attr filters=tag:application=dev

Sample output:

$ boundary host-sets create plugin \  -name dev \  -host-catalog-id $HOST_CATALOG_ID \  -attr filters=tag:application=dev Host Set information:  Created Time:        Tue, 18 Jul 2023 10:20:19 MDT  Host Catalog ID:     hcplg_Ia7R4E39oF  ID:                  hsplg_yG2pSNlbTM  Name:                dev  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:20:19 MDT  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Attributes:    filters:           tag:application=dev   Authorized Actions:    no-op    read    update    delete

Copy the dev host set ID from the output (hsplg_yG2pSNlbTM in this example) and store it in the DEV_HOST_SET_ID environment variable. 

$ export DEV_HOST_SET_ID=hsplg_yG2pSNlbTM

Lastly, create a host set that correspond to the application tag of production.

$ boundary host-sets create plugin \  -name production \  -host-catalog-id $HOST_CATALOG_ID \  -attr filters=tag:application=production

Sample output:

$ boundary host-sets create plugin \  -name production \  -host-catalog-id $HOST_CATALOG_ID \  -attr filters=tag:application=production Host Set information:  Created Time:        Tue, 18 Jul 2023 10:20:19 MDT  Host Catalog ID:     hcplg_Ia7R4E39oF  ID:                  hsplg_ZmoClk4HiD  Name:                production  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:20:19 MDT  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Attributes:    filters:           tag:application=production   Authorized Actions:    no-op    read    update    delete

Copy the production host set ID from the output (hsplg_ZmoClk4HiD in this example) and store it in the PRODUCTION_HOST_SET_ID environment variable. 

$ export PRODUCTION_HOST_SET_ID=hsplg_ZmoClk4HiD

Verify catalog membership

With the database, dev, and prod host sets defined within the aws host catalog, the next step is to verify that the four instances listed as members of the catalog are dynamically included in the correct host sets.

Host membership can be verified by reading the host set details and verifying its membership IDs.

First, verify that the database host set contains all four members of the aws host catalog.

Perform a read on the host set named database to view its members.

$ boundary host-sets read -id $DATABASE_HOST_SET_ID Host Set information:  Created Time:        Tue, 18 Jul 2023 10:20:19 MDT  Host Catalog ID:     hcplg_N9p1Woaq8l  ID:                  hsplg_sSCs67KYGD  Name:                database  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:20:19 MDT  Version:             3   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Attributes:    filters:           tag:service-type=database   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_4JwY7y6il2    hplg_6PlZJpEzgx    hplg_7KjFRrcLxD    hplg_FQzmhOmDqn

If the Host IDs section is missing, expand the troubleshooting accordion to diagnose what could be wrong.

If the host catalog is misconfigured, hosts will not be discoverable by Boundary.

At this point in the tutorial hosts are contained within the host catalog, but not appearing in one or more host sets. This implies that the host set itself is misconfigured.

Above, you performed a read on the database host set. Check the Attributes section, and verify it matches the correctly defined filter:

  Attributes:    filters:           tag:service-type=database

If the tag is incorrectly assigned, perform an update on the affected host set to fix the filter:

$ boundary host-sets update plugin \  -id $DATABASE_HOST_SET_ID \  -name production \  -attr filters=tag:application=production

After updating the filter, Boundary will automatically refresh the host set.

Note

Depending on the type of configuration issue, you will need to wait approximately 5 - 10 minutes for the existing host catalog or host sets to sync with the provider and refresh their values. If you do not want to wait a new host catalog and host set can be created from scratch, but these will also take several minutes to sync upon creation.

Check that the updated filter is working by performing another read on the database host set.

$ boundary host-sets read -id $DATABASE_HOST_SET_ID

If the dev or production host sets are affected by incorrect filters, follow the same procedure to update their filters accordingly.

Next, read the dev host set details. Verify the Host IDs are the correctly tagged hosts from the cloud provider.

$ boundary host-sets read -id $DEV_HOST_SET_ID Host Set information:  Created Time:        Tue, 18 Jul 2023 10:27:29 MDT  Host Catalog ID:     hcplg_Ia7R4E39oF  ID:                  hsplg_yG2pSNlbTM  Name:                dev  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:27:29 MDT  Version:             2   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Attributes:    filters:           tag:application=dev   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_6PlZJpEzgx    hplg_FQzmhOmDqn

Notice the Host IDs section of the output, which returns the two dev instances configured in AWS.

Next, read the production host set and verify its Host IDs.

$ boundary host-sets read -id $PRODUCTION_HOST_SET_ID Host Set information:  Created Time:        Tue, 18 Jul 2023 10:31:46 MDT  Host Catalog ID:     hcplg_Ia7R4E39oF  ID:                  hsplg_ZmoClk4HiD  Name:                production  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:31:46 MDT  Version:             2   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Attributes:    filters:           tag:application=production   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_4JwY7y6il2

Notice the Host IDs section of this output. Even though there are two production instances, only one is listed in the host set.

To figure out what could be wrong, compare the members of the production host set to the members of the database host set. Remember, members of the production and dev host sets are a sub-set of the database host set.

$ boundary host-sets read -id $DATABASE_HOST_SET_ID Host Set information:  Created Time:        Tue, 18 Jul 2023 10:35:54 MDT  Host Catalog ID:     hcplg_Ia7R4E39oF  ID:                  hsplg_qnQM03I6yT  Name:                database  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:35:54 MDT  Version:             3   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Attributes:    filters:           tag:service-type=database   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_4JwY7y6il2    hplg_6PlZJpEzgx    hplg_7KjFRrcLxD    hplg_FQzmhOmDqn

By comparing the Host IDs of the dev host catalog to the production catalog, notice that host hplg_7KjFRrcLxD is missing from the production host set, although it is contained within database.

Update the misconfigured host

Perform a read on the missing host.

$ boundary hosts read -id hplg_7KjFRrcLxD Host information:  Created Time:        Tue, 18 Jul 2023 10:37:22 MDT  External ID:         i-08c588f2d58120192  Host Catalog ID:     hcplg_Ia7R4E39oF  ID:                  hplg_7KjFRrcLxD  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:37:22 MDT  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Authorized Actions:    no-op    read   Host Set IDs:    hsplg_qnQM03I6yT   IP Addresses:    172.31.71.39    3.239.215.108   DNS Names:    ec2-3-239-215-108.compute-1.amazonaws.com    ip-172-31-71-39.ec2.internal

The External ID: field shows the ID of the misconfigured host (i-08c588f2d58120192 in this example). Copy this value.

Recall that host set membership is defined based on the instance tags.

Describe the instance's details, and query for its tag values.

$ aws ec2 describe-instances \  --instance-ids i-08c588f2d58120192 \  --output table \  --query 'Reservations[*].Instances[*].Tags[?contains(Key, `aws`) == `false`]'

Sample output:

$ aws ec2 describe-instances \  --instance-ids i-08c588f2d58120192 \  --output table \  --query 'Reservations[*].Instances[*].Tags[?contains(Key, `aws`) == `false`]' -------------------------------------------|            DescribeInstances            |+---------------+-------------------------+|      Key      |          Value          |+---------------+-------------------------+|  service-type |  database               ||  Name         |  boundary-4-production  ||  application  |  prod                   |+---------------+-------------------------+

Notice that the application tag is misconfigured as prod, instead of production. An easy mistake to make!

Remember the filter defined for the production host set:

tag:application=production

The tag's value must equal production exactly to be included in this host set.

Update the application tag to production for the misconfigured instance using the aws ec2 create-tags command, which will overwrite the existing tag value.

$ aws ec2 create-tags --resources i-08c588f2d58120192 --tags Key=application,Value=production

Re-run the describe-instances command to directly query for the updated tag values.

Sample output:

$ aws ec2 describe-instances \  --instance-ids i-08c588f2d58120192 \  --output table \  --query 'Reservations[*].Instances[*].Tags[?contains(Key, `aws`) == `false`]' -------------------------------------------|            DescribeInstances            |+---------------+-------------------------+|      Key      |          Value          |+---------------+-------------------------+|  application  |  production             ||  service-type |  database               ||  Name         |  boundary-4-production  |+---------------+-------------------------+

Boundary will update the production host set automatically the next time it refreshes. This process could take up to ten minutes.

After waiting, read the production host set again and verify that its Host IDs contain the updated host as a member.

$ boundary host-sets read -id $PRODUCTION_HOST_SET_ID Host Set information:  Created Time:        Tue, 18 Jul 2023 10:52:31 MDT  Host Catalog ID:     hcplg_Ia7R4E39oF  ID:                  hsplg_ZmoClk4HiD  Name:                production  Type:                plugin  Updated Time:        Tue, 18 Jul 2023 10:52:31 MDT  Version:             6   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_1PRTDb78iY    Name:              aws   Attributes:    filters:           tag:application=production   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_4JwY7y6il2    hplg_7KjFRrcLxD

Cleanup and teardown

  1. Delete the CloudFormation boundary-dynamic-hosts stack

    $ aws cloudformation delete-stack --stack-name boundary-dynamic-hosts

    Wait a minute, then check that the StackStatus has changed to DELETE_COMPLETE.

    $ aws cloudformation list-stacks{  "StackSummaries": [      {          "StackId": "arn:aws:cloudformation:us-east-1:157470686136:stack/boundary-dynamic-hosts/05ea64a0-7bbf-11ec-9531-120baac28e1f",          "StackName": "boundary-dynamic-hosts",          "TemplateDescription": "AWS CloudFormation template for Boundary Dynamic Hosts tutorial. Deploying this template will incur costs to your AWS account.",          "CreationTime": "2022-01-22T20:08:11.060000+00:00",          "DeletionTime": "2022-01-22T21:22:37.577000+00:00",          "StackStatus": "DELETE_COMPLETE",          "DriftInformation": {              "StackDriftStatus": "NOT_CHECKED"          }      }  ]}

  2. Delete the boundary-keypair EC2 keypair.

    $ aws ec2 delete-key-pair --key-name boundary-keypair

  3. Delete the boundary user access key.

    Before deleting the boundary user all items attached to the user must be deleted. For more on deleting an IAM user, check the User Guide docs.

    First list the boundary user access keys.

    $ aws iam list-access-keys --user-name boundary{  "AccessKeyMetadata": [      {          "UserName": "boundary",          "AccessKeyId": "AKIASWVU2XLZLFLIDMVW",          "Status": "Active",          "CreateDate": "2022-01-19T02:26:11+00:00"      }  ]}

    Copy the AccessKeyId value (AKIASWVU2XLZLFLIDMVW in this example).

    Now delete the access key.

    $ aws iam delete-access-key --user-name boundary --access-key-id AKIASWVU2XLZLFLIDMVW

  4. Delete the BoundaryDescribeInstances policy.

    $ aws iam delete-user-policy --user-name boundary --policy-name BoundaryDescribeInstances

  5. Delete the boundary IAM user.

    $ aws iam delete-user --user-name boundary

  6. Stop Boundary

    Log in to the HCP portal and delete the HCP Boundary instance.

Next steps

This tutorial demonstrated the steps to set up a dynamic host catalog using the AWS host plugin. You deployed and tagged hosts within AWS, configured a plugin-type host catalog within Boundary, and created three host sets that filtered for the hosts based on their tag values.

To learn more about integrating Boundary with cloud providers like AWS and Azure, check out the OIDC Authentication tutorial.

Previous

Worker-aware targets

 Next

Dynamic host catalogs on Azure