Using Pulumi with AWS Secrets Manager

While building Pyrra’s new infrastructure I needed to figure out how to handle secrets. Pulumi has the ability to store encrypted secrets in config files using pulumi config set --secret [name] [value] but this doesn’t solve the problem of where does [value] come from in the first place. And it doesn’t help a developer find out what a secret is if they need, for example, a password to connect directly to a database from their laptop.

As we’re using AWS the obvious place to canonically store secrets is in Secrets Manager. This way they can be retrieved (with an audit trail) by a human, or programmatically with Pulumi’s aws.secretsmanager.secret or using AWS SDKs. This still doesn’t solve the problem of generating secrets, but it’s a good place to store them. Though using AWS SDKs to retrieve secrets means modifying application code that often expects secrets via environment variables (for better or for worse), or creating some kind of shim to create those env vars.

With some guidance from Pulumi’s Slack channel I settled on the following pattern

Generate secrets (eg passwords) in my Pulumi code using pulumi.random

import * as random from "@pulumi/random";
const password = new random.RandomPassword(fullName, {
    length: 12
});

Set them in AWS Secrets Manager. Note that you need both a Secret and a SecretVersion, because the value of a secret can change due to eg a rotation policy.

  // Specify 'name' so that Pulumi doesn't autoname, which
  // would be counterproductive if anything that's not Pulumi
  // ever retrieves a secret.
  const secret = new aws.secretsmanager.Secret(
    secretName,
    { name: fullName },
  );
  const secretVersion = new aws.secretsmanager.SecretVersion(fullName, {
    secretId: secret.id,
    secretString: password.result, // Note you need .result to get the actual value
  });

Also use them when setting up a service that requires a password, eg RDS

const postgres = new aws.rds.Instance("postgresdb", {
  // ...
  password: password.result,
});

And then finally export the secret so that another project / stack can reference it, if needed. I have two projects, to separate application config from infrastructure config. I create my database password in infra and use it in apps to create a connection string for my applications.

// in 'infra' project
export const RDSAdminPassword = password.result

In my apps stack I can use the exported secret

// in 'apps' project
const infra = new pulumi.StackReference(`pyrra/infra/${env}`);
const dbConnectionString = pulumi.interpolate`postgresql://username:${infra.getOutput("RDSAdminPassword")}@postgres.pyrra.dev.int`

Craig Glennie

Using Pulumi with AWS Secrets Manager