Creating and Storing an EC2 SSH Key in Secrets Manager

ACM.81 Altering code that uses SSM Parameter Store to use AWS Secrets Manager

This is a continuation of my series of posts on Automating Cybersecurity Metrics.

I’ve been writing about creation of an EC2 SSH Key for use with an EC2 instance. I explained that I want to create the key in a manner that provides non-repudiation and only allows an authorized user to access their own key. Those concepts and the prior code we created to create and store the SSH Key in SSM Parameter store are covered in the prior posts.

Here’s the code including the shred command discussed in the post on when deleted files are not really deleted.

Now we’re going to alter this code to store the credentials in Secrets Manager instead. We’re going to add a KMS key and only allow developers to use the KMS key.

Developer Group and Role

Before we create a KMS key for our secret we need to create a Developer Group and Role. I currently have my developer user assigned to an “AppDeployment” group. However, whatever is deploying applications should not have access to developer SSH keys. I want a group specific to developers in this case. It’s easy to use our existing code to add a new group and add the Developer IAM User.

It’s also easy to add a new role via the deployment script.

We also need to add the role policy that function is expecting that allows the developers to access Secrets Manager and the Developer KMS key. In order to create the policy that allows developers to use the key, we first need to create the key.

A bit of refactoring before we create our KMS key

Next we need to create a KMS key for the developer group to use. Because our policies can restrict access with IAM on a per developer basis we don’t need a KMS key for each developer in this case, but we could do that if we had a high-security environment that needed additional security for extra protection.

Adding a new key should have been easy but somehow I forgot to check in some prior code so I fixed that. I should also mention for those following along from the beginning that I removed some error checking in the deploy.sh script for KMS keys that checks for an empty output value and moved it to the function where I get the output value from a stack.

Also, I have started to realize that the “service” name I give my CloudFormation stacks generally match my AWS CLI values. Eventually I will get rid “service” and just use “profile” instead.

The profile and service variables both exist in my shared function bash scripts now for each subdirectory. I can remove passing the profile into every function call. Little by little I’m streamlining the code where I can so there’s less of it.

Adding a new KMS key for the Developers Group

It’s easy to add a new key to our key deployment stack. Copy and modify the code for the key above it:

And add an alias by copying and altering one of our other aliases:

Update the Developer Role IAM Policy

Now we want our Developer Role to be able to use their own policy and developers need to be able to access a secret with a name matching their own.

So here’s what I wanted to do. I wanted to use the ${aws:username} parameter the way you can use it in an IAM change password policy like this:

I tried to add the aws:username variable to my policy like this:

It didn’t work. Why. What if the Sub is not properly handling the username as a variable? If I remove the Sub and added the resource like this:

“*:secret:${aws:username}”

Can’t do that either:

What if I just use * from the region and account id?

“arn:aws:secretsmanager:*:*:secret:${aws:username}”

CloudFormation deployed the above. I’d rather have a specific account number and region. Searching around I found this question on how to escape a ${} in a Sub function:

How to escape “${}” in cloudformations “Fn::Sub”

Let’s try it.

“arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:${!aws:username}”

Nope:

Well, our policy is not perfect but it’s pretty close. It would be nice if AWS would fix Sub to exclude the special value aws:username. #awswishlist.

But will it work? See subsequent posts for a working solution. For now let’s store our SSH Key in Secrets Manager.

Update SSH key creation function to use Secrets Manager

Last we want to change our deployment script to save the new SSH key to Secrets Manager instead of parameter store. Here’s the command we want to use:

create-secret – AWS CLI 2.0.33 Command Reference

While we are at it, we can eliminate storing the key to a file at all and transform it in memory and then immediately upload it to secrets manager. I had to fix a bug in the delete and re-create logic from the last post.

Of course we have to give the IAM administrator permissions to create the secrets and use the key.

Change the key policy to give the IAM permission to create (encrypt) the key.

Presuming you are using my updated code where I fixed a typo, the code should now deploy a secret and store the SSH key we cerated in Secrets Manager, encrypted with the specified KMS key.

However, as you will see in subsequent posts, although we got our code to deploy the secret and store the value, we have some issues with our policies. We need to make some adjustments to allow users to access their own secrets, and no one else’s secrets, while maintaining non-repudiation (meaning the IAM admin nor the KMS admin can view the secrets either.)

Follow for updates.

Teri Radichel

If you liked this story please clap and follow:

Medium: Teri Radichel or Email List: Teri Radichel
Twitter: @teriradichel or @2ndSightLab
Requests services via LinkedIn: Teri Radichel or IANS Research

© 2nd Sight Lab 2022

All the posts in this series:

Automating Cybersecurity Metrics (ACM)

____________________________________________

Author:

Cybersecurity for Executives in the Age of Cloud on Amazon

Need Cloud Security Training? 2nd Sight Lab Cloud Security Training

Is your cloud secure? Hire 2nd Sight Lab for a penetration test or security assessment.

Have a Cybersecurity or Cloud Security Question? Ask Teri Radichel by scheduling a call with IANS Research.

Cybersecurity & Cloud Security Resources by Teri Radichel: Cybersecurity and Cloud security classes, articles, white papers, presentations, and podcasts

Creating and Storing an EC2 SSH Key in Secrets Manager was originally published in Cloud Security on Medium, where people are continuing the conversation by highlighting and responding to this story.