AWS Secrets Manager provides a way for you to store and retrieve secrets securely. They provide a really nice tutorial to help you get started.

Looking to test and integrate this from the command line, I wanted to see how this extraction works and how it would look like if an application or wrapper was using the secret.

Here is how the secret looks like from the console

When using the aws secretsmanager CLI command here is how it looks like:

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ
{
    "ARN": "arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ",
    "Name": "TestSecret",
    "VersionId": "faa157d9-7432-4c4a-a5eb-38f00adf0d6c",
    "SecretString": "{\"Value1\":\"This IS A Secret\"}",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": 1559164502.537
}

So now I want to use the handy dandy tool jq to extract values from the json.

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ | jq .SecretString

"{\"Value1\":\"This IS A Secret\"}"

So here was the crux of the issue… we have double escaped values here, not your typical json. This is where I wasted a ton of time!

I found a bunch of jq blogs and posts like this, and this, and this. All of them are awesome, but until I focused on the double escaped values, I found this post. Thanks 0day for listening to my frustrations and the pointing that out!

Using all of the above, here is my one-liner:

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ | jq -c '.SecretString | fromjson' | jq .Value1

"This IS A Secret"

The key here is not the -c option, which is basically optional, but actually the fromjson function from jq! I don’t see this function in --help but of course it’s the man page:

Convert to/from JSON
       The tojson and fromjson builtins dump values as JSON texts or parse JSON texts into values, respectively. The tojson builtin differs from tostring in that tostring returns strings unmod‐
       ified, while tojson encodes strings as JSON strings.

           jq ´[.[]|tostring]´
              [1, "foo", ["foo"]]
           => ["1","foo","[\"foo\"]"]

           jq ´[.[]|tojson]´
              [1, "foo", ["foo"]]
           => ["1","\"foo\"","[\"foo\"]"]

           jq ´[.[]|tojson|fromjson]´
              [1, "foo", ["foo"]]
           => [1,"foo",["foo"]]

Which then takes me to the tojson field, seems like that could be useful in the future.

So in the end we’ll take out the -c and use -r to give us the raw text:

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ | jq '.SecretString | fromjson' | jq -r .Value1

This IS A Secret

Hope you you find this as useful as I did. Next steps for this would be to build a wrapper using boto in the attempt to retrofit an application that is using flat files, for example, for storing its secrets. 🙁

If this resonates with you, drop me a note, I would love to hear from you.

About the Author Ayman Elsawah

Cloud Security | Author | Educator | Coffee Nerd

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.