How to commit encrypted files to Git with Mozilla SOPS

devops Aug 22, 2021

It's a common scenario where it's needed to share some keys/secrets with other members of the project team and it's also common that those keys are closely attached to some source code. SOPS allows you to do that securely using Git or other VCS.

A Git repository would be a perfect solution, but Git works in plain text, like any other VCS (Version Control Service) for source code, even when the communication channels and storage system are encrypted. That means, if someone has access to the source code, it'll also have access to all keys inside of the code.

I think the best way to learn the basics is to see this asciinema record. The entire post is only if you want to know what you're doing (which should also be important):

If you prefer to watch videos, I recommend you to skip this post and go to the video from the original SOPS creator:

How the encryption process works

SOPS is an editor of encrypted files that supports YAML, JSON, ENV, INI, and BINARY formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP.
https://github.com/mozilla/sops

We'll use SOPS in the following example with age (github.com/FiloSottile/age) as the encryption solution. SOPS will be our editor or interface to encrypt and decrypt files using age.

The following diagram represents a flow of how users can share encrypted files with a git repository:

Requirements
- A secure vault. I'm using the 1password for this example.
  This vault will be used to store the "age" key.
- Both users must have access to the same secrets.
Steps
- User 1 pushing an encrypted file
  1. Get "age" key from 1password vault
    This is a plain text key.
  2. Encrypt files using SOPS
  3. Commit files to your local repository
  4. Push commit to the remote git repository (like GitHub, GitLab, etc)
- User 2 pull and decrypt the previously file.
  1. Pull commit from the remote git repository
    Also, get the same "age" key from 1password vault used by "User 1"
  2. Decrypt file locally using SOPS

the flow of how to use **sops** + **git**

How to encrypt files using age

age is a simple, modern, and secure file encryption tool, format, and Go library:
https://github.com/FiloSottile/age

You can use other encryption methods that could be even better for your use-case, for example, the AWS KMS if you already have an AWS account. That also allows you to use the AWS IAM to manage access to the encryption keys. SOPS is the perfect solution only when you have no other encryption method.

First, download the age binary for your system and add it to your path:

# Download the age binary
# the '/usr/local/dir' must be in your PATH env
# 
# I recommend you to check the GitHub repository for the latest version:
# https://github.com/FiloSottile/age/releases
wget -O /tmp/age.tar.gz \
    https://github.com/FiloSottile/age/releases/download/v1.0.0-rc.3/age-v1.0.0-rc.3-linux-amd64.tar.gz
sudo tar -zxv --strip 1 \
    -C /usr/local/bin/ \
    -f /tmp/age.tar.gz \
    age/age age/age-keygen

download the age binary

Now you have the tool to generate your encryption key:

# Create an encryption key with age

# go to a directory to store your key
# I'll consider the '~/project/keys' dir
mkdir -p ~/project/keys
cd ~/project/keys
age-keygen -o curitiba-key.txt

# Now check your key
cat ./curitiba-key.txt
# Should be something like this
# > # created: 2021-08-22T04:17:02-03:00
# > # public key: age1huu8lrhxqg9edurj2u2srd8ty3rz5ql0dh3e9j6gn3nuhu5elqesvu06x0
# > AGE-SECRET-KEY-1QU95037P9W9AVDNYLCUVEXANZC4H4KMRQMR94VAR4GZFN2PKW3ES437LJS

create an encryption key with age

It's not the scope of this tutorial, but I'll guide you in this simple example of how to encrypt any binary file using exclusively age. Those encrypted files can be committed to a git repository too:

# Encrypt a binary file using age only

# Check your key pair
# public key in on the comment
# private key starts with 'AGE-SECRET-KEY-'
cat ./curitiba-key.txt

# Copy 'top' binary as an example binary
cp $(whereis top | cut -d' ' -f 2) ./

# Encrypt the 'top' binary file
# using your public key
# the 'top.age' file will be created
cat ./top | age -r age1huu8lrhxqg9edurj2u2srd8ty3rz5ql0dh3e9j6gn3nuhu5elqesvu06x0 > top.age

# Check the files
ls -lah
# > total 268
# > drwxr-xr-x 2 luiz luiz 4.0K Aug 22 04:44 .
# > drwxr-xr-x 3 luiz luiz 4.0K Aug 22 04:16 ..
# > -rw------- 1 luiz luiz  189 Aug 22 04:17 curitiba-key.txt
# > -rwxr-xr-x 1 luiz luiz 127K Aug 22 04:44 top
# > -rw-r--r-- 1 luiz luiz 127K Aug 22 04:44 top.age

# Decrypt using your age key file
age --decrypt -i ./curitiba-key.txt top.age > top-new

# Test if the decryption worked
chmod +x ./top-new
./top-new
# [Ctrl]+C to exit

# Clean up
rm top
rm top.age
rm top-new

encrypt a binary file using age only

How to edit yaml files using SOPS

The first will be to install SOPS just like we did with age:

# Download the SOPS binary
# the '/usr/local/dir' must be in your PATH env
# 
# I recommend you to check the GitHub repository for the latest version:
# https://github.com/mozilla/sops/releases
sudo wget -O /usr/local/bin/sops \
	https://github.com/mozilla/sops/releases/download/v3.7.1/sops-v3.7.1.linux
chmod +x /usr/local/bin/sops

install sops

Now that you have an age encryption key, let's finally start doing the main part of this post and learn how to create text files encrypted using SOPS.

# Load your SOPS env vars
# each terminal session you require you
# to have those environment variables.

# Check your key pair
# public key in on the comment
# private key starts with 'AGE-SECRET-KEY-'
cat ~/project/keys/curitiba-key.txt

# Set variables
export SOPS_AGE_RECIPIENTS=age1huu8lrhxqg9edurj2u2srd8ty3rz5ql0dh3e9j6gn3nuhu5elqesvu06x0
export SOPS_AGE_KEY_FILE=~/project/keys/curitiba-key.txt

load your SOPS env vars

# Create an encrypted file for our project
mkdir -p ~/project/curitiba
cd ~/project/curitiba

# Create a fresh new file
# Set some content
sops config.yaml

# Check the encrypted content
cat ./config.yaml

# Edit the encrypted file content
sops config.yaml

create an encrypted file for our project

Set the following file content:

---
games:
    rpg:
        - Yakuza 0
        - Final Fantasy XV
    racing:
        - Forza Horizon 5

config.yaml (non-encrypted)

Save and exit. Now check the file content again using cat ./config.yaml:

games:
    rpg:
        - ENC[AES256_GCM,data:Vs2CbFo4f2s=,iv:dN2YG3WjfKmjx4X3FCTw3LKh4DdekpYfvG+1VYei6r4=,tag:CwCLz77Mz2lRDmMWXM2joA==,type:str]
        - ENC[AES256_GCM,data:qPLxtmCBZG+Pj/63aCa8Jw==,iv:ApLVfEyY/uEKL90ukFb3SNXsvmr3OArQ2dJhAy75gRE=,tag:pPHK0Q4pqBJp+HQLdUlZfA==,type:str]
    racing:
        - ENC[AES256_GCM,data:4Ck75WqIOvJZTNEn9eUy,iv:bgC66CZ5/bPuDCEkUWQZvnEnLBQwLnTtekO1YOlebDU=,tag:ZMU2kKXxChCnVpdN44UFzA==,type:str]
sops:
    kms: []
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    age:
        - recipient: age1huu8lrhxqg9edurj2u2srd8ty3rz5ql0dh3e9j6gn3nuhu5elqesvu06x0
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB2RFMveDQxTnNDV2lWaCs0
            bng1dUxTY2U0dWg2anFjaHk3Nkl5SWtVd3lVCnRPbmtvaFpqY0pHMnBmS3RFYmJr
            MXhTNnBsMWdvYUpOMCtnSW9uNTVPVlUKLS0tIDRnVE5KeGwyc0FPQjRzZU0zQnhr
            bmZpQmZsbG0za3hmNkhTNFE1ajRycGsK/QWlXTaYAeStA3LvLpA1tjOt/OL80hp3
            e8GOmcIpPMMx5mqio7B6vYAcxo+WitdvPluGoI1XpgvAKKzIAZ11Mw==
            -----END AGE ENCRYPTED FILE-----
    lastmodified: "2021-08-22T08:31:00Z"
    mac: ENC[AES256_GCM,data:xCsj1HeF0q2QSR2JLTxDGGyM4GWvRTGnTqaCHA7B8amDcVtdcsovJzdtxmrke8gSVqHXI2F55ERdRtoNeFBPRAucuvWka6vEkGSeLrpjUBv34XJFiG3I2FxEE+0wF2lF4fKbcJ1LM2MmbY/qPxfm9ZAkl00UXQeSaUl3GJtDXOY=,iv:Gda7u+81MS+1c9HqAerVdfYApgxYyQB066rAYjRewqc=,tag:iASkpTNwGWjV93gsjtT6yw==,type:str]
    pgp: []
    unencrypted_suffix: _unencrypted
    version: 3.7.1

config.yaml (encrypted)

Perfect! :) now you know how to create and edit.

[extra] How to limit which var should be encrypted WIP

creation_rules:
        - path_regex: \.dev\.yaml$
          hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/secondkey"
        - path_regex: \.prod\.yaml$
          hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/thirdkey"

.sops.yaml