Crypto 101: A Brief Tour of Practical Crypto in Golang

2017-12-14 Programming Rob Baruch

Crypto 101:

Golang offers a rich collection of packages supporting cryptographic operations. From a beginner’s perspective, maybe too many offerings! I offer up an overview of what’s available and an introduction to some practical uses of cryptography in Golang. Implementation details are always critical when discussing crypto. We’ll discuss some general implications of making poor choices and how such choices can completely undermine any uses of these tools.

What’ in the box?

The top-level crypto package is comprised of a little over a dozen sub-packages that offer:

AES encryption (also known as FIPS 197)
Block Ciphers
DES and TDEA (also known as FIPS 46-3)
Digital Signature Algorithms (DSA FIPS 186-3)
Hashed Message Authentication Code (HMAC FIPS-198)
Hashing: MD5 Hashing Algorithm (RFC 1321), SHA1 (RFC 3174), SHA256/SHA512 (FIPS 180-4)
RSA: RSA PKCS#1 encryption implementation
X509/TLS Certificate and Key based comms and general certificates/key manipulation

Lots of capabilities in the sub-packages, each one requiring a deep-dive to truly understand what functions are available and more importantly, how to properly use them.

Before we can do that though, we need to start off with a few definitions.

Ciphers and Keys

Cryptography is all about transforming plain-text into cipher-text - text that is hopefully inaccessible to anyone not in possession of a cipher key.

In simple terms. we can think of a cipher as a function that takes a clear-text message $m$ and a key, $k$ and transforms our message into a cipher text $c$

$$ f(m,k) = c$$ and the inverse function takes the cipher text, $c$ and key, $k$, and produces the plain-text message, $m$.

$$ f^{-1}(c,k) = m$$

More specifically, we refer to these types of ciphers as symmetric-key algorithms - meaning that the key, $k$, used to encrypt is the same $k$ key used to decrypt. Asymmetric Cryptographic Ciphers employ different keys to encrypt and decrypt. In golang, the crypto/rsa package implements such an encryption scheme. Moreover, in these asymmetric key systems, the two keys are referred as the public and private keys. We encrypt with the public key and decrypt with the private key.

Hashing

In cryptography, hashing refers to the mathematical functions that are one-way-functions which takes a message, $m$, and produces a digest, $d$ such that the inverse function can’t be found to restore $m$ from $d$. Here are the other properties of a well-designed hashing function:

deterministic - the same message always results in the same hash
computationally efficient to compute the hash value for any given message
infeasible to generate a message from its hash value except by brute-force
small changes to a message result in new hash value uncorrelated with the old hash value
difficult to find two different messages with the same hash value

Why are hashes important in cryptography? We can think of them as functions that produce authentications for messages. Digital signatures and message authentication codes MAC can be used to verify the integrity and authenticity of messages.

Let’s look at an example of a well-known hashing function, sha256. Straight out of the godocs, we see an example of how we can compute the sha256 hash on a file

package main

import (
	"crypto/sha256"
	"fmt"
	"io"
	"log"
	"os"
)

func main() {
	f, err := os.Open("file.txt")
	if err != nil {
		log.Fatal(err)
	}
	defer f.Close()

	h := sha256.New()
	if _, err := io.Copy(h, f); err != nil {
		log.Fatal(err)
	}

	fmt.Printf("%x", h.Sum(nil))
}

The code above reads in our file.txt, creates a new hash, h, and copies the file contents, f, into the hash, h. Once all of the data is copied into the hash we can compute the hash sum, h.Sum(nil)

Convince yourself that a small change to your input file file.txt produces a totally different hash.

For example, changing only one-bit results in these two different hashes:

4ef21708627697b11f0e8c67fa7a650d000f0fe515bddee5d702f6324d7c99f0

493131d459cfd0404fbdc74051bb5c17e16aac805247b6e0ff1beec645e4c7d5

We can use (with some caution) the hash to verify the integrity of our file so long as we can guarantee that the hash value itself cannot be modified.

Digital Signatures

Digital signatures take hashing a step further and solve the problem of guaranteeing that the hash isn’t modified by using asymmetric public/private key cryptography. Once we compute a hash, we can “sign” the hash with our private key, and then verify the integrity of the signature using our public key.

So, given the hash computation above, let’s look at an example of using a digital signature to ensure the integrity of the hash. Before we can work with digital signatures, we’ll nee to create an asymmetric key-pair.


// generate a 1024-bit private-key
priv, err := rsa.GenerateKey(rand.Reader, 1024)

// extract the public key from the private key
pub := priv.Public()

Now that we have keys, we can create a digital signature for a message using our keys:

message := []byte("message to be signed")

	hashed := sha256.Sum256(message)

	signature, err := rsa.SignPKCS1v15(rand.Reader, priv, crypto.SHA256, hashed[:])
	if err != nil {
		fmt.Printf("Error from signing: %s\n", err)
		return
    }

So now that we have a signature, we can verify the integrity of it by comparing the message with the signature computed from the private key, and verified with the signature and the public key:

    err = rsa.VerifyPKCS1v15(&priv.PublicKey, crypto.SHA256, hashed[:], signature)
	if err != nil {
		fmt.Printf("Error from verification: %s\n", err)
		return
	} else {
	    fmt.Printf("signature is verified\n")
    }

By using public/private keys and digital signatures, we can maintain the integrity of any message digest that is signed with our private key.

A note on hashing and digital signatures in practice. Why didn’t we just sign the file directly? Digital signatures require the messages that we sign be relatively small. Therefore, computing a hash of the underlying larger message allows us to sign the hash representing the larger file without any limitations on signing large messages.

We jumped ahead a little introducing public/private keys without using them for encryption and decryption. Next, let’s cycle back and look specifically at ciphers - both symmetric and asymmetric.

Encryption/Decryption - Symmetric Ciphers

Now that we’re steeped in the definitions and understanding of keys, hashes, and ciphers, let’s look at how we might encrypt and decrypt files using a symmetric key and cipher.

AES Encrypt/Decrypt

In order to encrypt or decrypt using a symmetric cipher like AES for example, we need to create a key. AES keys are either 16 or 32 bytes and are of type []byte slices.

package main

import (
        "crypto/aes"
        "crypto/cipher"
        "crypto/rand"
        "fmt"
        "io"
)

func main() {
        // The key argument should be the AES key, either 16 or 32 bytes
        // to select AES-128 or AES-256.
        key := []byte("0123456789ABCDEF")
        plaintext := []byte("Apple")

        block, err := aes.NewCipher(key)
        if err != nil {
                panic(err.Error())
        }

        nonce := make([]byte, 12)
        if false {
                if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
                        panic(err.Error())
                }
        }

        fmt.Printf("nonce: %x\n", nonce)

        aesgcm, err := cipher.NewGCM(block)
        if err != nil {
                panic(err.Error())
        }

        ciphertext := aesgcm.Seal(nil, nonce, plaintext, nil)
        fmt.Printf("cipher:%xn", ciphertext)
}

In the example above, we will use the Galois Counter Mode (GCM) 128-bit, block cipher with a standard nonce.

We create an AES key, block, from our key, and create a GCM block cipher, aesgcm from our aes key, block.

We then use the aesgcm cipher to encrypt using the aesgcm.Seal() method on our plaintext, resulting in our cipher-text.

What’s up with the nonce? In cryptography, a nonce is an arbitrary number that can only be used once. We create a random nonce as part of our crypto-system to harden our key. Notice that I left the nonce as a default value, namely 12 bytes of zero. In practice, a nonce should be used (and saved) so that our key is strengthened and decryption is further protected from key-attack cryptanalysis.

Decryption using the aes key and GCM block cipher works symmetrically as shown below for a given cipher-text encrypted above:

        ciphertext, _ := hex.DecodeString("08f24c28f0fc9aef5812a35ce66235bc2488d6c29b")

        nonce, _ := hex.DecodeString("000000000000000000000000")

        block, err := aes.NewCipher(key)
        if err != nil {
                panic(err.Error())
        }

        aesgcm, err := cipher.NewGCM(block)
        if err != nil {
                panic(err.Error())
        }

        plaintext, err := aesgcm.Open(nil, nonce, ciphertext, nil)
        if err != nil {
                panic(err.Error())
        }

Aside from decrypting, we see the use of the hex package. In addition to cryptography packages, we need to understand how to use the utility packages that aid in the encoding and decoding of cryptographic objects.

Encoding, Hex, and PEM

Notice the utility function above hex.DecodeString(). Given a string representing a hexadecimal array, we can parse and return a []byte value of the string. Similarly, we can encode a []byte value into a string using hex.EncodeString()

    src := []byte("feedme")
    encodedStr := hex.EncodeToString(src)

	fmt.Printf("%s\n", encodedStr)

In addition to hex encoding, we need to understand PEM encoding and decoding. PEM stands for Privacy Enhanced Mail. What does email have to do with crypto? Well PEM was an early implementation of public/private crypto used to ensure the integrity of email between two parties. The names stuck as the standard used to encode public and private key information became known as PEM-encoding.

Let’s look at how we might encode and decode our rsa keys we generated in our digital signature examples above. We’re going to use both the encoding/pem and crypto/x509 packages to do the encoding.

func ExportRsaPrivateKeyAsPemStr(privkey *rsa.PrivateKey) string {
    privkey_bytes := x509.MarshalPKCS1PrivateKey(privkey)
    privkey_pem := pem.EncodeToMemory(
            &pem.Block{
                    Type:  "RSA PRIVATE KEY",
                    Bytes: privkey_bytes,
            },
    )
    return string(privkey_pem)
}

Turns out that there’s another level of encoding that is occurring here. The Marshal function turns our rsa private key data-structure into an ASN.1 DER encoded []byte.

Once in this format, we can encode it into PEM. Simply adding the following to our example above will give us a PEM-string:

    pem_priv := ExportRsaPrivateKeyAsPemStr(priv)
	fmt.Printf("%s\n", pem_priv)
}

Our private key, pem-encoded is below:

We can save this to a file, but a word of caution. We need to protect this file! Anyone reading the file can PEM-decode it and reproduce our key. There is no password on the key!

We need to decode our pem-key in order to turn it back into an rsa private key data structure. We can do that as follows:

func ParseRsaPrivateKeyFromPemStr(privPEM string) (*rsa.PrivateKey, error) {
	block, _ := pem.Decode([]byte(privPEM))
	if block == nil {
		return nil, errors.New("failed to parse PEM block containing the key")
	}

	priv, err := x509.ParsePKCS1PrivateKey(block.Bytes)
	if err != nil {
		return nil, err
	}

	return priv, nil
}

We pass in our PEM-string, decode the PEM-block and then Parse the block using the x509.ParsePKCS1PrivateKey() function, returning an error or a private key if the PEM-block was valid.

We can create similar functions to encode and parse public keys:

Export a public key to a string:

func ExportRsaPublicKeyAsPemStr(pubkey *rsa.PublicKey) (string, error) {
    pubkey_bytes, err := x509.MarshalPKIXPublicKey(pubkey)
    if err != nil {
            return "", err
    }
    pubkey_pem := pem.EncodeToMemory(
            &pem.Block{
                    Type:  "RSA PUBLIC KEY",
                    Bytes: pubkey_bytes,
            },
    )

    return string(pubkey_pem), nil
}

Parse a public key (string) into a public-key data-structure

func ParseRsaPublicKeyFromPemStr(pubPEM string) (*rsa.PublicKey, error) {
    block, _ := pem.Decode([]byte(pubPEM))
    if block == nil {
            return nil, errors.New("failed to parse PEM block containing the key")
    }

    pub, err := x509.ParsePKIXPublicKey(block.Bytes)
    if err != nil {
            return nil, err
    }

    switch pub := pub.(type) {
    case *rsa.PublicKey:
            return pub, nil
    default:
            break // fall through
    }
    return nil, errors.New("Key type is not RSA")
}

A note about the ParseRsaPublicKeyFromPemStr() function above. Keep in mind that Public and Private Keys are interfaces in golang. Therefore we need to inspect the type-value of the key and solely return the rsa-typed key. The x509.ParsePKIXPublicKey() function can parse any type of Public key that implements the interface. We don’t want to return a dsa key for example, if the user is expecting an rsa key.

Public Private Key Encryption and Decryption

Now we can put it all together and use everything we’ve learned to encrypt and decrypt messages using our public private keys.

Using our keys generated above, we can generate cipher text as follows:

    ciphertext, err := rsa.EncryptOAEP(sha256.New(), rand.Reader, &priv.PublicKey, message, []byte("robs messages"))
	if err != nil {
		fmt.Printf("Error from encryption: %s\n", err)
		return
	}

	fmt.Printf("%x", ciphertext)

And we can decrypt our cipher-text using our public key:


	plaintext, err := rsa.DecryptOAEP(sha256.New(), rand.Reader, priv, ciphertext, []byte("robs messages"))
	if err != nil {
		fmt.Printf("Error from decryption: %s\n", err)
		return
	}

	fmt.Printf("\nPlaintext: %s\n", string(plaintext))

Final notes on implementation

At onset, we mentioned that implementation is the gotcha in cryptographic systems. Let’s look at a simple example of how we might totally compromise our system by making a poor implementation decision.

Securing our private key

In virtually all of our examples above, we end up with data structures that one way or another need to be protected. A private key stored in a file, a nonce, a password are all things that may leak unexpectedly resulting in a breach of our security expectation.

Let’s focus on the very basic and real use-case of interacting with an encryption password for example.

In simple terms, we need to convey the password to our program as it’s a parameter to an encrypt or decrypt function. But how do we securely do this.

Perhaps we can make this a flag an pass it as a parameter to our function. Swell! Problem solved! Or is it… What happens when a savvy user issues a ps(1) command and sees the executing process with the command line arguments passed to our program? Oops! there went our password!

Or we could write it to the filesystem, but here again we risk leaking our private information if we don’t properly secure the file. or worse yet, the operating system that secures the user and group account information on the system.

What are we left with? Well, one of the most secure solutions is to use a raw terminal input process to read a non-echoing password directly from the user. Golang provides such a facility in the package golang.org/x/crypto/ssh/terminal. Below is an example of how we might use this package to securely read our password directly from a user.

        oldState, err := terminal.MakeRaw(0)
        if err != nil {
                panic(err)
        }
        defer terminal.Restore(0, oldState)

        t := terminal.NewTerminal(os.Stdin, "> ")
        pw, err := t.ReadPassword(">> ")

        if err != nil {
                fmt.Printf("failed to read password: %s\n", err)
        } else {
                fmt.Printf("secret is %s\r\n", pw)
        }

We create a raw terminal and store the previous state for recovery when we are finished (executed with our defer() function). And then we use our terminal and the ReadPassword() method to securely read in a password.

Even though we have gone to great measures to seemingly mitigate security breaches, we may still be vulnerable. Never take security implementations for granted. In fact, just recently a vulnerability with the golang bignum math library proved to be a vulnerability for cryptographic implementations.

I encourage you to play around and experiment with all of the cool code in the packages referenced above! There’s a lot in there and much to learn!!! – Rob

Acknowledgements and Attributions:

The code to encode/decode keys was written by David W and posted on stack-overflow.